From tfheen at varnish-software.com  Tue Jul  2 07:36:12 2013
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Tue, 2 Jul 2013 09:36:12 +0200
Subject: Reminder to change the default Varnish version in trac bugs from
 "trunk" to "unknown"
In-Reply-To: <CANTn4cqReoRK_1DJWLAzr2umdeNiQ3jj46uK4bWNCKVUw3819Q@mail.gmail.com>
References: <CANTn4cqReoRK_1DJWLAzr2umdeNiQ3jj46uK4bWNCKVUw3819Q@mail.gmail.com>
Message-ID: <20130702073612.GC15457@err.no>

]] Martin Blix Grydeland

> Today, the default Varnish version attribute on new trac tickets is
> "trunk". Most tickets seen are 3.0 specific issues, but they are still
> reported on "trunk" because people forget to set the field. We should maybe
> have an "unknown" value by default to make it clear when the reporter
> hasn't changed the field.

I've added an "unknown" value to the list and made it the default now.
Sorry about not getting around to this earlier.

-- 
Tollef Fog Heen
Technical lead | Varnish Software AS
t: +47 21 98 92 64
We Make Websites Fly!



From phk at phk.freebsd.dk  Wed Jul  3 12:39:55 2013
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 03 Jul 2013 12:39:55 +0000
Subject: Synthetic objects
In-Reply-To: <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw@mail.gmail.com>
References: <40023.1372328938@critter.freebsd.dk>
 <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw@mail.gmail.com>
Message-ID: <45796.1372855195@critter.freebsd.dk>

In message <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw at mail.gmail.com>
, "Rogier 'DocWilco' Mulhuijzen" writes:

>If we're changing things, it would be useful to have something in synth
>other than a 3 digit status code.

We came up with an idea for this in the IRC channel:

We'll allow any number of digits (up to 32bits) but only the last
three will be used for the HTTP status code.

So you could for instance do
	return (synth(10001404));

and that would become a 404, but you get extra bits for your
own purposes.

Also, please notice that you have bereq to look at.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From dridi.boukelmoune at zenika.com  Wed Jul  3 13:38:29 2013
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Wed, 3 Jul 2013 15:38:29 +0200
Subject: Synthetic objects
In-Reply-To: <45796.1372855195@critter.freebsd.dk>
References: <40023.1372328938@critter.freebsd.dk>
 <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw@mail.gmail.com>
 <45796.1372855195@critter.freebsd.dk>
Message-ID: <CABtDKm6zNsQE2Pb=nvJErKxYqwSwvkCo6vhjwGzH8OHZtEDABQ@mail.gmail.com>

Hi,

I'm not sure I understand the purpose, is it for a separations of:
- true errors (from backends/varnish or hand made)
- on-the-fly static responses (redirects, static files...)

In this case it could be interesting to jump to vcl_backend_synth from vcl_recv:
if (req.url ~ "...") {
  return(synth(42302));
}

The syntax looks inconsistent, I'd rather have something like:
return(transition [, args])

Examples:
return(lookup);
return(synth, 10001404);
return(error, 10001503);
return(error, 10001503, "message");

It would also help to have constants for magic status codes:
===
// outside of sub functions
const BACKEND_DOWN = 503;
const MAINTENANCE = 1503;

sub vcl_backend_response {
  if (beresp looks maintenance-ish) {
    return(error, MAINTENANCE);
  }
}

sub vcl_error {
  if (obj.status == BACKEND_DOWN) {
    synthetic "dammit";
  }
  if (obj.status == MAINTENANCE) {
    synthetic std.fileread("maintenance.html");
  }
}
===

About Varnish turning into a web server, there's nothing to worry
about with just that, synthetic+std.fileread can't handle most
binary files.

Best Regards,
Dridi

On Wed, Jul 3, 2013 at 2:39 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw at mail.gmail.com>
> , "Rogier 'DocWilco' Mulhuijzen" writes:
>
>>If we're changing things, it would be useful to have something in synth
>>other than a 3 digit status code.
>
> We came up with an idea for this in the IRC channel:
>
> We'll allow any number of digits (up to 32bits) but only the last
> three will be used for the HTTP status code.
>
> So you could for instance do
>         return (synth(10001404));
>
> and that would become a 404, but you get extra bits for your
> own purposes.
>
> Also, please notice that you have bereq to look at.
>
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev



From phk at phk.freebsd.dk  Wed Jul  3 15:14:17 2013
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 03 Jul 2013 15:14:17 +0000
Subject: Synthetic objects
In-Reply-To: <CABtDKm6zNsQE2Pb=nvJErKxYqwSwvkCo6vhjwGzH8OHZtEDABQ@mail.gmail.com>
References: <40023.1372328938@critter.freebsd.dk>
 <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw@mail.gmail.com>
 <45796.1372855195@critter.freebsd.dk>
 <CABtDKm6zNsQE2Pb=nvJErKxYqwSwvkCo6vhjwGzH8OHZtEDABQ@mail.gmail.com>
Message-ID: <51836.1372864457@critter.freebsd.dk>

In message <CABtDKm6zNsQE2Pb=nvJErKxYqwSwvkCo6vhjwGzH8OHZtEDABQ at mail.gmail.com>
, Dridi Boukelmoune writes:

>I'm not sure I understand the purpose, is it for a separations of:
>- true errors (from backends/varnish or hand made)
>- on-the-fly static responses (redirects, static files...)

Until now vcl_error{} has lived in sort of a no-mans-land between
the client side and the backend side of varnish.

What I'm proposing to do in VCL4 is to split it into
	vcl_error{} = client side
and
	vcl_backend_synth{} = backend side.

So from vcl_recv{} you'd go to vcl_error{} to deliver a one-off
synthetic response to a single client.  That object cannot and will
not be cached.

vcl_backend_synth{} on the other hand, will allow you to create objects
which _can_ be cached, although by default the will probably not be.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From varnish at bsdchicks.com  Wed Jul  3 15:32:02 2013
From: varnish at bsdchicks.com (Rogier 'DocWilco' Mulhuijzen)
Date: Wed, 3 Jul 2013 08:32:02 -0700
Subject: Synthetic objects
In-Reply-To: <51977.1372865208@critter.freebsd.dk>
References: <40023.1372328938@critter.freebsd.dk>
 <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw@mail.gmail.com>
 <45796.1372855195@critter.freebsd.dk>
 <CABtDKm6zNsQE2Pb=nvJErKxYqwSwvkCo6vhjwGzH8OHZtEDABQ@mail.gmail.com>
 <51836.1372864457@critter.freebsd.dk>
 <CANTTouUPmhbMi4eqq77g-UOFKADagYqxNmdmTVsOTrvsHEp0QQ@mail.gmail.com>
 <51977.1372865208@critter.freebsd.dk>
Message-ID: <CANTTouX+q9PT-vnNQYpDJwm=6YpsnJLAQ8FD5ZgWAnsMfUy4hw@mail.gmail.com>

(Oops, think I forgot to hit reply all on my previous email)

In addition to do_esi, what about do_gzip. Especially with ESI in the mix,
compression would be nice to have. But even without ESI, things can get
pretty big. Having synthetics of 100K is not unheard of, with inline images
and such.
On Jul 3, 2013 8:26 AM, "Poul-Henning Kamp" <phk at phk.freebsd.dk> wrote:

> In message <
> CANTTouUPmhbMi4eqq77g-UOFKADagYqxNmdmTVsOTrvsHEp0QQ at mail.gmail.com>
> , "Rogier 'DocWilco' Mulhuijzen" writes:
>
> >Can the backend synthetics get the esi flag set? It would be useful to be
> >able to generate synthetic pages with esi includes in them.
>
> Ohh...  Neat idea!
>
> I won't promise this in the first iteration, but we should certainy not
> forget that idea...
>
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130703/7068c1b9/attachment.html>

From phk at phk.freebsd.dk  Wed Jul  3 15:49:35 2013
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 03 Jul 2013 15:49:35 +0000
Subject: Synthetic objects
In-Reply-To: <CANTTouX+q9PT-vnNQYpDJwm=6YpsnJLAQ8FD5ZgWAnsMfUy4hw@mail.gmail.com>
References: <40023.1372328938@critter.freebsd.dk>
 <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw@mail.gmail.com>
 <45796.1372855195@critter.freebsd.dk>
 <CABtDKm6zNsQE2Pb=nvJErKxYqwSwvkCo6vhjwGzH8OHZtEDABQ@mail.gmail.com>
 <51836.1372864457@critter.freebsd.dk>
 <CANTTouUPmhbMi4eqq77g-UOFKADagYqxNmdmTVsOTrvsHEp0QQ@mail.gmail.com>
 <51977.1372865208@critter.freebsd.dk>
 <CANTTouX+q9PT-vnNQYpDJwm=6YpsnJLAQ8FD5ZgWAnsMfUy4hw@mail.gmail.com>
Message-ID: <52377.1372866575@critter.freebsd.dk>

In message <CANTTouX+q9PT-vnNQYpDJwm=6YpsnJLAQ8FD5ZgWAnsMfUy4hw at mail.gmail.com>
, "Rogier 'DocWilco' Mulhuijzen" writes:

>In addition to do_esi, what about do_gzip. Especially with ESI in the mix,
>compression would be nice to have. But even without ESI, things can get
>pretty big. Having synthetics of 100K is not unheard of, with inline images
>and such.

Hand the devil a finger... :-)

I'll add it to the list

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From varnish at bsdchicks.com  Wed Jul  3 15:50:40 2013
From: varnish at bsdchicks.com (Rogier 'DocWilco' Mulhuijzen)
Date: Wed, 3 Jul 2013 08:50:40 -0700
Subject: Synthetic objects
In-Reply-To: <52377.1372866575@critter.freebsd.dk>
References: <40023.1372328938@critter.freebsd.dk>
 <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw@mail.gmail.com>
 <45796.1372855195@critter.freebsd.dk>
 <CABtDKm6zNsQE2Pb=nvJErKxYqwSwvkCo6vhjwGzH8OHZtEDABQ@mail.gmail.com>
 <51836.1372864457@critter.freebsd.dk>
 <CANTTouUPmhbMi4eqq77g-UOFKADagYqxNmdmTVsOTrvsHEp0QQ@mail.gmail.com>
 <51977.1372865208@critter.freebsd.dk>
 <CANTTouX+q9PT-vnNQYpDJwm=6YpsnJLAQ8FD5ZgWAnsMfUy4hw@mail.gmail.com>
 <52377.1372866575@critter.freebsd.dk>
Message-ID: <CANTTouWKV_0zaPZgj9b9bS2jyLNjYPrp6_f2Sj=MyxSMfEOtGg@mail.gmail.com>

I prefer daemon over devil...
On Jul 3, 2013 8:49 AM, "Poul-Henning Kamp" <phk at phk.freebsd.dk> wrote:

> In message <CANTTouX+q9PT-vnNQYpDJwm=
> 6YpsnJLAQ8FD5ZgWAnsMfUy4hw at mail.gmail.com>
> , "Rogier 'DocWilco' Mulhuijzen" writes:
>
> >In addition to do_esi, what about do_gzip. Especially with ESI in the mix,
> >compression would be nice to have. But even without ESI, things can get
> >pretty big. Having synthetics of 100K is not unheard of, with inline
> images
> >and such.
>
> Hand the devil a finger... :-)
>
> I'll add it to the list
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130703/b5256056/attachment.html>

From fgsch at lodoss.net  Thu Jul  4 15:19:43 2013
From: fgsch at lodoss.net (Federico Schwindt)
Date: Thu, 4 Jul 2013 16:19:43 +0100
Subject: [PATCH] Update graph
Message-ID: <CAJV_h0Y7Gx1khvFptz_tsf=tSRO5XTTJzegp=Bts76_YG-Hbaw@mail.gmail.com>

As mentioned on the #varnish irc channel, the diff below updates the graph
so it's clearer that vcl_recv() goes to vcl_hash() on lookup, pass and pipe.

Thanks,

f.-

diff --git a/bin/varnishd/cache_center.c b/bin/varnishd/cache_center.c
index 599d0e7..d11d600 100644
--- a/bin/varnishd/cache_center.c
+++ b/bin/varnishd/cache_center.c
@@ -1077,7 +1077,9 @@ DOT       lookup2 [
 DOT            shape=diamond
 DOT            label="obj.f.pass ?"
 DOT    ]
-DOT    hash -> lookup [label="hash",style=bold,color=green]
+DOT    hash -> lookup [style=bold,color=green]
+DOT    hash -> pipe [style=bold,color=orange]
+DOT    hash -> pass2 [style=bold,color=red]
 DOT    lookup -> lookup2 [label="yes",style=bold,color=green]
 DOT }
 DOT lookup2 -> hit [label="no", style=bold,color=green]
@@ -1364,11 +1366,11 @@ DOT             label="vcl_recv()|req."
 DOT    ]
 DOT }
 DOT RESTART -> recv
-DOT recv -> pipe [label="pipe",style=bold,color=orange]
-DOT recv -> pass2 [label="pass",style=bold,color=red]
+DOT recv -> hash [label="lookup",style=bold,color=green]
+DOT recv -> hash [label="pass",style=bold,color=red]
+DOT recv -> hash [label="pipe",style=bold,color=orange]
 DOT recv -> err_recv [label="error"]
 DOT err_recv [label="ERROR",shape=plaintext]
-DOT recv -> hash [label="lookup",style=bold,color=green]
  */

 static int
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130704/fab6de6e/attachment.html>

From perbu at varnish-software.com  Thu Jul 18 10:52:23 2013
From: perbu at varnish-software.com (Per Buer)
Date: Thu, 18 Jul 2013 12:52:23 +0200
Subject: Adding certain vmods to the distribution
Message-ID: <CAOXZevAs4CeAwzwdp6YOkaMoP+Rh8i2bVFzpH+H4rxT1GTswRQ@mail.gmail.com>

Hi.

There are certain vmods that are really very useful. I think it makes sense
to add some of these to the Varnish distribution so they will be readily
available for all our users.

In particular I'm thinking about the variable (variables in vcl), boltsort
(querystring argument sorting) and cookie (cookie manipulation) vmods.

What do you think?
-- 
 <http://www.varnish-software.com/> *Per Buer*
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130718/057ef4d2/attachment.html>

From magnus at hagander.net  Thu Jul 18 11:06:33 2013
From: magnus at hagander.net (Magnus Hagander)
Date: Thu, 18 Jul 2013 12:06:33 +0100
Subject: Adding certain vmods to the distribution
In-Reply-To: <CAOXZevAs4CeAwzwdp6YOkaMoP+Rh8i2bVFzpH+H4rxT1GTswRQ@mail.gmail.com>
References: <CAOXZevAs4CeAwzwdp6YOkaMoP+Rh8i2bVFzpH+H4rxT1GTswRQ@mail.gmail.com>
Message-ID: <CABUevEw_HyuaKnhjSZV=CpihToqWktLOPgUJv4S2umy5TaFDhA@mail.gmail.com>

On Thu, Jul 18, 2013 at 11:52 AM, Per Buer <perbu at varnish-software.com>wrote:

> Hi.
>
> There are certain vmods that are really very useful. I think it makes
> sense to add some of these to the Varnish distribution so they will be
> readily available for all our users.
>
> In particular I'm thinking about the variable (variables in vcl), boltsort
> (querystring argument sorting) and cookie (cookie manipulation) vmods.
>
> What do you think?
>

+1 - or more if I get more votes :)

In particular, getting it included in the RPMs and DEBs will make a big
difference. I don't particularly care if they're in the source tarball, but
the fact that it's quite a PITA to get them installed if your starting
point is RPM or DEB is a real bummer today.

If they were either to be in the varnish package, or just there were an
extra package like varnish-modules or something like that available in the
same repositories, that would be extremely helpful.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130718/d4cf3d42/attachment.html>

From mattias at nucleus.be  Thu Jul 18 11:35:50 2013
From: mattias at nucleus.be (Mattias Geniar)
Date: Thu, 18 Jul 2013 11:35:50 +0000
Subject: Adding certain vmods to the distribution
In-Reply-To: <CABUevEw_HyuaKnhjSZV=CpihToqWktLOPgUJv4S2umy5TaFDhA@mail.gmail.com>
Message-ID: <CE0D9F73.6ED70%mattias@nucleus.be>

In particular I'm thinking about the variable (variables in vcl), boltsort (querystring argument sorting) and cookie (cookie manipulation) vmods.

What do you think?

If they're included in official packages users would assume they are officially supported by Varnish Software as well, is that the case?
If not, how would you make it bluntly obvious that they are user contributed 'at your own risk'?

Regards,
Mattias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130718/42db7d99/attachment.html>

From perbu at varnish-software.com  Thu Jul 18 12:04:01 2013
From: perbu at varnish-software.com (Per Buer)
Date: Thu, 18 Jul 2013 14:04:01 +0200
Subject: Adding certain vmods to the distribution
In-Reply-To: <CE0D9F73.6ED70%mattias@nucleus.be>
References: <CABUevEw_HyuaKnhjSZV=CpihToqWktLOPgUJv4S2umy5TaFDhA@mail.gmail.com>
 <CE0D9F73.6ED70%mattias@nucleus.be>
Message-ID: <CAOXZevCOm+BrB=q2wOGYD4YH1iegOOCEb94aJNAc+WUP5WLXAA@mail.gmail.com>

On Thu, Jul 18, 2013 at 1:35 PM, Mattias Geniar <mattias at nucleus.be> wrote:

>     In particular I'm thinking about the variable (variables in vcl),
>> boltsort (querystring argument sorting) and cookie (cookie manipulation)
>> vmods.
>>
>>  What do you think?
>>
>
>  If they're included in official packages users would assume they are
> officially supported by Varnish Software as well, is that the case?
>

It would be. The modules would be moved from their various github
repositories to the varnish-cache git tree and maintained there, have tests
that would be run on release etc.


-- 
 <http://www.varnish-software.com/> *Per Buer*
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130718/1aad949b/attachment.html>

From allard at byte.nl  Thu Jul 18 18:57:45 2013
From: allard at byte.nl (Allard Hoeve)
Date: Thu, 18 Jul 2013 20:57:45 +0200
Subject: Adding certain vmods to the distribution
In-Reply-To: <CAOXZevCOm+BrB=q2wOGYD4YH1iegOOCEb94aJNAc+WUP5WLXAA@mail.gmail.com>
References: <CABUevEw_HyuaKnhjSZV=CpihToqWktLOPgUJv4S2umy5TaFDhA@mail.gmail.com>
 <CE0D9F73.6ED70%mattias@nucleus.be>
 <CAOXZevCOm+BrB=q2wOGYD4YH1iegOOCEb94aJNAc+WUP5WLXAA@mail.gmail.com>
Message-ID: <CAKLkcmMfy8tXkC1Mnyh3Cx=D3C882uh4Cg3J8UHjEk47Kag_nw@mail.gmail.com>

+1!

Alsof, vmod-header would be awesome.

Best,

Allard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130718/b1b78927/attachment.html>

From varnish at bsdchicks.com  Thu Jul 18 20:03:59 2013
From: varnish at bsdchicks.com (Rogier 'DocWilco' Mulhuijzen)
Date: Thu, 18 Jul 2013 22:03:59 +0200
Subject: Adding certain vmods to the distribution
In-Reply-To: <CAKLkcmMfy8tXkC1Mnyh3Cx=D3C882uh4Cg3J8UHjEk47Kag_nw@mail.gmail.com>
References: <CABUevEw_HyuaKnhjSZV=CpihToqWktLOPgUJv4S2umy5TaFDhA@mail.gmail.com>
 <CE0D9F73.6ED70%mattias@nucleus.be>
 <CAOXZevCOm+BrB=q2wOGYD4YH1iegOOCEb94aJNAc+WUP5WLXAA@mail.gmail.com>
 <CAKLkcmMfy8tXkC1Mnyh3Cx=D3C882uh4Cg3J8UHjEk47Kag_nw@mail.gmail.com>
Message-ID: <CANTTouWSDSLBjoNrbX7qpDJrG7yhV-bs8D2wU=FOXh1HsCMR2w@mail.gmail.com>

I would humbly suggest urlcode, it's tiny.


On Thu, Jul 18, 2013 at 8:57 PM, Allard Hoeve <allard at byte.nl> wrote:

> +1!
>
> Alsof, vmod-header would be awesome.
>
> Best,
>
> Allard
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130718/42c91732/attachment.html>

From jos at dwim.org  Thu Jul 18 20:27:46 2013
From: jos at dwim.org (Jos Boumans)
Date: Thu, 18 Jul 2013 13:27:46 -0700
Subject: Adding certain vmods to the distribution
In-Reply-To: <CABUevEw_HyuaKnhjSZV=CpihToqWktLOPgUJv4S2umy5TaFDhA@mail.gmail.com>
References: <CAOXZevAs4CeAwzwdp6YOkaMoP+Rh8i2bVFzpH+H4rxT1GTswRQ@mail.gmail.com>
 <CABUevEw_HyuaKnhjSZV=CpihToqWktLOPgUJv4S2umy5TaFDhA@mail.gmail.com>
Message-ID: <ECC57E00-4B9D-4D91-B772-72D740F62725@dwim.org>


On 18 Jul 2013, at 04:06, Magnus Hagander wrote:
> In particular, getting it included in the RPMs and DEBs will make a big difference. I don't particularly care if they're in the source tarball, but the fact that it's quite a PITA to get them installed if your starting point is RPM or DEB is a real bummer today. 

Agreed; if it were easy/possible to have our vmods
hosted from the varnish debian repository (and a way
to have them built against different versions of
varnish?), that'd be a huge win.

May even negate the need to bundle them with the
varnish distribution itself.

My $0.02;

-Jos


From martin at varnish-software.com  Mon Jul 22 12:20:51 2013
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 22 Jul 2013 14:20:51 +0200
Subject: [PATCH] Fix seg fault in VRT_synth_page when the string list has a
 NULL pointer as the first element.
Message-ID: <1374495651-27758-1-git-send-email-martin@varnish-software.com>

Fixes: #1287
---
 bin/varnishd/cache/cache_vrt.c   |    3 +--
 bin/varnishtest/tests/r01287.vtc |   21 +++++++++++++++++++++
 2 files changed, 22 insertions(+), 2 deletions(-)
 create mode 100644 bin/varnishtest/tests/r01287.vtc

diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c
index c63be80..3c86a6b 100644
--- a/bin/varnishd/cache/cache_vrt.c
+++ b/bin/varnishd/cache/cache_vrt.c
@@ -422,9 +422,8 @@ VRT_synth_page(const struct vrt_ctx *ctx, unsigned flags, const char *str, ...)
 	vsb = SMS_Makesynth(ctx->req->obj);
 	AN(vsb);
 
-	VSB_cat(vsb, str);
 	va_start(ap, str);
-	p = va_arg(ap, const char *);
+	p = str;
 	while (p != vrt_magic_string_end) {
 		if (p == NULL)
 			p = "(null)";
diff --git a/bin/varnishtest/tests/r01287.vtc b/bin/varnishtest/tests/r01287.vtc
new file mode 100644
index 0000000..8560ce3
--- /dev/null
+++ b/bin/varnishtest/tests/r01287.vtc
@@ -0,0 +1,21 @@
+varnishtest "#1287 - check NULL as first pointer to VRT_synth_page"
+
+server s1 {
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_recv {
+		return (error(200, "OK"));
+	}
+	sub vcl_error {
+		synthetic obj.http.blank;
+		return (deliver);
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+	expect resp.body == "(null)"
+} -run
-- 
1.7.10.4




From josh.bussdieker at moovweb.com  Sun Jul 28 16:05:21 2013
From: josh.bussdieker at moovweb.com (Josh Bussdieker)
Date: Sun, 28 Jul 2013 09:05:21 -0700
Subject: Key HTTP Response Header Field
Message-ID: <CADKvSTHrc_excP9B5K8f-wd2n1k4TCU9Am9ck2-Wy4tWkoQUfQ@mail.gmail.com>

I'm considering adding support for the HTTP Key response header as
described here: http://tools.ietf.org/html/draft-fielding-http-key-02

I had a few questions about how to proceed:
1. Are there any plans to implement this feature?
2. Would this be possible using a vmod?
3. Assuming it wouldn't be possible with a vmod. Would a patch to add
support be welcomed?

Thanks,

-- 
Joshua B. Bussdieker
Operations Engineer
Moovweb
201 Spear St
3rd Floor
San Francisco, CA 94107
http://moovweb.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130728/cee168ee/attachment.html>

From dridi.boukelmoune at zenika.com  Sun Jul 28 18:41:22 2013
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Sun, 28 Jul 2013 20:41:22 +0200
Subject: Key HTTP Response Header Field
In-Reply-To: <CADKvSTHrc_excP9B5K8f-wd2n1k4TCU9Am9ck2-Wy4tWkoQUfQ@mail.gmail.com>
References: <CADKvSTHrc_excP9B5K8f-wd2n1k4TCU9Am9ck2-Wy4tWkoQUfQ@mail.gmail.com>
Message-ID: <CABtDKm41bn2rTp0iPD4t-Xy=hKvQfhhoDAWXZdv0dBnnVWzeTA@mail.gmail.com>

Hi,

AFAIK Varnish can use any data to build the cache key, the default VCL
uses the host and url from the request:

///
sub vcl_hash {
    hash_data(req.url);
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
    return (hash);
}
///

The 'Key' header being 'cache policy', I believe it belongs in your
VCL and not in Varnish. After reading bits of the draft, VCL should be
enough to build such a header, no vmod required.

My 2 cents :)
Dridi

PS. thank you for the link, this is very interesting


On Sun, Jul 28, 2013 at 6:05 PM, Josh Bussdieker
<josh.bussdieker at moovweb.com> wrote:
> I'm considering adding support for the HTTP Key response header as described
> here: http://tools.ietf.org/html/draft-fielding-http-key-02
>
> I had a few questions about how to proceed:
> 1. Are there any plans to implement this feature?
> 2. Would this be possible using a vmod?
> 3. Assuming it wouldn't be possible with a vmod. Would a patch to add
> support be welcomed?
>
> Thanks,
>
> --
> Joshua B. Bussdieker
> Operations Engineer
> Moovweb
> 201 Spear St
> 3rd Floor
> San Francisco, CA 94107
> http://moovweb.com
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev



From mrkafk at gmail.com  Tue Jul 30 10:58:19 2013
From: mrkafk at gmail.com (Marcin Krol)
Date: Tue, 30 Jul 2013 12:58:19 +0200
Subject: rdbms as backend
Message-ID: <51F79C4B.5050506@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

This is a peculiar topic that I think goes beyond typical use of
varnish so I post it here.

At my company we have a need peculiar sort of infrastructural subsystem:

- - HTTP requests are done to find if smth is cached
- - RDBMS (mysql, oracle) backends
- - Other subsystems as backends

Clients use unified protocol based on (simple) http requests to get
their data. (it's for this reason that we do not use caches built into
rdbms directly, as well as we do not want to do tight coupling of a
particular client to a particular rdbms or subsystem)


Either we write the whole thing ourselves or we use smth else like
varnish.


I like the thought of using varnish, although I'm not sure if it this
is not shoehorning it into such role. However, when it comes to
caching, load balancing, failover and cached HTTP results serving it's
ideal in such role.

The only problem is backend. Essentially, what we need is e.g. for
mysql backend:

on cache miss:

- - connect to mysql

- - run the query we received in GET/POST/whatever

- - JSONIFy result (query results are not big in our application,
limited size of the result is a tolerable limitation for us)

- - cache result, return it


on cache hit:

- - retrieve from cache, return it


(and so on for other backends)


Is this feasible? Is it even sane? Should I use smth else maybe?

Essentially, what we need are pluggable, modular backends. (obviously
we can handle writing the part that transforms particular backend
response into HTTP response, the snag is how to plug this correctly
into backend usable by varnish)

I was thinking about using VMODs but none of the modules available
seem to meddle with backends themselves somehow.


Thanks!
MK

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR95xLAAoJEFMgHzhQQ7hOXOYH/2QZgry4O55fRvvEE0I9tPnw
xLCXV/snfnH2jawW01sFDZ87JV3S9s10bqSoqTs21vMkd7M+hHsp3I/wKTg2dwqR
kQ0M9p2aPiLyijI7v6FCcpaorL10wZ0/12i6A+RnsyPLN2FDeDpCCDeB0oVNauqc
K/yE2lfsAvVu+jxlmzygePlV9ZZ+B455G8GFIIvN+S10QFly1rAOLbCg7Mi1KRq9
4BYPsISZnYDIbAUKjQWLQDmfhoXfbgIZQJMLzIJFiQC2+G1WFIVp94Z5JRsVQlHm
PcIx3Gt0zq5Qd198+97qTDZLPO1irNSpYpWLidd9dnA+2rI85HZU+9B0Ktuwc1g=
=/D+o
-----END PGP SIGNATURE-----



From bilbo at hobbiton.org  Tue Jul 30 12:46:42 2013
From: bilbo at hobbiton.org (Leif Pedersen)
Date: Tue, 30 Jul 2013 07:46:42 -0500
Subject: rdbms as backend
In-Reply-To: <51F79C4B.5050506@gmail.com>
References: <51F79C4B.5050506@gmail.com>
Message-ID: <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>

Hi,

I'm not a varnish dev but I've been working with varnish and DBs for a long
time.

My knee jerk is that you'll end up with more application logic in vcl than
vcl is suitable for. Vcl can't loop or touch response bodies (without
vmods). It'd be kind of neat to skip middleware and plug varnish straight
into the db, but I wouldn't guess it to be worth the effort.

I'd instead write a mini web server using uwsgi or CherryPy to act as
middleware. These tools have great memory footprints and performance.

- Leif
On 2013-07-30 6:30 AM, "Marcin Krol" <mrkafk at gmail.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello,
>
> This is a peculiar topic that I think goes beyond typical use of
> varnish so I post it here.
>
> At my company we have a need peculiar sort of infrastructural subsystem:
>
> - - HTTP requests are done to find if smth is cached
> - - RDBMS (mysql, oracle) backends
> - - Other subsystems as backends
>
> Clients use unified protocol based on (simple) http requests to get
> their data. (it's for this reason that we do not use caches built into
> rdbms directly, as well as we do not want to do tight coupling of a
> particular client to a particular rdbms or subsystem)
>
>
> Either we write the whole thing ourselves or we use smth else like
> varnish.
>
>
> I like the thought of using varnish, although I'm not sure if it this
> is not shoehorning it into such role. However, when it comes to
> caching, load balancing, failover and cached HTTP results serving it's
> ideal in such role.
>
> The only problem is backend. Essentially, what we need is e.g. for
> mysql backend:
>
> on cache miss:
>
> - - connect to mysql
>
> - - run the query we received in GET/POST/whatever
>
> - - JSONIFy result (query results are not big in our application,
> limited size of the result is a tolerable limitation for us)
>
> - - cache result, return it
>
>
> on cache hit:
>
> - - retrieve from cache, return it
>
>
> (and so on for other backends)
>
>
> Is this feasible? Is it even sane? Should I use smth else maybe?
>
> Essentially, what we need are pluggable, modular backends. (obviously
> we can handle writing the part that transforms particular backend
> response into HTTP response, the snag is how to plug this correctly
> into backend usable by varnish)
>
> I was thinking about using VMODs but none of the modules available
> seem to meddle with backends themselves somehow.
>
>
> Thanks!
> MK
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.20 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR95xLAAoJEFMgHzhQQ7hOXOYH/2QZgry4O55fRvvEE0I9tPnw
> xLCXV/snfnH2jawW01sFDZ87JV3S9s10bqSoqTs21vMkd7M+hHsp3I/wKTg2dwqR
> kQ0M9p2aPiLyijI7v6FCcpaorL10wZ0/12i6A+RnsyPLN2FDeDpCCDeB0oVNauqc
> K/yE2lfsAvVu+jxlmzygePlV9ZZ+B455G8GFIIvN+S10QFly1rAOLbCg7Mi1KRq9
> 4BYPsISZnYDIbAUKjQWLQDmfhoXfbgIZQJMLzIJFiQC2+G1WFIVp94Z5JRsVQlHm
> PcIx3Gt0zq5Qd198+97qTDZLPO1irNSpYpWLidd9dnA+2rI85HZU+9B0Ktuwc1g=
> =/D+o
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130730/04a710f6/attachment.html>

From mrkafk at gmail.com  Tue Jul 30 13:12:01 2013
From: mrkafk at gmail.com (Marcin Krol)
Date: Tue, 30 Jul 2013 15:12:01 +0200
Subject: rdbms as backend
In-Reply-To: <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
References: <51F79C4B.5050506@gmail.com>
 <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
Message-ID: <51F7BBA1.8080500@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Leif,

Thanks for answer!


What you write about is what we tried at first, using Tornado. It
works, except it simply does not have performance good enough: 600
req/sec while we need 30,000 req/sec (yes 30 thousand). This is for
entire subsystem which can consist of more than 1 machine but at
Tornado performance that would be at least 50 cores over many serves +
HA redundancy + all the associated overhead...

Currently we're into node.js which is able to serve 4000-5000 req/sec,
still not quite good enough.

Varnish performance is so high that it would be very attractive in
this role.

Re application logic in VCL I don't think we'd suffer this much: it's
"just" about fast cache. About the only serious problem I could see is
if failover and load balancing in varnish can't be made work well
enough (detect failure quickly and start using spare/backup
systematically instead of timing out on failed backends, cause uneven
load on backends, etc).


Thanks!
MK



W dniu 7/30/2013 14:46, Leif Pedersen pisze:
> Hi,
> 
> I'm not a varnish dev but I've been working with varnish and DBs
> for a long time.
> 
> My knee jerk is that you'll end up with more application logic in
> vcl than vcl is suitable for. Vcl can't loop or touch response
> bodies (without vmods). It'd be kind of neat to skip middleware and
> plug varnish straight into the db, but I wouldn't guess it to be
> worth the effort.
> 
> I'd instead write a mini web server using uwsgi or CherryPy to act
> as middleware. These tools have great memory footprints and
> performance.
> 
> - Leif
> 
> On 2013-07-30 6:30 AM, "Marcin Krol" <mrkafk at gmail.com 
> <mailto:mrkafk at gmail.com>> wrote:
> 
> Hello,
> 
> This is a peculiar topic that I think goes beyond typical use of 
> varnish so I post it here.
> 
> At my company we have a need peculiar sort of infrastructural
> subsystem:
> 
> - HTTP requests are done to find if smth is cached - RDBMS (mysql,
> oracle) backends - Other subsystems as backends
> 
> Clients use unified protocol based on (simple) http requests to
> get their data. (it's for this reason that we do not use caches
> built into rdbms directly, as well as we do not want to do tight
> coupling of a particular client to a particular rdbms or
> subsystem)
> 
> 
> Either we write the whole thing ourselves or we use smth else like 
> varnish.
> 
> 
> I like the thought of using varnish, although I'm not sure if it
> this is not shoehorning it into such role. However, when it comes
> to caching, load balancing, failover and cached HTTP results
> serving it's ideal in such role.
> 
> The only problem is backend. Essentially, what we need is e.g. for 
> mysql backend:
> 
> on cache miss:
> 
> - connect to mysql
> 
> - run the query we received in GET/POST/whatever
> 
> - JSONIFy result (query results are not big in our application, 
> limited size of the result is a tolerable limitation for us)
> 
> - cache result, return it
> 
> 
> on cache hit:
> 
> - retrieve from cache, return it
> 
> 
> (and so on for other backends)
> 
> 
> Is this feasible? Is it even sane? Should I use smth else maybe?
> 
> Essentially, what we need are pluggable, modular backends.
> (obviously we can handle writing the part that transforms
> particular backend response into HTTP response, the snag is how to
> plug this correctly into backend usable by varnish)
> 
> I was thinking about using VMODs but none of the modules available 
> seem to meddle with backends themselves somehow.
> 
> 
> Thanks! MK
> 
> 
> _______________________________________________ varnish-dev mailing
> list varnish-dev at varnish-cache.org
> <mailto:varnish-dev at varnish-cache.org> 
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR97uhAAoJEFMgHzhQQ7hOSPkIAIRCFwLmt5FtIa/VHVJoMxZR
BeLx5yKZBSCHJPhBIeywK4sd7+bW7AMLAEh9VYRfe5c7yuZO8mxAXYmvVXPmD/cl
ySUqZJD0LnpdL1kos25K/r8vn6PRPul2jm22u0SloKqQ5ME8TGNsk/SLmFbiLwCt
nhfL7ia+19/ZVK3XSZ5pcvsCyrM8flFS3TdM5TeKWDBhxu0uuccaBkuYy0qOK0zs
5YoLUD9GRwj9WN0pt2xFWYncgFP2wGsPzO9YrpVyc73ExXviI1LF+i+1K2wrOk+v
nYqekDAHufQmLfeupY5Dq/EHyWnN2Yec/cLaiUQADZKs7ua43GkiUqqJ/pxIUbk=
=hymW
-----END PGP SIGNATURE-----



From mrkafk at gmail.com  Tue Jul 30 13:26:35 2013
From: mrkafk at gmail.com (Marcin Krol)
Date: Tue, 30 Jul 2013 15:26:35 +0200
Subject: rdbms as backend
In-Reply-To: <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
References: <51F79C4B.5050506@gmail.com>
 <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
Message-ID: <51F7BF0B.3050002@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


P.S. What I want to do is precisely avoiding modifying response
bodies: JSONIFy/format in the backend handler, handing over typical
HTTP objects to varnish in "Fetch from backend" operation:

https://www.varnish-software.com/static/book/_images/vcl.png




W dniu 7/30/2013 14:46, Leif Pedersen pisze:
> Hi,
> 
> I'm not a varnish dev but I've been working with varnish and DBs
> for a long time.
> 
> My knee jerk is that you'll end up with more application logic in
> vcl than vcl is suitable for. Vcl can't loop or touch response
> bodies (without vmods). It'd be kind of neat to skip middleware and
> plug varnish straight into the db, but I wouldn't guess it to be
> worth the effort.
> 
> I'd instead write a mini web server using uwsgi or CherryPy to act
> as middleware. These tools have great memory footprints and
> performance.
> 
> - Leif
> 
> On 2013-07-30 6:30 AM, "Marcin Krol" <mrkafk at gmail.com 
> <mailto:mrkafk at gmail.com>> wrote:
> 
> Hello,
> 
> This is a peculiar topic that I think goes beyond typical use of 
> varnish so I post it here.
> 
> At my company we have a need peculiar sort of infrastructural
> subsystem:
> 
> - HTTP requests are done to find if smth is cached - RDBMS (mysql,
> oracle) backends - Other subsystems as backends
> 
> Clients use unified protocol based on (simple) http requests to
> get their data. (it's for this reason that we do not use caches
> built into rdbms directly, as well as we do not want to do tight
> coupling of a particular client to a particular rdbms or
> subsystem)
> 
> 
> Either we write the whole thing ourselves or we use smth else like 
> varnish.
> 
> 
> I like the thought of using varnish, although I'm not sure if it
> this is not shoehorning it into such role. However, when it comes
> to caching, load balancing, failover and cached HTTP results
> serving it's ideal in such role.
> 
> The only problem is backend. Essentially, what we need is e.g. for 
> mysql backend:
> 
> on cache miss:
> 
> - connect to mysql
> 
> - run the query we received in GET/POST/whatever
> 
> - JSONIFy result (query results are not big in our application, 
> limited size of the result is a tolerable limitation for us)
> 
> - cache result, return it
> 
> 
> on cache hit:
> 
> - retrieve from cache, return it
> 
> 
> (and so on for other backends)
> 
> 
> Is this feasible? Is it even sane? Should I use smth else maybe?
> 
> Essentially, what we need are pluggable, modular backends.
> (obviously we can handle writing the part that transforms
> particular backend response into HTTP response, the snag is how to
> plug this correctly into backend usable by varnish)
> 
> I was thinking about using VMODs but none of the modules available 
> seem to meddle with backends themselves somehow.
> 
> 
> Thanks! MK
> 
> 
> _______________________________________________ varnish-dev mailing
> list varnish-dev at varnish-cache.org
> <mailto:varnish-dev at varnish-cache.org> 
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR978LAAoJEFMgHzhQQ7hOugUH/2WLtSaYO+/iLlyX/moGNBqR
HTa0E1W+43vIVIeq9OLzKmmxzJm2075Erq952B9/UIVnxNDZ4jo+whOk8foo1K0Q
h4A8XloKrMG1BSNnYqCsdop202YIIPI1AxX8V30slx1btjttdVedAQxwlGZ1j6NC
S43z5EqJ0dKjOAv1JVTLkeAkCWJCXmYj0xIiPYNCcRQ9uR8QCA1Nm+cL3MDuzcIS
TQuEW/wihy4bfNeKe5H7+GutARGHHtiJXzvzgiGFLcto0IasTUroTKN05MTuIjEE
e4t7n4MfLxwninEWitF23UjCQUvoq05cZMihxMD1XjSPtG+qsK+BmyTHUqTab98=
=bT0L
-----END PGP SIGNATURE-----



From varnish at bsdchicks.com  Tue Jul 30 22:09:55 2013
From: varnish at bsdchicks.com (Rogier 'DocWilco' Mulhuijzen)
Date: Wed, 31 Jul 2013 00:09:55 +0200
Subject: rdbms as backend
In-Reply-To: <51F7BF0B.3050002@gmail.com>
References: <51F79C4B.5050506@gmail.com>
 <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
 <51F7BF0B.3050002@gmail.com>
Message-ID: <CANTTouXXp8B9V88PmceVs1vrT2yut73gExKdhjrpRhWi+PtjKA@mail.gmail.com>

My kneejerk reaction is somewhat the same. Varnish is HTTP, and should stay
HTTP only.

It would be the best solution to have an HTTP to your RDBMS layer. Call it
middleware if you will, or maybe in interface.

You don't really need logic for the situation you outlined, if Varnish can
do the caching, all you need to do is set your HTTP-to-RDBMS layer as
backend. Then it's just the standard hit/miss stuff from Varnish point of
view.

If you can't write anything yourself that's fast enough to do that (in any
language), it's not going to be any faster if it's part of Varnish. So last
ditch effort, write something that's multithreaded or async in C to handle
it.

If that is fast enough, and there's a really good reason to integrate it
with Varnish we can always revisit putting it into Varnish. :)


On Tue, Jul 30, 2013 at 3:26 PM, Marcin Krol <mrkafk at gmail.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> P.S. What I want to do is precisely avoiding modifying response
> bodies: JSONIFy/format in the backend handler, handing over typical
> HTTP objects to varnish in "Fetch from backend" operation:
>
> https://www.varnish-software.com/static/book/_images/vcl.png
>
>
>
>
> W dniu 7/30/2013 14:46, Leif Pedersen pisze:
> > Hi,
> >
> > I'm not a varnish dev but I've been working with varnish and DBs
> > for a long time.
> >
> > My knee jerk is that you'll end up with more application logic in
> > vcl than vcl is suitable for. Vcl can't loop or touch response
> > bodies (without vmods). It'd be kind of neat to skip middleware and
> > plug varnish straight into the db, but I wouldn't guess it to be
> > worth the effort.
> >
> > I'd instead write a mini web server using uwsgi or CherryPy to act
> > as middleware. These tools have great memory footprints and
> > performance.
> >
> > - Leif
> >
> > On 2013-07-30 6:30 AM, "Marcin Krol" <mrkafk at gmail.com
> > <mailto:mrkafk at gmail.com>> wrote:
> >
> > Hello,
> >
> > This is a peculiar topic that I think goes beyond typical use of
> > varnish so I post it here.
> >
> > At my company we have a need peculiar sort of infrastructural
> > subsystem:
> >
> > - HTTP requests are done to find if smth is cached - RDBMS (mysql,
> > oracle) backends - Other subsystems as backends
> >
> > Clients use unified protocol based on (simple) http requests to
> > get their data. (it's for this reason that we do not use caches
> > built into rdbms directly, as well as we do not want to do tight
> > coupling of a particular client to a particular rdbms or
> > subsystem)
> >
> >
> > Either we write the whole thing ourselves or we use smth else like
> > varnish.
> >
> >
> > I like the thought of using varnish, although I'm not sure if it
> > this is not shoehorning it into such role. However, when it comes
> > to caching, load balancing, failover and cached HTTP results
> > serving it's ideal in such role.
> >
> > The only problem is backend. Essentially, what we need is e.g. for
> > mysql backend:
> >
> > on cache miss:
> >
> > - connect to mysql
> >
> > - run the query we received in GET/POST/whatever
> >
> > - JSONIFy result (query results are not big in our application,
> > limited size of the result is a tolerable limitation for us)
> >
> > - cache result, return it
> >
> >
> > on cache hit:
> >
> > - retrieve from cache, return it
> >
> >
> > (and so on for other backends)
> >
> >
> > Is this feasible? Is it even sane? Should I use smth else maybe?
> >
> > Essentially, what we need are pluggable, modular backends.
> > (obviously we can handle writing the part that transforms
> > particular backend response into HTTP response, the snag is how to
> > plug this correctly into backend usable by varnish)
> >
> > I was thinking about using VMODs but none of the modules available
> > seem to meddle with backends themselves somehow.
> >
> >
> > Thanks! MK
> >
> >
> > _______________________________________________ varnish-dev mailing
> > list varnish-dev at varnish-cache.org
> > <mailto:varnish-dev at varnish-cache.org>
> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.20 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR978LAAoJEFMgHzhQQ7hOugUH/2WLtSaYO+/iLlyX/moGNBqR
> HTa0E1W+43vIVIeq9OLzKmmxzJm2075Erq952B9/UIVnxNDZ4jo+whOk8foo1K0Q
> h4A8XloKrMG1BSNnYqCsdop202YIIPI1AxX8V30slx1btjttdVedAQxwlGZ1j6NC
> S43z5EqJ0dKjOAv1JVTLkeAkCWJCXmYj0xIiPYNCcRQ9uR8QCA1Nm+cL3MDuzcIS
> TQuEW/wihy4bfNeKe5H7+GutARGHHtiJXzvzgiGFLcto0IasTUroTKN05MTuIjEE
> e4t7n4MfLxwninEWitF23UjCQUvoq05cZMihxMD1XjSPtG+qsK+BmyTHUqTab98=
> =bT0L
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130731/91461c3e/attachment.html>

From bilbo at hobbiton.org  Wed Jul 31 00:45:30 2013
From: bilbo at hobbiton.org (Leif Pedersen)
Date: Tue, 30 Jul 2013 19:45:30 -0500
Subject: rdbms as backend
In-Reply-To: <51F7BF0B.3050002@gmail.com>
References: <51F79C4B.5050506@gmail.com>
 <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
 <51F7BF0B.3050002@gmail.com>
Message-ID: <CAK-wPOjYH1heFPS2Bpa_pX_WFDD5rM_frnsf82Q-j7Ym6mh6MQ@mail.gmail.com>

Interesting. So you're saying that you need 30k rps to the DB, and the
request rate to the front end of Varnish is much higher? If the Varnish
front end is "only" getting 30k rps, then perhaps there's something you can
do to improve the cacheability of objects. Otherwise, that's impressive
even by my standards.

You're sure your DB can keep up in a useful way with 30k rps? That's not
the real bottleneck, is it? I've made that mistake myself, so I feel
compelled to ask. :) If so, also quite impressive that your DB can keep up.

Have you tried implementing this sort of middleware in node.js? Sounds like
you've used node.js a lot, but not for this particular problem if I
understand correctly. Perhaps it would be worth the experiment.

Here's a great read of WSGI servers with surprising performance
differences. Tornado looks okay, but there is better.
http://nichol.as/benchmark-of-python-web-servers

It sounds like the middleware is really trivial, and compared to the entry
bar for adapting Varnish, probably worth trying several WSGI servers and/or
node.js if need be.

I am indeed intrigued by connecting directly to the DB, but you can
probably see my skepticism between the lines here in bright orange. :)
Seems like a much more flexible approach to use middleware, worth a bit of
extra hardware...on the other hand, maybe not worth it if the cost
difference really is 10x.

- Leif


On Tue, Jul 30, 2013 at 8:26 AM, Marcin Krol <mrkafk at gmail.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> P.S. What I want to do is precisely avoiding modifying response
> bodies: JSONIFy/format in the backend handler, handing over typical
> HTTP objects to varnish in "Fetch from backend" operation:
>
> https://www.varnish-software.com/static/book/_images/vcl.png
>
>
>
>
> W dniu 7/30/2013 14:46, Leif Pedersen pisze:
> > Hi,
> >
> > I'm not a varnish dev but I've been working with varnish and DBs
> > for a long time.
> >
> > My knee jerk is that you'll end up with more application logic in
> > vcl than vcl is suitable for. Vcl can't loop or touch response
> > bodies (without vmods). It'd be kind of neat to skip middleware and
> > plug varnish straight into the db, but I wouldn't guess it to be
> > worth the effort.
> >
> > I'd instead write a mini web server using uwsgi or CherryPy to act
> > as middleware. These tools have great memory footprints and
> > performance.
> >
> > - Leif
> >
> > On 2013-07-30 6:30 AM, "Marcin Krol" <mrkafk at gmail.com
> > <mailto:mrkafk at gmail.com>> wrote:
> >
> > Hello,
> >
> > This is a peculiar topic that I think goes beyond typical use of
> > varnish so I post it here.
> >
> > At my company we have a need peculiar sort of infrastructural
> > subsystem:
> >
> > - HTTP requests are done to find if smth is cached - RDBMS (mysql,
> > oracle) backends - Other subsystems as backends
> >
> > Clients use unified protocol based on (simple) http requests to
> > get their data. (it's for this reason that we do not use caches
> > built into rdbms directly, as well as we do not want to do tight
> > coupling of a particular client to a particular rdbms or
> > subsystem)
> >
> >
> > Either we write the whole thing ourselves or we use smth else like
> > varnish.
> >
> >
> > I like the thought of using varnish, although I'm not sure if it
> > this is not shoehorning it into such role. However, when it comes
> > to caching, load balancing, failover and cached HTTP results
> > serving it's ideal in such role.
> >
> > The only problem is backend. Essentially, what we need is e.g. for
> > mysql backend:
> >
> > on cache miss:
> >
> > - connect to mysql
> >
> > - run the query we received in GET/POST/whatever
> >
> > - JSONIFy result (query results are not big in our application,
> > limited size of the result is a tolerable limitation for us)
> >
> > - cache result, return it
> >
> >
> > on cache hit:
> >
> > - retrieve from cache, return it
> >
> >
> > (and so on for other backends)
> >
> >
> > Is this feasible? Is it even sane? Should I use smth else maybe?
> >
> > Essentially, what we need are pluggable, modular backends.
> > (obviously we can handle writing the part that transforms
> > particular backend response into HTTP response, the snag is how to
> > plug this correctly into backend usable by varnish)
> >
> > I was thinking about using VMODs but none of the modules available
> > seem to meddle with backends themselves somehow.
> >
> >
> > Thanks! MK
> >
> >
> > _______________________________________________ varnish-dev mailing
> > list varnish-dev at varnish-cache.org
> > <mailto:varnish-dev at varnish-cache.org>
> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.20 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR978LAAoJEFMgHzhQQ7hOugUH/2WLtSaYO+/iLlyX/moGNBqR
> HTa0E1W+43vIVIeq9OLzKmmxzJm2075Erq952B9/UIVnxNDZ4jo+whOk8foo1K0Q
> h4A8XloKrMG1BSNnYqCsdop202YIIPI1AxX8V30slx1btjttdVedAQxwlGZ1j6NC
> S43z5EqJ0dKjOAv1JVTLkeAkCWJCXmYj0xIiPYNCcRQ9uR8QCA1Nm+cL3MDuzcIS
> TQuEW/wihy4bfNeKe5H7+GutARGHHtiJXzvzgiGFLcto0IasTUroTKN05MTuIjEE
> e4t7n4MfLxwninEWitF23UjCQUvoq05cZMihxMD1XjSPtG+qsK+BmyTHUqTab98=
> =bT0L
> -----END PGP SIGNATURE-----
>



-- 

As implied by email protocols, the information in this message is
not confidential.  Any middle-man or recipient may inspect, modify,
copy, forward, reply to, delete, or filter email for any purpose unless
said parties are otherwise obligated.  As the sender, I acknowledge that
I have a lower expectation of the control and privacy of this message
than I would a post-card.  Further, nothing in this message is
legally binding without cryptographic evidence of its integrity.

http://bilbo.hobbiton.org/wiki/Eat_My_Sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130730/5fa48744/attachment.html>

From mrkafk at gmail.com  Wed Jul 31 08:51:45 2013
From: mrkafk at gmail.com (Marcin Krol)
Date: Wed, 31 Jul 2013 10:51:45 +0200
Subject: rdbms as backend
In-Reply-To: <CAK-wPOjYH1heFPS2Bpa_pX_WFDD5rM_frnsf82Q-j7Ym6mh6MQ@mail.gmail.com>
References: <51F79C4B.5050506@gmail.com>
 <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
 <51F7BF0B.3050002@gmail.com>
 <CAK-wPOjYH1heFPS2Bpa_pX_WFDD5rM_frnsf82Q-j7Ym6mh6MQ@mail.gmail.com>
Message-ID: <51F8D021.8020105@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Leif!

W dniu 7/31/2013 02:45, Leif Pedersen pisze:
> Interesting. So you're saying that you need 30k rps to the DB, and
> the request rate to the front end of Varnish is much higher? If the
> Varnish front end is "only" getting 30k rps, then perhaps there's
> something you can do to improve the cacheability of objects.
> Otherwise, that's impressive even by my standards.

> You're sure your DB can keep up in a useful way with 30k rps?

That's not a single DB, that's *all the subsystems* for which we cache
the results: 30+ mysql and oracle instances (all replicated into
additional set of mirrors for HA course, 3 DBAs babysitting all this
rubbish), several other specialized subsystems. If you aggregate
traffic for all those, it's about 30K rps, more in peak hrs and under
some circumstances actually.

That those cannot keep up is precisely why we're developing caching
layer (actually we have one, but it's spaghetti C developed years ago,
basically unmaintainable by now). The load on those machines is high
at all times, as queries are not very expensive but they're not
trivially cheap either.


That's not
> the real bottleneck, is it? I've made that mistake myself, so I
> feel compelled to ask. :) If so, also quite impressive that your DB
> can keep up.
> 
> Have you tried implementing this sort of middleware in node.js?
> Sounds like you've used node.js a lot,

We haven't, that's the problem: we're a Python shop (with some C
skills available too). Other divisions have used node.js a lot (hence
that's the next solution under consideration as we can get some
assistance).

Still not good enough. Let's face it: caching server is what C / C++ /
binary-compiled static-typing high-performance system-programming
language is built for. In ideal world I'd build this thing in D
(that's unavailable for obvious "chicken" lack of solutions and "egg"
lack of human skills dilemma).


but not for this particular problem if I
> understand correctly. Perhaps it would be worth the experiment.
> 
> Here's a great read of WSGI servers with surprising performance 
> differences. Tornado looks okay, but there is better. 
> http://nichol.as/benchmark-of-python-web-servers

Erm, I do not want to sound ungrateful but that's exactly one of the
pages I started with...

My colleagues investigated gevent. It fell by the waysides, mostly bc
of some Python's C-based extensions leaking memory under so much load.

I have investigated FAPWS3 with surprisingly good results: 3K rps, no
leaking (at least in my application..). It's little known but works
suprisingly well. I'm not sure if credit goes to libev or good FAPWS3
coding but there it is.

Still, not good enough. Etc etc. Memory leaks, crashes, lots of failed
requests, etc etc. Heck I modified hello world example from Cowboy
(Erlang-based http framework) to do stuff like talking over memcached
protocol (another thing on working pile for this caching server) and
got 5K rps but it's not like Erlang programmers are available in
numbers and we're not going to hire one for this project alone. Oh well.

I thought: "what the heck I'll give varnish a try" (on caching
http-interfaced subsystem backend, we have some of those apart from
rdbmses). Result: 7K rps at 1K concurrent connections.

And stays approximately at this rate with increasing number of
concurrent connections, basically up to 5K concurrent connections and
grand total of 4 failed requests (0 at lower rates).

Woohoo!

Other solutions were basically crushed like bugs under this number of
concurrent connections. That's why you should not trust those blog
pages with high benchmark results: it's all fine and dandy to get
those serially. But if the solution has to handle large number of
concurrent connections at any moment - that's where things turn sour.
I had more than one high-serial-performance Python solution fail
miserably under such circumstances.

But now I have a problem: how to plug mysql or oracle into varnish?
I've done some C coding but replacing http-oriented backend handling
in varnish is a little ambitious for me.


> 
> It sounds like the middleware is really trivial, and compared to
> the entry bar for adapting Varnish, probably worth trying several
> WSGI servers and/or node.js if need be.
> 
> I am indeed intrigued by connecting directly to the DB, but you
> can probably see my skepticism between the lines here in bright
> orange. :)

If I understand you correctly, you would not connect to DB directly
either?

We tried that in the past (in small scale). It works for the moment
and that's the problem: WI you need to say, upgrade mysql? Your entire
client logic layer changes, in most/all of the infrastructure. Tight
coupling. Sucks.


> Seems like a much more flexible approach to use middleware, worth a
> bit of extra hardware...on the other hand, maybe not worth it if
> the cost difference really is 10x.

That's what we're trying to reduce: infrastructure, power, maintenance
costs.


Regards,
MK
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR+NAhAAoJEFMgHzhQQ7hOPfYH/iWRxNhVICFZw2I7F/hIUHTv
fL+vk4a6mXLGdK4S8tjDqo9xPXBLpdBHUmQiPqVLZouwz/zy+E15l0zteMcj08Qg
LPrq8/m9E2smcFvPwKTTVpUtq0VmE+MoZqq289VbLxxoxN8v9mwzPy2C/iDBwMu9
hp939RCBTkATJ7XP+ilXvumKsMhRFVCfbdkpbQbSjNifEiDYplwGLV4FheuMOa9F
T4k3M0kFu3BOJmqIvkGrtV5n8ygRtOEn+aK+C/Kq8M2wUsumsLHVnfk+KFmoASev
sh3fRK7AV0lhXoxNM6TG994UqzFZ9zh3yZFTPwYED1ck4VYX8/D1xWuGjrYzN8s=
=VHHc
-----END PGP SIGNATURE-----



From tfheen at varnish-software.com  Wed Jul 31 08:57:30 2013
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 31 Jul 2013 10:57:30 +0200
Subject: [PATCH 1/3] Define a few names in the pkg-config data
Message-ID: <1375261052-15511-1-git-send-email-tfheen@varnish-software.com>

---
 varnishapi.pc.in |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/varnishapi.pc.in b/varnishapi.pc.in
index 7f5036a..059725e 100644
--- a/varnishapi.pc.in
+++ b/varnishapi.pc.in
@@ -9,7 +9,9 @@ datarootdir=@datarootdir@
 datadir=@datadir@
 pkgdatadir=${datadir}/@PACKAGE@
 pkgdataincludedir=${pkgdatadir}/include
+vmodincludedir=${pkgdataincludedir}
 vmoddir=${libdir}/@PACKAGE@/vmods
+vmodtool=${pkgdatadir}/vmodtool.py
 
 Name: VarnishAPI
 Description: Varnish API
-- 
1.7.10.4




From tfheen at varnish-software.com  Wed Jul 31 08:57:31 2013
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 31 Jul 2013 10:57:31 +0200
Subject: [PATCH 2/3] Use /usr/bin/env instead of hard coding python location
In-Reply-To: <1375261052-15511-1-git-send-email-tfheen@varnish-software.com>
References: <1375261052-15511-1-git-send-email-tfheen@varnish-software.com>
Message-ID: <1375261052-15511-2-git-send-email-tfheen@varnish-software.com>

---
 lib/libvcl/vmodtool.py |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/libvcl/vmodtool.py b/lib/libvcl/vmodtool.py
index 259f97f..d99d8c6 100755
--- a/lib/libvcl/vmodtool.py
+++ b/lib/libvcl/vmodtool.py
@@ -1,4 +1,4 @@
-#!/usr/local/bin/python
+#!/usr/bin/env python
 #-
 # Copyright (c) 2010-2011 Varnish Software AS
 # All rights reserved.
-- 
1.7.10.4




From tfheen at varnish-software.com  Wed Jul 31 08:57:32 2013
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 31 Jul 2013 10:57:32 +0200
Subject: [PATCH 3/3] Install autoconf macros
In-Reply-To: <1375261052-15511-1-git-send-email-tfheen@varnish-software.com>
References: <1375261052-15511-1-git-send-email-tfheen@varnish-software.com>
Message-ID: <1375261052-15511-3-git-send-email-tfheen@varnish-software.com>

Add a few macros that are useful when writing vmods.
---
 Makefile.am |    3 ++
 varnish.m4  |   96 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+)
 create mode 100644 varnish.m4

diff --git a/Makefile.am b/Makefile.am
index 87bbe7a..b8d73e1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -7,6 +7,9 @@ SUBDIRS += redhat
 pkgconfigdir = $(libdir)/pkgconfig
 pkgconfig_DATA = varnishapi.pc
 
+m4dir = $(datadir)/aclocal
+m4_DATA = varnish.m4
+
 CLEANFILES = cscope.in.out cscope.out cscope.po.out
 EXTRA_DIST = LICENSE autogen.sh varnishapi.pc.in
 
diff --git a/varnish.m4 b/varnish.m4
new file mode 100644
index 0000000..16e27c5
--- /dev/null
+++ b/varnish.m4
@@ -0,0 +1,96 @@
+# varnish.m4 - Macros to locate Varnish header files.            -*- Autoconf -*-
+# serial 1 (varnish-4.0)
+
+# Copyright (c) 2013 Varnish Software AS
+# All rights reserved.
+#
+# Author: Tollef Fog Heen <tfheen at varnish-software.com>
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+#
+
+# VARNISH_VMOD_INCLUDE_DIR([])
+# ----------------------------
+
+AC_DEFUN([VARNISH_VMOD_INCLUDES],
+[
+m4_pattern_forbid([^_?VARNISH[A-Z_]+$])
+m4_pattern_allow([^VARNISH_VMOD(_INCLUDE_DIR|TOOL)$])
+# Check for pkg-config
+PKG_CHECK_EXISTS([varnishapi],[],[
+	if test -n "$PKG_CONFIG"; then
+		AC_MSG_FAILURE(
+[The pkg-config script could not be found or is too old.  Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.])
+	else
+		AC_MSG_FAILURE(
+[pkg-config was unable to locate the varnishapi configuration data.
+
+Please check config.log or adjust the PKG_CONFIG_PATH environment
+variable if you installed software in a non-standard prefix.])
+	fi
+])
+
+VARNISH_PKG_GET_VAR([VMOD_INCLUDE_DIR], [vmodincludedir])
+VARNISH_PKG_GET_VAR([VAPI_INCLUDE_DIR], [pkgincludedir])
+_CPPFLAGS="$CPPFLAGS"
+VMOD_INCLUDES="-I$VMOD_INCLUDE_DIR -I$VAPI_INCLUDE_DIR"
+CPPFLAGS="$VMOD_INCLUDES $CPPFLAGS"
+AC_CHECK_HEADERS([vsha256.h cache/cache.h])
+CPPFLAGS="$_CPPFLAGS"
+AC_SUBST([VMOD_INCLUDES])
+])# VARNISH_VMOD_INCLUDE_DIR
+
+# VARNISH_VMOD_DIR([])
+# --------------------
+
+AC_DEFUN([VARNISH_VMOD_DIR],
+[
+VARNISH_PKG_GET_VAR([VMOD_DIR], [vmoddir])
+AC_SUBST([VMOD_DIR])
+])
+
+# VARNISH_VMODTOOL([])
+# --------------------
+
+AC_DEFUN([VARNISH_VMODTOOL],
+[
+VARNISH_PKG_GET_VAR([VMODTOOL], [vmodtool])
+AC_SUBST([VMODTOOL])
+])
+
+# VARNISH_PKG_GET_VAR([VARIABLE, PC_VAR_NAME])
+# -------------------------------
+
+AC_DEFUN([VARNISH_PKG_GET_VAR],
+[
+# Uses internal function for now..
+pkg_failed=no
+_PKG_CONFIG([$1], [variable=][$2], [varnishapi])
+if test "$pkg_failed" = "yes"; then
+   AC_MSG_FAILURE([$2][ not defined, too old Varnish?])
+fi
+AS_VAR_COPY([$1], [pkg_cv_][$1])
+])
-- 
1.7.10.4




From lkarsten at varnish-software.com  Wed Jul 31 13:46:49 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 31 Jul 2013 15:46:49 +0200
Subject: [PATCH] Implement std.ip() to simplify ACL checking in VCL
Message-ID: <20130731134648.GB14202@immer.varnish-software.com>

Hello all.

I've extended the std vmod to include an ip() method, which
converts a string into VCL IP. The result can be used for
matching against a VCL ACL.

Attached is a patch against current master. Documentation and
test case included.

Please consider this for merging into the 4.0 release.

-- 
With regards,
Lasse Karstensen
Varnish Software AS
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Implement-std.ip-to-simplify-ACL-checking-in-VCL.patch
Type: text/x-diff
Size: 4071 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130731/419a2c42/attachment.bin>