From bluethundr at gmail.com  Sun Aug  2 18:34:32 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Sun, 2 Aug 2015 14:34:32 -0400
Subject: 504 errors with basic auth in varnish 4 vcl
In-Reply-To: <CAOZy0en9af5kTkRsjfQ3hHPt_rz_h3qGKQ4JyQaSpzegrFoXyA@mail.gmail.com>
References: <CAOZy0en9af5kTkRsjfQ3hHPt_rz_h3qGKQ4JyQaSpzegrFoXyA@mail.gmail.com>
Message-ID: <CAOZy0enG-Qr-sadA1aEG+81DKFKX5HDi_NCtTWPxWMFEemQP4g@mail.gmail.com>

Guys,

 I know I've been a pain in the ass about this. Sorry about that. But I
just wanted to let you know that I got this problem sorted out. The problem
was in fact that the web servers were too damn far away from the varnish
caches. I spun up a few VMs in the new york data center where the varnish
machines were located. And voila! Everything fell into place.

My setup is and always has been a little odd. I have two haproxy load
balancers in AWS / northern Virginia. And I have two databases in the same
location. Everything is done on the free tier.

But vanish would have trouble running on a t2 micro instance so I sprung
for two low cost $10 a month hosts at digital ocean to be the varnish
cashes. The web servers remained at amazon using the free tier until I
could work out that the distance was the problem.

So now I've sprung for 3 low cost $5 a month web servers that only have
512MB of ram. But are definitely fine as web servers.

So even tho my LB's and Databases are in northern virginia on AWS and my
web servers and varnish are in New York on Digital Ocean, everything turned
out fine. The distance between the load balancers and the databases don't
appear to be an issue the way it was for the varnish cache.

And another reason I had put varnish in New York at D.O. was because that's
where I'm developing my app. On a 2GB web server in that location. But I
also have a wiki that I wanted to cache on varnish. And that's what this
project was about.

I'm glad that everything's resolved.

I appreciated your input and advice as always.

Thanks,
Tim

On Wed, Jul 29, 2015 at 10:18 PM, Tim Dunphy <bluethundr at gmail.com> wrote:

> Hey all,
>
> I'm having trouble getting basic auth to work under varnish 4. I'm setting
> it up in front of a mediawiki site.
>
> If I hit the page from either a web browser I get a 504 The server didn't
> respond in time error. This doesn't happen when basic auth is not enabled
> in either the apache config or the varnish config. Which makes me think I
> may be handling basic auth wrong somewhere in my setup.
>
> I am able to curl the health check file through varnish. I'm doing this on
> the varnish server itself:
>
>     #time curl --user admin http://wiki.example.com/healthcheck.php
>     Enter host password for user 'admin':
>     good
>
>     real    0m3.080s
>     user    0m0.003s
>     sys     0m0.004s
>
> The health check file contains only the word 'good'.
>
> On the web server, the healthcheck.php file is in the doc root of the wiki
> site and is readable by the apache user:
>
>     #ls -l /var/www/jf/wiki/healthcheck.php
>     -rw-r--r--. 1 apache ftpgroup 5 Jul 17 00:42
> /var/www/jf/wiki/healthcheck.php
>
> I've setup a no auth exception in the apache vhost for the site:
>
>     <VirtualHost *:80>
>        ServerName wiki.example.com
>        ServerAlias www.wiki.example.com
>        Options -Indexes +FollowSymlinks
>        LogLevel debug
>        ErrorLog logs/wiki-error.log
>        LogFormat "%h %l %u %t \"%r\" %>s %b" common
>        CustomLog logs/wiki-access_log common
>        DocumentRoot /var/www/jf/wiki
>        SetEnvIf Request_URI ^/healthcheck.php noauth=1
>
>        <Directory /var/www/jf/wiki>
>           Options -Indexes
>           AuthType Basic
>           AuthName "JF Wiki Page"
>           AuthUserFile /etc/httpd/auth
>           Require valid-user
>           #equire all granted
>           Allow from env=noauth
>        </Directory>
>
>        <Directory /var/www/jf/wiki/images>
>                     Options -Indexes
>        </Directory>
>     </VirtualHost>
>
>
> On the varnish end I installed the 'basicauth' vmod, and imported it. Then
> set it up in the VCL. Here's how my VCL is looking:
>
>     #egrep -v '#|^$' default.vcl
>     vcl 4.0;
>     import std;
>     import directors;
>     import basicauth;
>     backend web1 {
>       .host = "10.10.10.25"; # <-- not a real IP
>       .port = "80";
>       .connect_timeout = 45s;
>       .first_byte_timeout = 45s;
>       .between_bytes_timeout = 45s;
>       .max_connections = 800;
>       .probe = {
>                     .request =
>                      "GET /healthcheck.php HTTP/1.1"
>                      "Host: wiki.example.com"
>                      "Authorization: Basic LongBasicAuthBase64Hash=="
>                      "Connection: close";
>                     .timeout = 10s;
>                     .interval = 1s;
>                     .window = 15;
>                     .threshold = 8;
>       }
>     }
>     sub vcl_init {
>         new wiki = directors.round_robin();
>         wiki.add_backend(web1);
>     }
>     sub vcl_recv {
>         set req.backend_hint = wiki.backend();
>         if (!basicauth.match("/etc/httpd/auth",  req.http.Authorization)) {
>             return(synth(401, "Authentication required"));
>         }
>     }
>     sub vcl_backend_response {
>     }
>     sub vcl_deliver {
>     }
>     sub vcl_synth {
>         if (resp.status == 401) {
>                     set resp.http.WWW-Authenticate = "Basic";
>          }
>     }
>
>
> You can see in my VCL that I'm attempting to pass basic auth headers to
> the healthcheck .probe.
>
> In varnishlog, when I'm getting the 504 errors in the browser, I'm seeing
> the following:
>
>     -   Timestamp      Process: 1438220128.357217 5.381197 0.000029
>     -   RespHeader     Transfer-Encoding: chunked
>     -   Debug          "RES_MODE 8"
>     -   RespHeader     Connection: close
>     -   RespHeader     Accept-Ranges: bytes
>     -   Debug          "Write error, retval = -1, len = 14553, errno =
> Connection reset by peer"
>     -   Timestamp      Resp: 1438220128.357317 5.381297 0.000101
>     -   Debug          "XXX REF 1"
>     -   ReqAcct        506 0 506 0 0 0
>     -   End
>
> And in the apache error log for the site I'm seeing this authorization
> error that corresponds with the time that I'm getting the 504 error:
>
>     [Thu Jul 30 01:37:43.197847 2015] [authz_core:debug] [pid 29441]
> mod_authz_core.c(809): [client 10.10.10.19:47588] AH01626: authorization
> result of Require valid-user : denied (no authenticated user yet)
>
> I'm hoping to get some suggestions that will get this to work!
>
> Thanks
> Tim
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>
>


-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150802/b6300e03/attachment.html>

From japrice at gmail.com  Mon Aug  3 17:28:56 2015
From: japrice at gmail.com (Jason Price)
Date: Mon, 3 Aug 2015 13:28:56 -0400
Subject: Varnish4 and variable access?
Message-ID: <CAChvjRBL1N7VjuFvC2jNHbAzuhXSZLtVEEffWa9AFC+Fn0xNFQ@mail.gmail.com>

Two things:

1: The Varnish Book (for v3) has this cool table:
https://www.varnish-software.com/book/3/VCL_functions.html which indicates
which variables are accessible in what ways in whatever subroutine you find
yourself.   Is there an updated version for varnish 4?  I could

2: The varnish 3 to 4 guide says that "obj is now read-only. obj.last_use
has been retired."  I'm doing some hacking of error messages in the new
vcl_synth... what variable should I hack on there instead?  Previously it
was obj.* (specifically obj.status and obj.response).  Is it now resp.* ?

-Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150803/4210ec22/attachment.html>

From kervin at adevsoft.com  Mon Aug  3 18:05:56 2015
From: kervin at adevsoft.com (Kervin L. Pierre)
Date: Mon, 3 Aug 2015 18:05:56 +0000
Subject: VCL 4 resp.reason, vcl_synth and redirection
Message-ID: <1438625156.4725.16.camel@adevsoft.com>

One common practice with VCL 3 was to use vcl_error() for redirect, and
with the reason carrying the location.

This was useful since the location may vary and was generated in
vcl_recv().  Hence there needed a way for the generated HTTP Location
header to be sent to vcl_error.

With VCL 4, this can still be done with resp.reason *but* only if the
resp.reason is used *before* resp.status is set.  This is because
setting resp.status seems to reset resp.reason

Hence the following works ( if I've previously set resp.reason to the
location in vcl_recv )...

sub vcl_synth {
    if (resp.status == 750) {
        set resp.http.Location = resp.reason;
        set resp.status = 301;
        return(deliver);
    }
}

But the following does NOT work...

sub vcl_synth {
    if (resp.status == 750) {
        set resp.status = 301; // Resets reason
        set resp.http.Location = resp.reason;
        return(deliver);
    }
}

Is there a "better" or "clearer" way to pass a location to vcl_synth?

Best regards,
Kervin


From japrice at gmail.com  Mon Aug  3 22:16:17 2015
From: japrice at gmail.com (Jason Price)
Date: Mon, 3 Aug 2015 18:16:17 -0400
Subject: logging errors in varnish 4
Message-ID: <CAChvjRB4O4-AsOAWNS=wO_mqOROy5GpBE3NPvsDV8po0SZXSAg@mail.gmail.com>

In varnish 3, I did the following to log odd errors to syslog:

        std.syslog((3*8)+5, "returning backend error page. client.ip is " +
client.ip +
               " req.xid is " + req.xid +
               " req.url is " + req.url +
               " original url is " + req.http.originalurl +
               " beresp.status is " + beresp.status);

In varnish 4, vcl_backend_error does not have access to client.* or req.*,
making this problematic at best.

The 'go to' solution seems to be an external log monitor... a daemonized
version of varnishlog which filters out stuff like this.  Certainly doable,
but this feels like a poor solution, compared with the above.

Am I missing something?

-Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150803/e2a20904/attachment.html>

From apj at mutt.dk  Tue Aug  4 05:56:17 2015
From: apj at mutt.dk (Andreas Plesner)
Date: Tue, 4 Aug 2015 07:56:17 +0200
Subject: VCL 4 resp.reason, vcl_synth and redirection
In-Reply-To: <1438625156.4725.16.camel@adevsoft.com>
References: <1438625156.4725.16.camel@adevsoft.com>
Message-ID: <20150804055617.GB2656@nerd.dk>

On Mon, Aug 03, 2015 at 06:05:56PM +0000, Kervin L. Pierre wrote:
> 
> Is there a "better" or "clearer" way to pass a location to vcl_synth?

return(synth(700, "Reason"));

-- 
Andreas



From apj at mutt.dk  Tue Aug  4 05:59:31 2015
From: apj at mutt.dk (Andreas Plesner)
Date: Tue, 4 Aug 2015 07:59:31 +0200
Subject: logging errors in varnish 4
In-Reply-To: <CAChvjRB4O4-AsOAWNS=wO_mqOROy5GpBE3NPvsDV8po0SZXSAg@mail.gmail.com>
References: <CAChvjRB4O4-AsOAWNS=wO_mqOROy5GpBE3NPvsDV8po0SZXSAg@mail.gmail.com>
Message-ID: <20150804055931.GC2656@nerd.dk>

On Mon, Aug 03, 2015 at 06:16:17PM -0400, Jason Price wrote:

> In varnish 3, I did the following to log odd errors to syslog:
> 
>         std.syslog((3*8)+5, "returning backend error page. client.ip is " +
> client.ip +
>                " req.xid is " + req.xid +
[...]
> Am I missing something?

You're right that client is not accessible in backend functions (for IMS there
may not even be a client), however bereq.* should be a fine replacement for
req.*.
You could also just put the log in the client side of things, i.e. in
vcl_deliver, where you have access to req.* and client.*

Also, syslog is a poor choice performance-wise. The correct solution is, and
has always been, to monitor the shared memory log.

-- 
Andreas



From apj at mutt.dk  Tue Aug  4 06:02:55 2015
From: apj at mutt.dk (Andreas Plesner)
Date: Tue, 4 Aug 2015 08:02:55 +0200
Subject: Varnish4 and variable access?
In-Reply-To: <CAChvjRBL1N7VjuFvC2jNHbAzuhXSZLtVEEffWa9AFC+Fn0xNFQ@mail.gmail.com>
References: <CAChvjRBL1N7VjuFvC2jNHbAzuhXSZLtVEEffWa9AFC+Fn0xNFQ@mail.gmail.com>
Message-ID: <20150804060255.GD2656@nerd.dk>

On Mon, Aug 03, 2015 at 01:28:56PM -0400, Jason Price wrote:
> 
> 1: The Varnish Book (for v3) has this cool table:
> https://www.varnish-software.com/book/3/VCL_functions.html which indicates
> which variables are accessible in what ways in whatever subroutine you find
> yourself.   Is there an updated version for varnish 4?  I could

Don't know about the book, but it's not available in table form in the docs,
however man vcl has information on which variables are available where.

> 2: The varnish 3 to 4 guide says that "obj is now read-only. obj.last_use
> has been retired."  I'm doing some hacking of error messages in the new
> vcl_synth... what variable should I hack on there instead?  Previously it
> was obj.* (specifically obj.status and obj.response).  Is it now resp.* ?

Yes. obj.* is the cached object, resp.* is the response.

-- 
Andreas



From japrice at gmail.com  Tue Aug  4 21:38:56 2015
From: japrice at gmail.com (Jason Price)
Date: Tue, 4 Aug 2015 17:38:56 -0400
Subject: logging errors in varnish 4
In-Reply-To: <20150804055931.GC2656@nerd.dk>
References: <CAChvjRB4O4-AsOAWNS=wO_mqOROy5GpBE3NPvsDV8po0SZXSAg@mail.gmail.com>
 <20150804055931.GC2656@nerd.dk>
Message-ID: <CAChvjRA5hd-4pVzfsLT+m94xJ+NvjuRUh1jtUORPHMiDXr1RTQ@mail.gmail.com>

Thank you.  This can work.

(yes, syslog is slow.  It's only triggered for certain kinds of errors)

On Tue, Aug 4, 2015 at 1:59 AM, Andreas Plesner <apj at mutt.dk> wrote:

> On Mon, Aug 03, 2015 at 06:16:17PM -0400, Jason Price wrote:
>
> > In varnish 3, I did the following to log odd errors to syslog:
> >
> >         std.syslog((3*8)+5, "returning backend error page. client.ip is
> " +
> > client.ip +
> >                " req.xid is " + req.xid +
> [...]
> > Am I missing something?
>
> You're right that client is not accessible in backend functions (for IMS
> there
> may not even be a client), however bereq.* should be a fine replacement for
> req.*.
> You could also just put the log in the client side of things, i.e. in
> vcl_deliver, where you have access to req.* and client.*
>
> Also, syslog is a poor choice performance-wise. The correct solution is,
> and
> has always been, to monitor the shared memory log.
>
> --
> Andreas
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150804/6560eaa3/attachment.html>

From japrice at gmail.com  Wed Aug  5 01:07:42 2015
From: japrice at gmail.com (Jason Price)
Date: Tue, 4 Aug 2015 21:07:42 -0400
Subject: reliable varnish crash using pcre
Message-ID: <CAChvjRBcctXZoLBbj72dvT6j6Bxw6yo990fC+kmgCmHTgp11Jg@mail.gmail.com>

Given the following vcl:

vcl 4.0;
import std;
backend localhost {
    .host = "127.0.0.1";
    .port = "8080";
}
sub vcl_recv {
    if (req.url ~ "^((?!_product).)*$") {
        std.log("made it");
    }
}

The following request causes the crash:

curl "
http://localhost/internal/v2/location?geocode=33.33,44.44&apiKey=hithere"

This ticket discusses the issue:
https://www.varnish-cache.org/trac/ticket/1576   It's closed, however the
remedies mentioned don't help ("-p thread_pool_stack=64k").  The ticket did
mention a request for minimal VCL to replicate.

Version is 4.0.3.  Rpm's come from EPEL.  I tried to get the repo from
varnish-cache.org to work, but they blew up on dependencies around
libpcre.so.0 (despite pcre and pcre-devel being installed).

OS is CentOS 7.1
PCRE version is pcre-8.32-14

-Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150804/58bc2c94/attachment.html>

From geoff at uplex.de  Wed Aug  5 03:26:47 2015
From: geoff at uplex.de (Geoff Simmons)
Date: Wed, 05 Aug 2015 05:26:47 +0200
Subject: reliable varnish crash using pcre
In-Reply-To: <CAChvjRBcctXZoLBbj72dvT6j6Bxw6yo990fC+kmgCmHTgp11Jg@mail.gmail.com>
References: <CAChvjRBcctXZoLBbj72dvT6j6Bxw6yo990fC+kmgCmHTgp11Jg@mail.gmail.com>
Message-ID: <55C18277.3090009@uplex.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 8/5/15 3:07 AM, Jason Price wrote:
> 
> if (req.url ~ "^((?!_product).)*$") {

Are you certain that this regex really does what you want it to do? It
matches a URL with this sequence:

* start of string, followed by
* zero or more repetitions of
  * a point at which "_product" does not match
  * followed by any character
* followed by end of string

My poor imagination fails to come up with a real-world use case for that
.

For your example:

> "/internal/v2/location?geocode=33.33,44.44&apiKey=hithere"

... the regex matches the 'e' at the end of the string (try it at
http://regex101.com/).

If I'm reading it right, your regex will match the empty string, or
the final character of any non-empty string, provided that the final
character is not preceded by "_product". Again, I suspect that that's
not what you wanted.

> This ticket discusses the issue: 
> https://www.varnish-cache.org/trac/ticket/1576   It's closed,
> however the remedies mentioned don't help ("-p
> thread_pool_stack=64k").  The ticket did mention a request for
> minimal VCL to replicate.

That ticket is about regexen that send the PCRE matcher into deep
recursion, and your regex might have also sent PCRE off the rails
(negative lookahead within a zero-or-more quantifier could conceivably
do that).

I think you'd be much better off by rewriting your regex, first of all
because I doubt that it accomplishes functionally what you're trying
to do. But also because deep recursion in PCRE, or generally a regex
that is expensive to match, is in my experience the one thing you can
do in VCL to get Varnish to run slowly. Poorly written regexen in VCL
can slow down whole web sites, I've seen it happen (if not to say:
I've done it myself).


HTH,
Geoff
- -- 
UPLEX Systemoptimierung
Scheffelstra?e 32
22301 Hamburg
http://uplex.de/
Mob: +49-176-63690917
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)

iQIcBAEBCAAGBQJVwYJ2AAoJEOUwvh9pJNUR8vwQAIXX9dPtb8nocSv/Fcmt7Imo
YBbryWrCtz+qakwGc1xMdmk2ul/OgSsZCpoViX9DfVyrxUBsShBtSTsq/bEaOccK
YXBlwj6TlwX1x4uKLfZIT2mc7QOKz9qYEjyy8Aak/+ImaWC6aUs/6y3VhpdLgkuJ
bm3Cncs8bWbT1C9KWwViaMNZ4kf19/x62PtGUaP1LzdD3Wqh1STSaKmqC7rN7W4g
KguTX5jPiuMnGv2ZkT+RtfodCyItAw0b82lQ4Yo0XlA1IMQYFLoDiK95SQUlL0Ad
/GGWwdUSVoXdQkwJiLH1QooFHqtUPHeF4nDQClwQ5E4pNThg6Za10vjrG57wjMGl
TUT5OmAVY+tGn9i0kxlR0Kkg0T7Z1k3nMxpzt9ehhR6uyj99lRyytwc9TTD3jcp6
/y2xOcdMVnbWZasznBO1HNF4ix5l4vas4eicsw2vwFSqk8Onsu/KhWKJ8ti2xH3W
WsLXtAQYP/iQA2Ouvox7J5ynE/2CFABp23D3DurpBAhnCJ6FZMNYZ4hlrNdZ17Si
Csu0JJwy+kZodNL4Lp8qWBDCfDWAHc+kSfXFpayYlva7NIrgZi6DOx/f/w71AcyR
UjxYNFibtZisqUxrnofzpSu2jleIoID3bTVIhOyFNTaqRICQN0ONG8Btl7JBCr4r
2Mj23zYEcQM6o4CA5JE8
=X851
-----END PGP SIGNATURE-----



From japrice at gmail.com  Wed Aug  5 15:01:59 2015
From: japrice at gmail.com (Jason Price)
Date: Wed, 5 Aug 2015 11:01:59 -0400
Subject: reliable varnish crash using pcre
In-Reply-To: <55C18277.3090009@uplex.de>
References: <CAChvjRBcctXZoLBbj72dvT6j6Bxw6yo990fC+kmgCmHTgp11Jg@mail.gmail.com>
 <55C18277.3090009@uplex.de>
Message-ID: <CAChvjRAXM1seMGdLX8a=ooAMR25e=f6kY5_BkAKM4ofx0Q_Wvw@mail.gmail.com>

Geoff: I'm not responsible for the Regex.  As you point out, it's ... ah...
ill-advised is about as polite as I can be.  I've fixed it in my side of
the world.

I discovered it in a varnish3->4 POC I'm doing. When I saw varnish crashes,
I got concerned, found that ticket, then found I was unable to register
with Trac to update the ticket... so I fell back to updating the mailing
list.

-Jason

On Tue, Aug 4, 2015 at 11:26 PM, Geoff Simmons <geoff at uplex.de> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 8/5/15 3:07 AM, Jason Price wrote:
> >
> > if (req.url ~ "^((?!_product).)*$") {
>
> Are you certain that this regex really does what you want it to do? It
> matches a URL with this sequence:
>
> * start of string, followed by
> * zero or more repetitions of
>   * a point at which "_product" does not match
>   * followed by any character
> * followed by end of string
>
> My poor imagination fails to come up with a real-world use case for that
> .
>
> For your example:
>
> > "/internal/v2/location?geocode=33.33,44.44&apiKey=hithere"
>
> ... the regex matches the 'e' at the end of the string (try it at
> http://regex101.com/).
>
> If I'm reading it right, your regex will match the empty string, or
> the final character of any non-empty string, provided that the final
> character is not preceded by "_product". Again, I suspect that that's
> not what you wanted.
>
> > This ticket discusses the issue:
> > https://www.varnish-cache.org/trac/ticket/1576   It's closed,
> > however the remedies mentioned don't help ("-p
> > thread_pool_stack=64k").  The ticket did mention a request for
> > minimal VCL to replicate.
>
> That ticket is about regexen that send the PCRE matcher into deep
> recursion, and your regex might have also sent PCRE off the rails
> (negative lookahead within a zero-or-more quantifier could conceivably
> do that).
>
> I think you'd be much better off by rewriting your regex, first of all
> because I doubt that it accomplishes functionally what you're trying
> to do. But also because deep recursion in PCRE, or generally a regex
> that is expensive to match, is in my experience the one thing you can
> do in VCL to get Varnish to run slowly. Poorly written regexen in VCL
> can slow down whole web sites, I've seen it happen (if not to say:
> I've done it myself).
>
>
> HTH,
> Geoff
> - --
> UPLEX Systemoptimierung
> Scheffelstra?e 32
> 22301 Hamburg
> http://uplex.de/
> Mob: +49-176-63690917
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
>
> iQIcBAEBCAAGBQJVwYJ2AAoJEOUwvh9pJNUR8vwQAIXX9dPtb8nocSv/Fcmt7Imo
> YBbryWrCtz+qakwGc1xMdmk2ul/OgSsZCpoViX9DfVyrxUBsShBtSTsq/bEaOccK
> YXBlwj6TlwX1x4uKLfZIT2mc7QOKz9qYEjyy8Aak/+ImaWC6aUs/6y3VhpdLgkuJ
> bm3Cncs8bWbT1C9KWwViaMNZ4kf19/x62PtGUaP1LzdD3Wqh1STSaKmqC7rN7W4g
> KguTX5jPiuMnGv2ZkT+RtfodCyItAw0b82lQ4Yo0XlA1IMQYFLoDiK95SQUlL0Ad
> /GGWwdUSVoXdQkwJiLH1QooFHqtUPHeF4nDQClwQ5E4pNThg6Za10vjrG57wjMGl
> TUT5OmAVY+tGn9i0kxlR0Kkg0T7Z1k3nMxpzt9ehhR6uyj99lRyytwc9TTD3jcp6
> /y2xOcdMVnbWZasznBO1HNF4ix5l4vas4eicsw2vwFSqk8Onsu/KhWKJ8ti2xH3W
> WsLXtAQYP/iQA2Ouvox7J5ynE/2CFABp23D3DurpBAhnCJ6FZMNYZ4hlrNdZ17Si
> Csu0JJwy+kZodNL4Lp8qWBDCfDWAHc+kSfXFpayYlva7NIrgZi6DOx/f/w71AcyR
> UjxYNFibtZisqUxrnofzpSu2jleIoID3bTVIhOyFNTaqRICQN0ONG8Btl7JBCr4r
> 2Mj23zYEcQM6o4CA5JE8
> =X851
> -----END PGP SIGNATURE-----
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150805/db83dfb2/attachment.html>

From andy at eventbase.com  Thu Aug 13 05:18:00 2015
From: andy at eventbase.com (Andy Fase)
Date: Wed, 12 Aug 2015 22:18:00 -0700
Subject: Multi varnish setup and purging
Message-ID: <1439443081493-a8f04205-b809b0d2-37139394@eventbase.com>

Hi there
We're in the process of taking our 3.0.X single varnish instance into a HA 
multi
varnish setup.
My plan for caching was to initially use a hash director to pass requests 
from
whatever the incoming varnish instance is to a specific one (based on the 
URL)
so that the cache for a given URL will only be kept in one place.
Hence the flow would be
Varnish -> varnish -> backend
However our overall application makes heavy use of wildcard purging via the 
ban
command. As each URL in a wildcard ban could be on different varnish 
instances I
don't see how this could actually work?
Does anyone have any experience in supporting purging multiple varnish 
servers
while supporting wildcard purges without having to get the application to 
send
separate purge requests to all varnish instances?
I was thinking of maybe a round robin director with all varnish instances
defined as backends and some vcl to actually do the ban but return a error 
(to
the calling varnish instance) so that the ban gets sent to all varnish 
instances
(because each of them return an error after the ban)
Any thoughts appreciated
Cheers Andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150812/4d6c6259/attachment.html>

From schmidt at ze.tum.de  Thu Aug 13 06:57:18 2015
From: schmidt at ze.tum.de (Gerhard Schmidt)
Date: Thu, 13 Aug 2015 08:57:18 +0200
Subject: Authentication required without -S Option.
Message-ID: <55CC3FCE.9070602@ze.tum.de>

Hi,

i'm in the progress of migrating from varnish 3 to varnish 4. Now there
is a slight problem with the cli interface.

I'm starting varnish with

/usr/local/sbin/varnishd -P /var/run/varnishd.pid -a
'129.187.39.180:80,127.0.0.1:443,[2001:4ca0:2e03::1:180]:80' -T
127.0.0.1:6082 -f /usr/local/etc/varnish/portal.vcl -s
file,/data/varnish/varnish.store,10g -h classic,16383 -u www -g www -l
512m -p vcc_allow_inline_c=on

these is no -S option there, but when I try to connect to the cli
interface.

telnet localhost 6082
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
107 59
rgbmmpgdcivehnfmoteaeezmeyziftex

Authentication required.

why is authentication required how can I deactivate it.

I'm running

varnishd (varnish-4.0.3 revision b8c4a34)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2014 Varnish Software AS

on FreeBSD 10.1-RELEASE-p17

Regards
   Estartu


-- 
----------------------------------------------------------
Gerhard Schmidt                | E-Mail: schmidt at ze.tum.de
Technische Universit?t M?nchen | Jabber: estartu at ze.tum.de
WWW & Online Services          |
Tel: +49 89 289-25270          | PGP-PublicKey
Fax: +49 89 289-25257          | on request
-------------- next part --------------
A non-text attachment was scrubbed...
Name: schmidt.vcf
Type: text/x-vcard
Size: 382 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150813/bb5c4bca/attachment.vcf>

From pprocacci at datapipe.com  Thu Aug 13 07:02:53 2015
From: pprocacci at datapipe.com (Paul A. Procacci)
Date: Thu, 13 Aug 2015 03:02:53 -0400
Subject: Authentication required without -S Option.
In-Reply-To: <55CC3FCE.9070602@ze.tum.de>
References: <55CC3FCE.9070602@ze.tum.de>
Message-ID: <20150813070253.GA57921@workvm.myhome>

> telnet localhost 6082
> Trying ::1...
> telnet: connect to address ::1: Connection refused
> Trying 127.0.0.1...
> Connected to localhost.
> Escape character is '^]'.
> 107 59
> rgbmmpgdcivehnfmoteaeezmeyziftex
> 
> Authentication required.

Use `varnishadm` instead of telnet.  I forget the specifics as to why,
(too tired to look it up) but it should work just fine.

~Paul



From ram9_pal at yahoo.com  Thu Aug 13 19:36:42 2015
From: ram9_pal at yahoo.com (Ram Palagummi)
Date: Thu, 13 Aug 2015 19:36:42 +0000 (UTC)
Subject: Urgent : Panic message:#012Assert error in vbf_fetch_thread()
Message-ID: <2142252265.4288521.1439494602514.JavaMail.yahoo@mail.yahoo.com>

Hi,
I'm using Varnish version 4.0.2 and noticing the following messages intermittently in production.?Is this fixed in version 4.0.3? Can we safely upgrade to that version assuming it will resolve the problem?
Can you please help me with this?

Child (18378) Panic message:#012Assert error invbf_fetch_thread(), cac

he/cache_fetch.c line 842:#012? Condition(uu ==bo->fetch_obj->len) not true.#012thread = (cache-worker)#012ident = 

Linux,2.6.32-504.3.3.el6.x86_64,x86_64,-smalloc,-sfile,-smalloc,-hcritbit,epoll#012Backtrace:#012?0x43b4ed: /usr/sb

in/varnishd() [0x43b4ed]#012? 0x43b7fd:/usr/sbin/varnishd() [0x43b7fd]#012? 0x425a9d: /usr/sbin/varnishd()[0x425a9

d]#012? 0x43e44f:/usr/sbin/varnishd(Pool_Work_Thread+0x4ce) [0x43e44f]#012? 0x456f84:/usr/sbin/varnishd() [0x456f8

4]#012? 0x4570ad: /usr/sbin/varnishd(WRK_thread+0x27)[0x4570ad]#012? 0x7f5a48e369d1: /lib64/libpthread.so.0(+0x79d1

) [0x7f5a48e369d1]#012? 0x7f5a48b838fd:/lib64/libc.so.6(clone+0x6d) [0x7f5a48b838fd]#012? busyobj =0x7f552b5ba020 

{#012??? ws = 0x7f552b5ba0e0{#012????? id ="bo",#012????? {s,f,r,e} = {0x7f552b5bc008,+2712,(nil),+57368},#012???

},#012? refcnt = 1#012? retries = 0#012?failed = 0#012? state = 3#012??? is_do_esi#012???is_uncacheable#012??? is_

is_gzip#012???is_should_close#012??? bodystatus = 2(chunked),#012??? },#012??? ws = 0x7f552b5ba270{#012????? id =

"obj",#012????? {s,f,r,e}= {0x7f552b4b0308,+536,(nil),+536},#012??? },#012? objcore(FETCH) = 0x7f553366c300 {#012

??? refcnt = 2#012??? flags =0x4#012??? objhead = 0x7f55337c5b00#012? }#012? obj(FETCH) = 0x7f552b4b0100 {#012??? 

vxid = 2147647956,#012??? http[obj] ={#012????? ws =(nil)[]#012???????"HTTP/1.1",#012???????"200",#012??????? "O

K",#012???????"Date: Wed, 12 Aug 2015 18:59:39GMT",#012??????? "Server:Apache-Coyote/1.1",#012???????"WWW-Authen

ticate: Basic realm="CT"",#012???????"X-UA-Compatible:IE=edge,chrome=1",#012???????"Content-Type: text/html;chars

et=utf-8",#012???????"Set-Cookie: UW_JSESSIONID=43865A1084328B57F912E68642490A59.vm08tcs_8080;Path=/",#012??????? 

"Set-Cookie: UW_JSESSIONID=43865A1084328B57F912E68642490A59.vm08tcs_8080;?Domain=.emc.com;? Path=/;? HttpOnly",#012

??????? "Via: 1.1poland.emc.com",#012??????? "Vary:Accept-Encoding",#012 ? ??


ThanksRam?


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150813/5cc98612/attachment.html>

From dridi at varni.sh  Thu Aug 13 21:24:25 2015
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Thu, 13 Aug 2015 23:24:25 +0200
Subject: Urgent : Panic message:#012Assert error in vbf_fetch_thread()
In-Reply-To: <2142252265.4288521.1439494602514.JavaMail.yahoo@mail.yahoo.com>
References: <2142252265.4288521.1439494602514.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CABoVN9C9AHg3CY1vWwUp_h8NhMKqPXAMj0XSGcAPnUXM=bAXpg@mail.gmail.com>

On Thu, Aug 13, 2015 at 9:36 PM, Ram Palagummi <ram9_pal at yahoo.com> wrote:
> Hi,
>
> I'm using Varnish version 4.0.2 and noticing the following messages
> intermittently in production.

Hi,

It looks like this bug:
https://www.varnish-cache.org/trac/ticket/1615

> Is this fixed in version 4.0.3? Can we safely upgrade to that version
> assuming it will resolve the problem?
>
> Can you please help me with this?

It should be fixed in 4.0.3 according to the changelog.

Cheers,
Dridi



From bluethundr at gmail.com  Sun Aug 16 04:01:36 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Sun, 16 Aug 2015 00:01:36 -0400
Subject: logical or statement failing
Message-ID: <CAOZy0enePxbi3duetjM_H2-dRact7u_of3vz1RPezbz72e-d8Q@mail.gmail.com>

Hey guys,

 I'm trying to allow logins on my site. So I setup the following in my VCL
under varnish 4:

# Allow the beta site to login
   if ( req.http.host ~ "^beta\.jokefire\.com$" && req.url ~
"^?oa_social_login_source=custom$" ) {
      return (pass);
   }

But when I go to do a syntax check on the VCL I get the following error:

#varnishd -C -f default.vcl
Message from VCC-compiler:
Regexp compilation error:

nothing to repeat

('input' Line 111 Pos 62)
   if ( req.http.host ~ "^beta\.jokefire\.com$" && req.url ~
"^?oa_social_login_source=custom$" ) {
-------------------------------------------------------------##################################----

Running VCC-compiler failed, exited with 2

VCL compilation failed


Could I get a little help with the syntax to get this working?

Thanks,
Tim
-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150816/fdfb1b30/attachment.html>

From glenn at squixa.com  Sun Aug 16 08:19:05 2015
From: glenn at squixa.com (Glenn Slaven)
Date: Sun, 16 Aug 2015 18:19:05 +1000
Subject: logical or statement failing
In-Reply-To: <CAOZy0enePxbi3duetjM_H2-dRact7u_of3vz1RPezbz72e-d8Q@mail.gmail.com>
References: <CAOZy0enePxbi3duetjM_H2-dRact7u_of3vz1RPezbz72e-d8Q@mail.gmail.com>
Message-ID: <CAPeL0O9d9BWwf0mmjUz58miih6SzrTiepww4Gma7xNQeGVwRtA@mail.gmail.com>

Hi Tim,

The problem is you need to escape the question mark in the regex like this:

"^\?oa_social_login_source=custom$"

The question maker is a regex special character meaning "match the previous
character one or zero times.  That doesn't really work with the start of
line character.

You can see this compiling here

http://www.vclfiddle.net/150816-1e0353b/3

As a side note, the req.url will include the slash at the start if people
put in this url

http://beta.jokefire.com/?oa_social_login_source=custom

which would be entirely valid, could I suggest something like this:

"^/?\?oa_social_login_source=custom$"

Glenn

On 16 August 2015 at 14:01, Tim Dunphy <bluethundr at gmail.com> wrote:

>
> Hey guys,
>
>  I'm trying to allow logins on my site. So I setup the following in my VCL
> under varnish 4:
>
> # Allow the beta site to login
>    if ( req.http.host ~ "^beta\.jokefire\.com$" && req.url ~
> "^?oa_social_login_source=custom$" ) {
>       return (pass);
>    }
>
> But when I go to do a syntax check on the VCL I get the following error:
>
> #varnishd -C -f default.vcl
> Message from VCC-compiler:
> Regexp compilation error:
>
> nothing to repeat
>
> ('input' Line 111 Pos 62)
>    if ( req.http.host ~ "^beta\.jokefire\.com$" && req.url ~
> "^?oa_social_login_source=custom$" ) {
>
> -------------------------------------------------------------##################################----
>
> Running VCC-compiler failed, exited with 2
>
> VCL compilation failed
>
>
> Could I get a little help with the syntax to get this working?
>
> Thanks,
> Tim
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
*Glenn Slaven*
Principal Engineer

Ph: (02) 9119 0444
M: 0433 826 255
www.squixa.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150816/b38e94fc/attachment.html>

From bluethundr at gmail.com  Sun Aug 16 18:00:20 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Sun, 16 Aug 2015 14:00:20 -0400
Subject: logical or statement failing
In-Reply-To: <CAPeL0O9d9BWwf0mmjUz58miih6SzrTiepww4Gma7xNQeGVwRtA@mail.gmail.com>
References: <CAOZy0enePxbi3duetjM_H2-dRact7u_of3vz1RPezbz72e-d8Q@mail.gmail.com>
 <CAPeL0O9d9BWwf0mmjUz58miih6SzrTiepww4Gma7xNQeGVwRtA@mail.gmail.com>
Message-ID: <CAOZy0en9Kn4Strwd9FoT2rv-ZTRTFYiDcA+BOU4DwhByNzooDw@mail.gmail.com>

Hi Glenn,

 Thanks for your reply! And yes that does indeed compile.

 # Allow the beta site to login
   if ( req.http.host ~ "^beta\.jokefire\.com$" &&
"^/?\?oa_social_login_source=custom$"  ) {
      return (pass);
   }

I actually remembered that the ? is a regex character and needed to be
escaped before I read your email. That got it to compile. But one thing
that I didn't think of was including the beginning slash at the start of
the regex. So instead I had this: "^\?oa_social_login_source=custom$". That
compiles too. But actually neither solution, yours nor mine, allows you to
login to my wordpress site.

So I might have to do a little more digging there on how to do this. But of
course if anyone on the list can suggest some good resources on how to
front wordpress with varnish that you like and shows you how to solve the
login problem I'm having, that would be great!

Thanks,
Tim

On Sun, Aug 16, 2015 at 4:19 AM, Glenn Slaven <glenn at squixa.com> wrote:

> Hi Tim,
>
> The problem is you need to escape the question mark in the regex like
> this:
>
> "^\?oa_social_login_source=custom$"
>
> The question maker is a regex special character meaning "match the
> previous character one or zero times.  That doesn't really work with the
> start of line character.
>
> You can see this compiling here
>
> http://www.vclfiddle.net/150816-1e0353b/3
>
> As a side note, the req.url will include the slash at the start if people
> put in this url
>
> http://beta.jokefire.com/?oa_social_login_source=custom
>
> which would be entirely valid, could I suggest something like this:
>
> "^/?\?oa_social_login_source=custom$"
>
> Glenn
>
> On 16 August 2015 at 14:01, Tim Dunphy <bluethundr at gmail.com> wrote:
>
>>
>> Hey guys,
>>
>>  I'm trying to allow logins on my site. So I setup the following in my
>> VCL under varnish 4:
>>
>> # Allow the beta site to login
>>    if ( req.http.host ~ "^beta\.jokefire\.com$" && req.url ~
>> "^?oa_social_login_source=custom$" ) {
>>       return (pass);
>>    }
>>
>> But when I go to do a syntax check on the VCL I get the following error:
>>
>> #varnishd -C -f default.vcl
>> Message from VCC-compiler:
>> Regexp compilation error:
>>
>> nothing to repeat
>>
>> ('input' Line 111 Pos 62)
>>    if ( req.http.host ~ "^beta\.jokefire\.com$" && req.url ~
>> "^?oa_social_login_source=custom$" ) {
>>
>> -------------------------------------------------------------##################################----
>>
>> Running VCC-compiler failed, exited with 2
>>
>> VCL compilation failed
>>
>>
>> Could I get a little help with the syntax to get this working?
>>
>> Thanks,
>> Tim
>> --
>> GPG me!!
>>
>> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
>
>
> --
> *Glenn Slaven*
> Principal Engineer
>
> Ph: (02) 9119 0444
> M: 0433 826 255
> www.squixa.com
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150816/133b0548/attachment.html>

From geoff at uplex.de  Sun Aug 16 22:06:53 2015
From: geoff at uplex.de (Geoff Simmons)
Date: Mon, 17 Aug 2015 00:06:53 +0200
Subject: logical or statement failing
In-Reply-To: <CAOZy0en9Kn4Strwd9FoT2rv-ZTRTFYiDcA+BOU4DwhByNzooDw@mail.gmail.com>
References: <CAOZy0enePxbi3duetjM_H2-dRact7u_of3vz1RPezbz72e-d8Q@mail.gmail.com>
 <CAPeL0O9d9BWwf0mmjUz58miih6SzrTiepww4Gma7xNQeGVwRtA@mail.gmail.com>
 <CAOZy0en9Kn4Strwd9FoT2rv-ZTRTFYiDcA+BOU4DwhByNzooDw@mail.gmail.com>
Message-ID: <55D1097D.5090008@uplex.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 8/16/15 8:00 PM, Tim Dunphy wrote:
> 
> "^/?\?oa_social_login_source=custom$"
> 
> "^\?oa_social_login_source=custom$"

The '^' anchors the regex at the beginning of the string. So the first
one will match exactly this URL:

	/?oa_social_login_source=custom

Meaning '/' with the query string 'oa_social_login_source=custom'.
Since that didn't work, that's apparently not the URL you wanted.

The second one matches exactly this string:

	?oa_social_login_source=custom

... which can never be a legal URL -- it's just a query string without
a path.

If you're trying to say that the URL should have that key-value pair
in the query string (and may have others), then just do this:

	req.url ~ "oa_social_login_source=custom"

... without either '^' or '$'.

Or if you're trying to say that the query string should be strictly
that key-value pair and nothing else, then:

	req.url ~ "\?oa_social_login_source=custom$"

That is, match from the '?' to end-of-string.


HTH,
Geoff
- -- 
UPLEX Systemoptimierung
Scheffelstra?e 32
22301 Hamburg
http://uplex.de/
Mob: +49-176-63690917
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)

iQIcBAEBCAAGBQJV0Ql8AAoJEOUwvh9pJNURwqEP/3OGbztdPC/gFnNM8KlQj+fU
32onqfVD67nUYPwdLhD8Y8kdhi9xebwmVVHsEe8h2yW3Ob+mH4ixm/+s7cbKQTaQ
gTs37/c49BtzRVTfV3jFItcXtIcHvUh9x9dOWfpJ3/ruHfQg2Srbj7OKQr7cgn4l
VmtPUaph329XBCsPUH0j8ByXaFFCpTiTTqhWahUnpvm9GI4KTEh+Dg/vusIOI9e9
z6c6QAylvHnFKBg46ocIqUHT0DzjkxXwt7epES5Lq8MTOAIvN/joCE3GbjkETIwi
LumjzXk83kVgMxqZUMpKNO1jnAqvxidNJPDD7T9cACHGEj3mvF1uyIlvZcnReC5R
PSBbgCBh6ZHtR6AqXbIgJ7cJpoohGBk+Ur9w06ohINq7UoudwjsU/mO6gpvFFCtB
fOP8+VE4nCmUxdF3pO4WQUzeHRm7VKzEkZsYMhqoRjPDaNyWBzJpyLhco0xniMY0
e+hqxxbu18ot5aVxNZPbh0X9x9F5G8CDbbhlBgoMJWaqou+0vnYfRBr6/E0pCazd
+QMYVtiCjLSgz/PIbAZ08ExDtQde6rlRHnWlERMgYwZvrdzY+PidAA8kBWM2lztz
KsJD9f9bRieSxjFjl16Zb/PeOe5bLpAZH+LM5qXRG9AsNO8fms5nSh/UV7BDGgil
NzypSOq96qDWezbCxBzS
=H9au
-----END PGP SIGNATURE-----



From ram9_pal at yahoo.com  Mon Aug 17 02:33:52 2015
From: ram9_pal at yahoo.com (Ram Palagummi)
Date: Mon, 17 Aug 2015 02:33:52 +0000 (UTC)
Subject: Urgent : Panic message:#012Assert error in vbf_fetch_thread()
In-Reply-To: <CABoVN9C9AHg3CY1vWwUp_h8NhMKqPXAMj0XSGcAPnUXM=bAXpg@mail.gmail.com>
References: <CABoVN9C9AHg3CY1vWwUp_h8NhMKqPXAMj0XSGcAPnUXM=bAXpg@mail.gmail.com>
Message-ID: <83294124.5486871.1439778832429.JavaMail.yahoo@mail.yahoo.com>

Thank you so much for your prompt reply.
-Ram 


     On Thursday, August 13, 2015 2:25 PM, Dridi Boukelmoune <dridi at varni.sh> wrote:
   

 On Thu, Aug 13, 2015 at 9:36 PM, Ram Palagummi <ram9_pal at yahoo.com> wrote:
> Hi,
>
> I'm using Varnish version 4.0.2 and noticing the following messages
> intermittently in production.

Hi,

It looks like this bug:
https://www.varnish-cache.org/trac/ticket/1615

> Is this fixed in version 4.0.3? Can we safely upgrade to that version
> assuming it will resolve the problem?
>
> Can you please help me with this?

It should be fixed in 4.0.3 according to the changelog.

Cheers,
Dridi


   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/770211e7/attachment.html>

From bluethundr at gmail.com  Mon Aug 17 03:29:57 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Sun, 16 Aug 2015 23:29:57 -0400
Subject: can't log in or create new users on mediawiki
Message-ID: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>

Hey all,

 I've setup my mediawiki behind varnish. Since doing that I am unable to
either create new users or log into the wiki itself.

I've tried putting the following into my vcl_recv, however that didn't seem
to help as I'd expected:

   if ( req.http.host ~ "^wiki\.example\.com$" && req.url ~
"^/index.php?title=Special:UserLogin&returnto=Main+Page&type=signup" ) {
    return (pass) ;
   }

   if ( req.http.host ~ "^wiki\.example\.com$" && req.url ~
"^/index.php?title=Special:UserLogin&returnto=Main+Page&returntoquery=action%3Dsubmitlogin%26type%3Dlogin$"
) {
    return (pass);
   }

What happens is that when you try to login, or create a new user instead of
doing either the page will just refresh. And the new user isn't created or
you are not logged in.

I've been able to verify that the new user isn't created by looking at the
users tables in the wiki database. And the user you are trying to create
doesn't appear.

Can someine please tell me what I'm doing wrong in my VCL?

Thanks,
Tim

-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150816/6b6b67f8/attachment.html>

From geoff at uplex.de  Mon Aug 17 06:24:44 2015
From: geoff at uplex.de (Geoff Simmons)
Date: Mon, 17 Aug 2015 08:24:44 +0200
Subject: can't log in or create new users on mediawiki
In-Reply-To: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
Message-ID: <55D17E2C.1030003@uplex.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 8/17/15 5:29 AM, Tim Dunphy wrote:
> 
> I've setup my mediawiki behind varnish. Since doing that I am 
> unable to either create new users or log into the wiki itself.

I don't know how it works with mediawiki, but let's talk about regexen
again. %^)

> req.http.host ~ "^wiki\.example\.com$"

If you're going for an exact string match -- '^' followed by a fixed
string followed by '$' -- you might as well just compare for string
equality:

	req.http.host == "wiki.example.com"

That might be a little faster, and it makes the intention of your code
much clearer.

> req.url ~ 
> "^/index.php?title=Special:UserLogin&returnto=Main+Page&type=signup"

>
> 
req.url ~
> "^/index.php?title=Special:UserLogin&returnto=Main+Page&returntoquery=action%3Dsubmitlogin%26type%3Dlogin$"

Same
> 
thing here, with the additional problem that '.', '?' and '+' are
regex metacharacters. "index.php?" will match "index.ph" possibly
followed by a "p", but it won't match the question mark (it also
matches anything where the dot is). "Main+" will match "Mai" followed
one or more "n"s, but it won't match the plus sign.

So your idea may or may not work with mediawiki, but at any rate the
regex matches were failing.

If you're sure you're looking for those exact strings, just use the
'==' comparison.

Are you certain that the URLs of interest will always be written
exactly those two ways? No one will ever use '/' instead of
'/index.php', and the key-value pairs in the query string will always
be in exactly that order?

If not, you'd be better off using regexen after all, something like this:

	req.url ~ "^/(index\.php)?" && req.url ~ "foo=bar"
	&& req.url ~ "baz=quux"

Meaning: the path must be '/' possibly followed by 'index.php', and
the key-value pairs must all appear (in any order).

Again, this is just to ensure that you're identifying the URLs you're
looking for, I can't say if mediawiki will then do as you expect.


HTH,
Geoff
- -- 
UPLEX Systemoptimierung
Scheffelstra?e 32
22301 Hamburg
http://uplex.de/
Mob: +49-176-63690917
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)

iQIcBAEBCAAGBQJV0X4rAAoJEOUwvh9pJNURKLsP/1qI5suvBWXW+BwwXtXmINQY
vPU78Z8JfkdDeXSxmc3lDaazO0L/KE2IwsNnkP5Vl/kJ6EbI3sU8Eq7eXZ73v563
W2d/poA6tlFKHuJovP/opGLhKu1Ls4qUGc74njywNTxiS+pqSvQKPx3OndoYEfyi
/MiPkaPAbSfa7YVKMQ6dHlaEKXP6VJQ3JOJwWKQl4ehl9V1EP1Ddcd2ABHAGf2s+
2yaEINvDVRVek/QBkzgzZbN//KXjtpQdtELjSriNAAAUV+Oi1ULP9ROL0QE+IVYO
Gqe9q0AxquOxn7D6DFgUyZmZWgEjDPy3V8JvNMI+drwTewhEk1g9A1yOI9v+5RJb
MDO6t7RWR+m9VTO0dsXiCXyRYZzUrn2nnwYysYvUy5PdfaHF+4xQBN9hniAu49AQ
nHqfwaY6C1eDATLkqlDvrsFcwySr5sF2Ax6H3BiKV5seuQ5n+dpy6RdKBi+YDIBy
GBy4YEYwDkPyLmXPyNt3izY3rxmmob/aaoKBRpBJOMuuVDyKiqRNaUmIVkvMHAUb
S0UFQ+dMdsuak6z/ei/32p/vNvZvEuQG5s97FLDui9BpVa1Q8W5Lkamyy96I2WSQ
Y0iPQludnzm1TQ9FLYp0tJPTW4UTbWi9tyUn35d8vKsaNkqUTTAk/UH1FQvaQ2n6
tm2gtaXXQc6n1yD6xY29
=89fY
-----END PGP SIGNATURE-----



From bluethundr at gmail.com  Mon Aug 17 15:02:39 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Mon, 17 Aug 2015 11:02:39 -0400
Subject: can't log in or create new users on mediawiki
In-Reply-To: <55D17E2C.1030003@uplex.de>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
 <55D17E2C.1030003@uplex.de>
Message-ID: <CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>

Hi Geoff,

I don't know how it works with mediawiki, but let's talk about regexen
> again. %^)
>
> > req.http.host ~ "^wiki\.example\.com$"
>
> If you're going for an exact string match -- '^' followed by a fixed
> string followed by '$' -- you might as well just compare for string
> equality:
>
>         req.http.host == "wiki.example.com"
>
> That might be a little faster, and it makes the intention of your code
> much clearer.
>

Thanks! That makes sense, and I've done that in my next attempt, which I'll
show soon.

req.url ~
> >
> "^/index.php?title=Special:UserLogin&returnto=Main+Page&returntoquery=action%3Dsubmitlogin%26type%3Dlogin$"
>
> Same
> >
> thing here, with the additional problem that '.', '?' and '+' are
> regex metacharacters. "index.php?" will match "index.ph" possibly
> followed by a "p", but it won't match the question mark (it also
> matches anything where the dot is). "Main+" will match "Mai" followed
> one or more "n"s, but it won't match the plus sign.
>
> So your idea may or may not work with mediawiki, but at any rate the
> regex matches were failing.
>
> If you're sure you're looking for those exact strings, just use the
> '==' comparison.
>
> Are you certain that the URLs of interest will always be written
> exactly those two ways? No one will ever use '/' instead of
> '/index.php', and the key-value pairs in the query string will always
> be in exactly that order?
>
> If not, you'd be better off using regexen after all, something like this:
>
>         req.url ~ "^/(index\.php)?" && req.url ~ "foo=bar"
>         && req.url ~ "baz=quux"
>
> Meaning: the path must be '/' possibly followed by 'index.php', and
> the key-value pairs must all appear (in any order).
>
> Again, this is just to ensure that you're identifying the URLs you're
> looking for, I can't say if mediawiki will then do as you expect.
>

Ok so for my next attempt, I attempted to clean up the regex a little bit.
Hopefully this makes a little more sense:

  if ( req.http.host == "wiki.jokefire.com" && req.url ~
"^/(index\.php)?\?title=Special:UserLogin(.*)" ) {
    return (pass) ;
   }


  So what I'm doing here is an exact match for the host name and I'm using
a regex for the URL. The URL regex means that the string must begin with
either a slash or with index.php and after that match
"\?title=Special:UserLogin" exactly and then allow any combination of
characters after that. I do that because I think that mediawiki varies this
URL with some cookie info appended at the end. This theory makes sense to
me because I did make some progress with this approach. On the next
attempt, instead of having the page simply refresh when I try to log in, I
am now getting a new error from mediawiki:

Login error
JF Wiki uses cookies to log in users. You have cookies disabled. Please
enable them and try again.

So I think I need to find a way to preserve the cookie that's being used to
identify the user at the end of the URL, which I'm pretty sure will be a
random string. Can you help me find a way to do this?

Thanks,
Tim

On Mon, Aug 17, 2015 at 2:24 AM, Geoff Simmons <geoff at uplex.de> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 8/17/15 5:29 AM, Tim Dunphy wrote:
> >
> > I've setup my mediawiki behind varnish. Since doing that I am
> > unable to either create new users or log into the wiki itself.
>
> I don't know how it works with mediawiki, but let's talk about regexen
> again. %^)
>
> > req.http.host ~ "^wiki\.example\.com$"
>
> If you're going for an exact string match -- '^' followed by a fixed
> string followed by '$' -- you might as well just compare for string
> equality:
>
>         req.http.host == "wiki.example.com"
>
> That might be a little faster, and it makes the intention of your code
> much clearer.
>
> > req.url ~
> > "^/index.php?title=Special:UserLogin&returnto=Main+Page&type=signup"
>
> >
> >
> req.url ~
> >
> "^/index.php?title=Special:UserLogin&returnto=Main+Page&returntoquery=action%3Dsubmitlogin%26type%3Dlogin$"
>
> Same
> >
> thing here, with the additional problem that '.', '?' and '+' are
> regex metacharacters. "index.php?" will match "index.ph" possibly
> followed by a "p", but it won't match the question mark (it also
> matches anything where the dot is). "Main+" will match "Mai" followed
> one or more "n"s, but it won't match the plus sign.
>
> So your idea may or may not work with mediawiki, but at any rate the
> regex matches were failing.
>
> If you're sure you're looking for those exact strings, just use the
> '==' comparison.
>
> Are you certain that the URLs of interest will always be written
> exactly those two ways? No one will ever use '/' instead of
> '/index.php', and the key-value pairs in the query string will always
> be in exactly that order?
>
> If not, you'd be better off using regexen after all, something like this:
>
>         req.url ~ "^/(index\.php)?" && req.url ~ "foo=bar"
>         && req.url ~ "baz=quux"
>
> Meaning: the path must be '/' possibly followed by 'index.php', and
> the key-value pairs must all appear (in any order).
>
> Again, this is just to ensure that you're identifying the URLs you're
> looking for, I can't say if mediawiki will then do as you expect.
>
>
> HTH,
> Geoff
> - --
> UPLEX Systemoptimierung
> Scheffelstra?e 32
> 22301 Hamburg
> http://uplex.de/
> Mob: +49-176-63690917
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
>
> iQIcBAEBCAAGBQJV0X4rAAoJEOUwvh9pJNURKLsP/1qI5suvBWXW+BwwXtXmINQY
> vPU78Z8JfkdDeXSxmc3lDaazO0L/KE2IwsNnkP5Vl/kJ6EbI3sU8Eq7eXZ73v563
> W2d/poA6tlFKHuJovP/opGLhKu1Ls4qUGc74njywNTxiS+pqSvQKPx3OndoYEfyi
> /MiPkaPAbSfa7YVKMQ6dHlaEKXP6VJQ3JOJwWKQl4ehl9V1EP1Ddcd2ABHAGf2s+
> 2yaEINvDVRVek/QBkzgzZbN//KXjtpQdtELjSriNAAAUV+Oi1ULP9ROL0QE+IVYO
> Gqe9q0AxquOxn7D6DFgUyZmZWgEjDPy3V8JvNMI+drwTewhEk1g9A1yOI9v+5RJb
> MDO6t7RWR+m9VTO0dsXiCXyRYZzUrn2nnwYysYvUy5PdfaHF+4xQBN9hniAu49AQ
> nHqfwaY6C1eDATLkqlDvrsFcwySr5sF2Ax6H3BiKV5seuQ5n+dpy6RdKBi+YDIBy
> GBy4YEYwDkPyLmXPyNt3izY3rxmmob/aaoKBRpBJOMuuVDyKiqRNaUmIVkvMHAUb
> S0UFQ+dMdsuak6z/ei/32p/vNvZvEuQG5s97FLDui9BpVa1Q8W5Lkamyy96I2WSQ
> Y0iPQludnzm1TQ9FLYp0tJPTW4UTbWi9tyUn35d8vKsaNkqUTTAk/UH1FQvaQ2n6
> tm2gtaXXQc6n1yD6xY29
> =89fY
> -----END PGP SIGNATURE-----
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/6e363d14/attachment.html>

From lists at redlab.be  Mon Aug 17 15:27:42 2015
From: lists at redlab.be (Balder VC)
Date: Mon, 17 Aug 2015 17:27:42 +0200
Subject: can't log in or create new users on mediawiki
In-Reply-To: <CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
 <55D17E2C.1030003@uplex.de>
 <CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>
Message-ID: <55D1FD6E.4080004@redlab.be>

On 08/17/2015 05:02 PM, Tim Dunphy wrote:

>
> So I think I need to find a way to preserve the cookie that's being
used to identify the user at the end of the URL, which I'm pretty sure
will be a random string. Can you help me find a way to do this?
>
> Thanks,
> Tim
>
> On Mon, Aug 17, 2015 at 2:24 AM, Geoff Simmons <geoff at uplex.de
<mailto:geoff at uplex.de>> wrote:
>


maybe you have a  unset req.http.Cookie; that shouldn't be called when a
user is logged in/going to login.

Maybe try to find a way to always call (pass) when a user is logged in.
I think you 'd be better of not caching when a user is logged in.
Of course we don't know the use case of your mediawiki, nor do I know
mediawiki myself .


Regards
Balder
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/321c8435/attachment.html>

From bluethundr at gmail.com  Mon Aug 17 15:52:45 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Mon, 17 Aug 2015 11:52:45 -0400
Subject: can't log in or create new users on mediawiki
In-Reply-To: <55D1FD6E.4080004@redlab.be>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
 <55D17E2C.1030003@uplex.de>
 <CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>
 <55D1FD6E.4080004@redlab.be>
Message-ID: <CAOZy0enkLrnBo3ZciXeC2zDeCr8YK2uL5Zk-cLgmsvbLbzqmSw@mail.gmail.com>

>
> maybe you have a  unset req.http.Cookie; that shouldn't be called when a
> user is logged in/going to login.
> Maybe try to find a way to always call (pass) when a user is logged in. I
> think you 'd be better of not caching when a user is logged in.
> Of course we don't know the use case of your mediawiki, nor do I know
> mediawiki myself .


That was it! I did in fact have  unset req.http.Cookie; set earlier in my
VCL! So now I can log in. Good stuff! However if I disable  unset
req.http.Cookie; altogether my site does not cache at all. Is there any way
to allow cookies just for the login page in the following section?

 if ( req.http.host == "wiki.example.com" && req.url ~
 "^/(index\.php)?\?title=Special:UserLogin(.*)" ) {
    return (pass) ;
   }

Thanks,
TIm

On Mon, Aug 17, 2015 at 11:27 AM, Balder VC <lists at redlab.be> wrote:

> On 08/17/2015 05:02 PM, Tim Dunphy wrote:
>
> >
> > So I think I need to find a way to preserve the cookie that's being used
> to identify the user at the end of the URL, which I'm pretty sure will be a
> random string. Can you help me find a way to do this?
> >
> > Thanks,
> > Tim
> >
> > On Mon, Aug 17, 2015 at 2:24 AM, Geoff Simmons <geoff at uplex.de
> <mailto:geoff at uplex.de> <geoff at uplex.de>> wrote:
> >
>
>
> maybe you have a  unset req.http.Cookie; that shouldn't be called when a
> user is logged in/going to login.
>
> Maybe try to find a way to always call (pass) when a user is logged in. I
> think you 'd be better of not caching when a user is logged in.
> Of course we don't know the use case of your mediawiki, nor do I know
> mediawiki myself .
>
>
> Regards
> Balder
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/79dcc2df/attachment.html>

From lists at redlab.be  Mon Aug 17 16:31:02 2015
From: lists at redlab.be (Balder VC)
Date: Mon, 17 Aug 2015 18:31:02 +0200
Subject: can't log in or create new users on mediawiki
In-Reply-To: <CAOZy0enkLrnBo3ZciXeC2zDeCr8YK2uL5Zk-cLgmsvbLbzqmSw@mail.gmail.com>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>	<55D17E2C.1030003@uplex.de>	<CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>	<55D1FD6E.4080004@redlab.be>
 <CAOZy0enkLrnBo3ZciXeC2zDeCr8YK2uL5Zk-cLgmsvbLbzqmSw@mail.gmail.com>
Message-ID: <55D20C46.4010304@redlab.be>

As advised on mediawiki's wiki (
https://www.mediawiki.org/wiki/Manual:Varnish_caching )

 # Pass requests from logged-in users directly.
        if (req.http.Authorization || req.http.Cookie)
           {return(pass);}      /* Not cacheable by default */
 

you could also check req.http.Cookie's content to check if the session
cookie is there, and not unset them in this case. In other cases when
user is not logged in, it's probably safe to unset them and show the
static version of the page from cache, instead of the logged in version.




Regards
Balder

On 08/17/2015 05:52 PM, Tim Dunphy wrote:
>
>     maybe you have a  unset req.http.Cookie; that shouldn't be called
>     when a user is logged in/going to login.
>     Maybe try to find a way to always call (pass) when a user is
>     logged in. I think you 'd be better of not caching when a user is
>     logged in.
>     Of course we don't know the use case of your mediawiki, nor do I
>     know mediawiki myself . 
>
>
> That was it! I did in fact have  unset req.http.Cookie; set earlier in
> my VCL! So now I can log in. Good stuff! However if I disable  unset
> req.http.Cookie; altogether my site does not cache at all. Is there
> any way to allow cookies just for the login page in the following section?
>
>  if ( req.http.host == "wiki.example.com <http://wiki.example.com>" &&
> req.url ~  "^/(index\.php)?\?title=Special:UserLogin(.*)" ) {
>     return (pass) ;
>    }
>
> Thanks,
> TIm
>
> On Mon, Aug 17, 2015 at 11:27 AM, Balder VC <lists at redlab.be
> <mailto:lists at redlab.be>> wrote:
>
>     On 08/17/2015 05:02 PM, Tim Dunphy wrote:
>
>     >
>     > So I think I need to find a way to preserve the cookie that's
>     being used to identify the user at the end of the URL, which I'm
>     pretty sure will be a random string. Can you help me find a way to
>     do this?
>     >
>     > Thanks,
>     > Tim
>     >
>     > On Mon, Aug 17, 2015 at 2:24 AM, Geoff Simmons <geoff at uplex.de
>     <mailto:geoff at uplex.de> <mailto:geoff at uplex.de>
>     <mailto:geoff at uplex.de>> wrote:
>     >
>
>
>     maybe you have a  unset req.http.Cookie; that shouldn't be called
>     when a user is logged in/going to login.
>
>     Maybe try to find a way to always call (pass) when a user is
>     logged in. I think you 'd be better of not caching when a user is
>     logged in.
>     Of course we don't know the use case of your mediawiki, nor do I
>     know mediawiki myself .
>
>
>     Regards
>     Balder
>
>     _______________________________________________
>     varnish-misc mailing list
>     varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org>
>     https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
>
>
> -- 
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net
> <http://pool.sks-keyservers.net> --recv-keys F186197B
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/cfbce5ca/attachment.html>

From phk at phk.freebsd.dk  Mon Aug 17 16:47:48 2015
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 17 Aug 2015 16:47:48 +0000
Subject: can't log in or create new users on mediawiki
In-Reply-To: <55D20C46.4010304@redlab.be>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
 <55D17E2C.1030003@uplex.de>
 <CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>
 <55D1FD6E.4080004@redlab.be>
 <CAOZy0enkLrnBo3ZciXeC2zDeCr8YK2uL5Zk-cLgmsvbLbzqmSw@mail.gmail.com>
 <55D20C46.4010304@redlab.be>
Message-ID: <80739.1439830068@critter.freebsd.dk>

--------

I think I saw a mediawiki/wikimedia (I can never figure out how
that split is supposed to work...) page about setting up Varnish
in front.  Have you checked if there's a explanation there ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From bluethundr at gmail.com  Mon Aug 17 19:43:54 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Mon, 17 Aug 2015 15:43:54 -0400
Subject: can't log in or create new users on mediawiki
In-Reply-To: <80739.1439830068@critter.freebsd.dk>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
 <55D17E2C.1030003@uplex.de>
 <CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>
 <55D1FD6E.4080004@redlab.be>
 <CAOZy0enkLrnBo3ZciXeC2zDeCr8YK2uL5Zk-cLgmsvbLbzqmSw@mail.gmail.com>
 <55D20C46.4010304@redlab.be> <80739.1439830068@critter.freebsd.dk>
Message-ID: <CAOZy0e=7hZ7Jq7hwj9Uw76DQ7OtZENsry2vWtGxpQNoet=L0Dg@mail.gmail.com>

I based my vanrish config on these recommendations from mediawiki:

https://www.mediawiki.org/wiki/Manual:Varnish_caching

And I do have that setting in my VLC that bypasses the cache for any
cookies it finds:

     # Pass requests from logged-in users directly.
     if (req.http.Authorization || req.http.Cookie)
           {return(pass);}      /* Not cacheable by default */

you could also check req.http.Cookie's content to check if the session
cookie is there, and not unset them in this case. In other cases when user
is not logged in, it's probably safe to unset them and show the static
version of the page from cache, instead of the logged in version.


That's a good suggestion. I'll look into how to do that!

In the meantime, I'll enclose my current VCL as an attachment, in case
there are any other suggestions based on what I have in the config.

Thanks
Tim


On Mon, Aug 17, 2015 at 12:47 PM, Poul-Henning Kamp <phk at phk.freebsd.dk>
wrote:

> --------
>
> I think I saw a mediawiki/wikimedia (I can never figure out how
> that split is supposed to work...) page about setting up Varnish
> in front.  Have you checked if there's a explanation there ?
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/02d2fc62/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: default.vcl
Type: application/octet-stream
Size: 7883 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/02d2fc62/attachment.obj>

From feld at feld.me  Mon Aug 17 20:06:36 2015
From: feld at feld.me (Mark Felder)
Date: Mon, 17 Aug 2015 15:06:36 -0500
Subject: can't log in or create new users on mediawiki
In-Reply-To: <CAOZy0e=7hZ7Jq7hwj9Uw76DQ7OtZENsry2vWtGxpQNoet=L0Dg@mail.gmail.com>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
 <55D17E2C.1030003@uplex.de>
 <CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>
 <55D1FD6E.4080004@redlab.be>
 <CAOZy0enkLrnBo3ZciXeC2zDeCr8YK2uL5Zk-cLgmsvbLbzqmSw@mail.gmail.com>
 <55D20C46.4010304@redlab.be> <80739.1439830068@critter.freebsd.dk>
 <CAOZy0e=7hZ7Jq7hwj9Uw76DQ7OtZENsry2vWtGxpQNoet=L0Dg@mail.gmail.com>
Message-ID: <1439841996.3592452.358573009.2557A7E4@webmail.messagingengine.com>



On Mon, Aug 17, 2015, at 14:43, Tim Dunphy wrote:
> I based my vanrish config on these recommendations from mediawiki:
> 
> https://www.mediawiki.org/wiki/Manual:Varnish_caching
> 
> And I do have that setting in my VLC that bypasses the cache for any
> cookies it finds:
> 
>      # Pass requests from logged-in users directly.
>      if (req.http.Authorization || req.http.Cookie)
>            {return(pass);}      /* Not cacheable by default */
> 

I've modified their config slightly because I ran into a few issues, but
I can't recall exactly what they were. This has worked pretty well for
me -- anonymous users get data served from cache, logged-in users get
images cached but page content is not.

I really only use my wiki once a year, but it gets heavy usage (few
hundred people making edits for 3 days straight) and has had good hit
rates and no complaints.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mediawiki.vcl
Type: application/octet-stream
Size: 4519 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/9d418b9c/attachment.obj>

From bluethundr at gmail.com  Mon Aug 17 20:15:41 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Mon, 17 Aug 2015 16:15:41 -0400
Subject: can't log in or create new users on mediawiki
In-Reply-To: <1439841996.3592452.358573009.2557A7E4@webmail.messagingengine.com>
References: <CAOZy0e=CN+28RF=rA49zWcnN_WEkqmtOT2Bg8t-ZJNq3xfv+qQ@mail.gmail.com>
 <55D17E2C.1030003@uplex.de>
 <CAOZy0e=GMyVANuhy4NmGCec2kmu--_zNwRYYT0cy_t=NLXCpbg@mail.gmail.com>
 <55D1FD6E.4080004@redlab.be>
 <CAOZy0enkLrnBo3ZciXeC2zDeCr8YK2uL5Zk-cLgmsvbLbzqmSw@mail.gmail.com>
 <55D20C46.4010304@redlab.be> <80739.1439830068@critter.freebsd.dk>
 <CAOZy0e=7hZ7Jq7hwj9Uw76DQ7OtZENsry2vWtGxpQNoet=L0Dg@mail.gmail.com>
 <1439841996.3592452.358573009.2557A7E4@webmail.messagingengine.com>
Message-ID: <CAOZy0ekTAH2WYaQ1waZ-ZjAh2=CUM1=xDSejbpfvWwM9-0R41g@mail.gmail.com>

Hey Mark,

 That's pretty awesome! Thanks, I'll check it out.

Tim

On Mon, Aug 17, 2015 at 4:06 PM, Mark Felder <feld at feld.me> wrote:

>
>
> On Mon, Aug 17, 2015, at 14:43, Tim Dunphy wrote:
> > I based my vanrish config on these recommendations from mediawiki:
> >
> > https://www.mediawiki.org/wiki/Manual:Varnish_caching
> >
> > And I do have that setting in my VLC that bypasses the cache for any
> > cookies it finds:
> >
> >      # Pass requests from logged-in users directly.
> >      if (req.http.Authorization || req.http.Cookie)
> >            {return(pass);}      /* Not cacheable by default */
> >
>
> I've modified their config slightly because I ran into a few issues, but
> I can't recall exactly what they were. This has worked pretty well for
> me -- anonymous users get data served from cache, logged-in users get
> images cached but page content is not.
>
> I really only use my wiki once a year, but it gets heavy usage (few
> hundred people making edits for 3 days straight) and has had good hit
> rates and no complaints.
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150817/998f30cb/attachment.html>

From thierry.leveque at imetrik.com  Wed Aug 19 15:36:05 2015
From: thierry.leveque at imetrik.com (Thierry Leveque)
Date: Wed, 19 Aug 2015 15:36:05 +0000
Subject: Feature question
Message-ID: <386A4826-D645-44CD-9262-48932F3EE7AA@imetrik.com>

Hi,

I just came across your website when I was searching about caching product.
And I am wondering if Varnish can support our needs.

Our need is really simple. We want to remove CPU load on a web service server.
If the same request is done to the web service, we want it to be process only once.
But the things here is that those same requests can be received within the same second. And the process from the web service can take several seconds. That mean that we want a cache service that when it is receiving the second identical request, will wait for the answer of the first one before using the answer.
So even if 10 identical request are send within a few millisecond, we want only one real request to reach the web service.

Is something like that possible with Varnish? Is my explanation clear enough?

Thanks

Thierry L?v?que
D?veloppeur S?nior / Senior Developer
iMetrik Global inc.
T : +1 514 448-6407 poste ????
T : +1 866 276-5382 (sans frais)
F : +1 514 904-0611
C : +1 514 241-5734

thierry.leveque at imetrik.com<mailto:pr%C3%A9nom.nom at imetrik.com>
740, rue Notre-Dame Ouest
Suite 1575 , Montr?al QC
H3C 3X6
www.imetrik.com<http://www.imetrik.com/>

Ce message et tous les fichiers attach?s sont confidentiels et s'adressent exclusivement au(x) destinataire(s) indiqu?(s) ci-dessus. Si vous recevez ce message par erreur, veuillez le retourner ? l'exp?diteur et le d?truire. Tout usage, copie ou distribution non autoris?s de ce courriel sont strictement interdits.

This message and any attachments are confidential and intended solely for the above-mentioned recipient(s). If you have received this message in error, please return it to the sender and delete it from your files. Unauthorized use, copying or distribution of this e-mail is strictly prohibited.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150819/ce41881e/attachment.html>

From thomas.lecomte at virtual-expo.com  Wed Aug 19 20:53:07 2015
From: thomas.lecomte at virtual-expo.com (Thomas Lecomte)
Date: Wed, 19 Aug 2015 22:53:07 +0200
Subject: Feature question
In-Reply-To: <386A4826-D645-44CD-9262-48932F3EE7AA@imetrik.com>
References: <386A4826-D645-44CD-9262-48932F3EE7AA@imetrik.com>
Message-ID: <CAJban9B9GVBCgNSJYLiZJ5-krnq2A2uD17VmuHbjUPWdRG40pQ@mail.gmail.com>

On Wed, Aug 19, 2015 at 5:36 PM, Thierry Leveque
<thierry.leveque at imetrik.com> wrote:
> Hi,
>
> [...]
>
> Is something like that possible with Varnish? Is my explanation clear
> enough?

Hello Thierry,

It is very clear, and indeed, that is how Varnish works. On the very
first request, an object will be created in the cache, and will be
marked as 'busy' until it is fully retrieved from the backend.
All the incoming requests for the same object will be queued, and as
soon as it will be fetched for the first request, it will be served to
all the waiting clients, which means the backend will see the request
only one time.

Regards,

-- 
Thomas Lecomte



From lkarsten at varnish-software.com  Thu Aug 20 09:01:33 2015
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 20 Aug 2015 11:01:33 +0200
Subject: Feature question
In-Reply-To: <CAJban9B9GVBCgNSJYLiZJ5-krnq2A2uD17VmuHbjUPWdRG40pQ@mail.gmail.com>
References: <386A4826-D645-44CD-9262-48932F3EE7AA@imetrik.com>
 <CAJban9B9GVBCgNSJYLiZJ5-krnq2A2uD17VmuHbjUPWdRG40pQ@mail.gmail.com>
Message-ID: <20150820090132.GB3475@immer.varnish-software.com>

On Wed, Aug 19, 2015 at 10:53:07PM +0200, Thomas Lecomte wrote:
> On Wed, Aug 19, 2015 at 5:36 PM, Thierry Leveque
> <thierry.leveque at imetrik.com> wrote:
> > [...]
> > Is something like that possible with Varnish? Is my explanation clear
> > enough?
> It is very clear, and indeed, that is how Varnish works. On the very
> first request, an object will be created in the cache, and will be
> marked as 'busy' until it is fully retrieved from the backend.
> All the incoming requests for the same object will be queued, and as
> soon as it will be fetched for the first request, it will be served to
> all the waiting clients, which means the backend will see the request
> only one time.

It can also be added that when the object TTL expires, Varnish will do
a conditional request asking the backend if the cached version is still
current.

While this backend refresh is ongoing, any clients requesting the object
in the meantime will receive the (slightly) stale/expired copy. This
means that there is no waiting/delay before serving clients in normal
operation.

This applies for Varnish 4.0 and is of course configurable.

-- 
Lasse Karstensen
Varnish Software AS



From yamakasi.014 at gmail.com  Sun Aug 23 23:41:10 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Mon, 24 Aug 2015 01:41:10 +0200
Subject: Rewrite domain and keep subdomain
Message-ID: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>

Hi Guys,

I'm figuring out how I can rewrite a domain in a req.http.Host

Like I type in my  addressbar: subdomain.myFIRSTdomain.tld

Where Varnish needs to catch this using req.http.Host filter out the
domainname and make req.http.Host actually:

subdomain.mySECONDdomain.tld

I'm figuring out how to do so, but I can't find the right way/docs.

I hope someone can help me out !

Thanks,

Matt



From yamakasi.014 at gmail.com  Mon Aug 24 07:05:50 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Mon, 24 Aug 2015 09:05:50 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <20150824063732.GI14788@suricate.otoh.org>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
 <20150824063732.GI14788@suricate.otoh.org>
Message-ID: <CAPNQp06SfGT9cZH4CshJWFkwyF7cpyYFXCo+u+f1ATtyRLd1JA@mail.gmail.com>

Hi Paul,

Thanks,that looks good. But what if I don't know which subdomain I
will receive ? The domains are known but the subdomains may differ.

I was thinking about distracting it from FIRST.tld and after that
attaching it again to the + SECOND.tld

Cheers,

Matt



2015-08-24 8:37 GMT+02:00 Paul Armstrong <varnish-cache at otoh.org>:
> At 2015-08-24T01:41+0200, Matt . wrote:
>> I'm figuring out how I can rewrite a domain in a req.http.Host
>>
>> Like I type in my  addressbar: subdomain.myFIRSTdomain.tld
>>
>> Where Varnish needs to catch this using req.http.Host filter out the
>> domainname and make req.http.Host actually:
>>
>> subdomain.mySECONDdomain.tld
>>
>> I'm figuring out how to do so, but I can't find the right way/docs.
>
> Putting the following in vcl_recv should do it:
> set req.http.Host = regsub(req.http.Host, "subdomain.myFIRSTdomain.tld",
> "subdomain.mySECONDdomain.tld?);
>
> Paul



From dridi at varni.sh  Mon Aug 24 07:59:28 2015
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Mon, 24 Aug 2015 09:59:28 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
Message-ID: <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>

On Mon, Aug 24, 2015 at 1:41 AM, Matt . <yamakasi.014 at gmail.com> wrote:
> Hi Guys,
>
> I'm figuring out how I can rewrite a domain in a req.http.Host
>
> Like I type in my  addressbar: subdomain.myFIRSTdomain.tld
>
> Where Varnish needs to catch this using req.http.Host filter out the
> domainname and make req.http.Host actually:
>
> subdomain.mySECONDdomain.tld
>
> I'm figuring out how to do so, but I can't find the right way/docs.

Hi Matt,

This can be achieved with a regular expression, please find enclosed a
test case demonstrating how to do that.

Best Regards,
Dridi

> I hope someone can help me out !
>
> Thanks,
>
> Matt
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: subdomain.vtc
Type: application/octet-stream
Size: 478 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150824/eb4f1cae/attachment.obj>

From yamakasi.014 at gmail.com  Mon Aug 24 09:27:56 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Mon, 24 Aug 2015 11:27:56 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
 <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>
Message-ID: <CAPNQp05i4EkTPqKQ=4yXOYsneqUGvfYNzx3TJhU3NT5ZFmU9Yg@mail.gmail.com>

Hi Dridi,

I'm not 100% sure about the example.

The idea is that when I have

domain1.tld
domain2.tld

acceces with subdomainX.domain1.tld or subdomainY.domain2.tld or
whatever.domain1.tld I set it in my vcl to

TheVisitedSubdomain.mymaindomain.tld

And use that one in my VLC...

Thanks so far!

Matt



2015-08-24 9:59 GMT+02:00 Dridi Boukelmoune <dridi at varni.sh>:
> On Mon, Aug 24, 2015 at 1:41 AM, Matt . <yamakasi.014 at gmail.com> wrote:
>> Hi Guys,
>>
>> I'm figuring out how I can rewrite a domain in a req.http.Host
>>
>> Like I type in my  addressbar: subdomain.myFIRSTdomain.tld
>>
>> Where Varnish needs to catch this using req.http.Host filter out the
>> domainname and make req.http.Host actually:
>>
>> subdomain.mySECONDdomain.tld
>>
>> I'm figuring out how to do so, but I can't find the right way/docs.
>
> Hi Matt,
>
> This can be achieved with a regular expression, please find enclosed a
> test case demonstrating how to do that.
>
> Best Regards,
> Dridi
>
>> I hope someone can help me out !
>>
>> Thanks,
>>
>> Matt
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From yamakasi.014 at gmail.com  Mon Aug 24 17:40:09 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Mon, 24 Aug 2015 19:40:09 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <CA+8iBq-oM4WbxYET+tSvSCOqRzYwha6OgT_HXeVScXG+pFW=rA@mail.gmail.com>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
 <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>
 <CAPNQp05i4EkTPqKQ=4yXOYsneqUGvfYNzx3TJhU3NT5ZFmU9Yg@mail.gmail.com>
 <CA+8iBq-oM4WbxYET+tSvSCOqRzYwha6OgT_HXeVScXG+pFW=rA@mail.gmail.com>
Message-ID: <CAPNQp06Ux9stETHewvF83w=1ojZhnyAEyOEApZvZydbt7D5FjA@mail.gmail.com>

Hi,

Nope I can't as Varnish does need to handle all.

2015-08-24 17:56 GMT+02:00 nick tailor <nick.tailor at gmail.com>:
> Couldn't you just use a rewrite rule in .htaccess? and then let varnish
> handle the actual domain you want served?
>
> Cheers
>
> Nick Tailor
> nicktailor.com
>
> On Mon, Aug 24, 2015 at 2:27 AM, Matt . <yamakasi.014 at gmail.com> wrote:
>>
>> Hi Dridi,
>>
>> I'm not 100% sure about the example.
>>
>> The idea is that when I have
>>
>> domain1.tld
>> domain2.tld
>>
>> acceces with subdomainX.domain1.tld or subdomainY.domain2.tld or
>> whatever.domain1.tld I set it in my vcl to
>>
>> TheVisitedSubdomain.mymaindomain.tld
>>
>> And use that one in my VLC...
>>
>> Thanks so far!
>>
>> Matt
>>
>>
>>
>> 2015-08-24 9:59 GMT+02:00 Dridi Boukelmoune <dridi at varni.sh>:
>> > On Mon, Aug 24, 2015 at 1:41 AM, Matt . <yamakasi.014 at gmail.com> wrote:
>> >> Hi Guys,
>> >>
>> >> I'm figuring out how I can rewrite a domain in a req.http.Host
>> >>
>> >> Like I type in my  addressbar: subdomain.myFIRSTdomain.tld
>> >>
>> >> Where Varnish needs to catch this using req.http.Host filter out the
>> >> domainname and make req.http.Host actually:
>> >>
>> >> subdomain.mySECONDdomain.tld
>> >>
>> >> I'm figuring out how to do so, but I can't find the right way/docs.
>> >
>> > Hi Matt,
>> >
>> > This can be achieved with a regular expression, please find enclosed a
>> > test case demonstrating how to do that.
>> >
>> > Best Regards,
>> > Dridi
>> >
>> >> I hope someone can help me out !
>> >>
>> >> Thanks,
>> >>
>> >> Matt
>> >>
>> >> _______________________________________________
>> >> varnish-misc mailing list
>> >> varnish-misc at varnish-cache.org
>> >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
>
>
> --
> Nick Tailor
>
> Senior Linux Systems Administrator
> BCLC, 2940 Virtual Way, Vancouver B.C. V5M 0A6
> T 604 228 3046  C 778 388 1397
>
> Connect with us:
> Twitter @BCLC| Twitter @BCLCGameSense | YouTube | Blog | bclc.com
>
>  Last year, more than $1 billion generated by BCLC gambling activities went
> back into health care, education and community groups across B.C.



From yamakasi.014 at gmail.com  Mon Aug 24 19:45:06 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Mon, 24 Aug 2015 21:45:06 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <CA+8iBq-TJM5X4ax1uDh-yU0404ZzrFYCFx26h-MRMAKdfeg5hQ@mail.gmail.com>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
 <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>
 <CAPNQp05i4EkTPqKQ=4yXOYsneqUGvfYNzx3TJhU3NT5ZFmU9Yg@mail.gmail.com>
 <CA+8iBq-oM4WbxYET+tSvSCOqRzYwha6OgT_HXeVScXG+pFW=rA@mail.gmail.com>
 <CAPNQp06Ux9stETHewvF83w=1ojZhnyAEyOEApZvZydbt7D5FjA@mail.gmail.com>
 <CA+8iBq-TJM5X4ax1uDh-yU0404ZzrFYCFx26h-MRMAKdfeg5hQ@mail.gmail.com>
Message-ID: <CAPNQp07OoqRmLYOoHyPZXRYu5ep=nf9kKzDHOxPKkv8j0oTx_g@mail.gmail.com>

Hi,

Yes I found that too, wasn't sure about the expression.

Cheers,

Matt

2015-08-24 19:47 GMT+02:00 nick tailor <nick.tailor at gmail.com>:
> Okay,
>
> Well not sure what your googling? :)
>
> But this is what you want to do, hope that helps.
>
> https://www.varnish-software.com/book/3/build/exercises/complete-rewrite_urls_and_headers.html?highlight=redirect
>
> Cheers
>
> Nick Tailor
> nicktailor.com
>
> On Mon, Aug 24, 2015 at 10:40 AM, Matt . <yamakasi.014 at gmail.com> wrote:
>>
>> Hi,
>>
>> Nope I can't as Varnish does need to handle all.
>>
>> 2015-08-24 17:56 GMT+02:00 nick tailor <nick.tailor at gmail.com>:
>> > Couldn't you just use a rewrite rule in .htaccess? and then let varnish
>> > handle the actual domain you want served?
>> >
>> > Cheers
>> >
>> > Nick Tailor
>> > nicktailor.com
>> >
>> > On Mon, Aug 24, 2015 at 2:27 AM, Matt . <yamakasi.014 at gmail.com> wrote:
>> >>
>> >> Hi Dridi,
>> >>
>> >> I'm not 100% sure about the example.
>> >>
>> >> The idea is that when I have
>> >>
>> >> domain1.tld
>> >> domain2.tld
>> >>
>> >> acceces with subdomainX.domain1.tld or subdomainY.domain2.tld or
>> >> whatever.domain1.tld I set it in my vcl to
>> >>
>> >> TheVisitedSubdomain.mymaindomain.tld
>> >>
>> >> And use that one in my VLC...
>> >>
>> >> Thanks so far!
>> >>
>> >> Matt
>> >>
>> >>
>> >>
>> >> 2015-08-24 9:59 GMT+02:00 Dridi Boukelmoune <dridi at varni.sh>:
>> >> > On Mon, Aug 24, 2015 at 1:41 AM, Matt . <yamakasi.014 at gmail.com>
>> >> > wrote:
>> >> >> Hi Guys,
>> >> >>
>> >> >> I'm figuring out how I can rewrite a domain in a req.http.Host
>> >> >>
>> >> >> Like I type in my  addressbar: subdomain.myFIRSTdomain.tld
>> >> >>
>> >> >> Where Varnish needs to catch this using req.http.Host filter out the
>> >> >> domainname and make req.http.Host actually:
>> >> >>
>> >> >> subdomain.mySECONDdomain.tld
>> >> >>
>> >> >> I'm figuring out how to do so, but I can't find the right way/docs.
>> >> >
>> >> > Hi Matt,
>> >> >
>> >> > This can be achieved with a regular expression, please find enclosed
>> >> > a
>> >> > test case demonstrating how to do that.
>> >> >
>> >> > Best Regards,
>> >> > Dridi
>> >> >
>> >> >> I hope someone can help me out !
>> >> >>
>> >> >> Thanks,
>> >> >>
>> >> >> Matt
>> >> >>
>> >> >> _______________________________________________
>> >> >> varnish-misc mailing list
>> >> >> varnish-misc at varnish-cache.org
>> >> >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>> >>
>> >> _______________________________________________
>> >> varnish-misc mailing list
>> >> varnish-misc at varnish-cache.org
>> >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>> >
>> >
>> >
>> >
>> > --
>> > Nick Tailor
>> >
>> > Senior Linux Systems Administrator
>> > BCLC, 2940 Virtual Way, Vancouver B.C. V5M 0A6
>> > T 604 228 3046  C 778 388 1397
>> >
>> > Connect with us:
>> > Twitter @BCLC| Twitter @BCLCGameSense | YouTube | Blog | bclc.com
>> >
>> >  Last year, more than $1 billion generated by BCLC gambling activities
>> > went
>> > back into health care, education and community groups across B.C.
>
>
>
>
> --
> Nick Tailor
>
> Senior Linux Systems Administrator
> BCLC, 2940 Virtual Way, Vancouver B.C. V5M 0A6
> T 604 228 3046  C 778 388 1397
>
> Connect with us:
> Twitter @BCLC| Twitter @BCLCGameSense | YouTube | Blog | bclc.com
>
>  Last year, more than $1 billion generated by BCLC gambling activities went
> back into health care, education and community groups across B.C.



From dridi at varni.sh  Tue Aug 25 08:42:22 2015
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Tue, 25 Aug 2015 10:42:22 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <CAPNQp05i4EkTPqKQ=4yXOYsneqUGvfYNzx3TJhU3NT5ZFmU9Yg@mail.gmail.com>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
 <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>
 <CAPNQp05i4EkTPqKQ=4yXOYsneqUGvfYNzx3TJhU3NT5ZFmU9Yg@mail.gmail.com>
Message-ID: <CABoVN9CBhaGojWP_Xih8rA-AMSR1RDSfwnCdijCjNuym+uNEcw@mail.gmail.com>

Hi Matt,

I believe that's what my example does. It is a test case, you can run
it with varnishtest.

Attached is the same test case, with a second client requesting the
other domain.

Dridi

On Mon, Aug 24, 2015 at 11:27 AM, Matt . <yamakasi.014 at gmail.com> wrote:
> Hi Dridi,
>
> I'm not 100% sure about the example.
>
> The idea is that when I have
>
> domain1.tld
> domain2.tld
>
> acceces with subdomainX.domain1.tld or subdomainY.domain2.tld or
> whatever.domain1.tld I set it in my vcl to
>
> TheVisitedSubdomain.mymaindomain.tld
>
> And use that one in my VLC...
>
> Thanks so far!
>
> Matt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: subdomain.vtc
Type: application/octet-stream
Size: 665 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150825/c73e5563/attachment.obj>

From yamakasi.014 at gmail.com  Tue Aug 25 12:55:40 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Tue, 25 Aug 2015 14:55:40 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <CABoVN9CBhaGojWP_Xih8rA-AMSR1RDSfwnCdijCjNuym+uNEcw@mail.gmail.com>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
 <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>
 <CAPNQp05i4EkTPqKQ=4yXOYsneqUGvfYNzx3TJhU3NT5ZFmU9Yg@mail.gmail.com>
 <CABoVN9CBhaGojWP_Xih8rA-AMSR1RDSfwnCdijCjNuym+uNEcw@mail.gmail.com>
Message-ID: <CAPNQp04gDUDwMjzhzD6MXvU2Vji526F=9GamMz-DOhFdOxrpmw@mail.gmail.com>

Hi Dridi,

Indeed, that works, I never tested that was as I test real code on dev machines.

That works! but now it really rewrites, but the best would be to
redirect because of certificates that don't match.

I would make sure domain one is accessed @ port 80, redirected to
domain2 at 443 as it has a valid certificate.

That should be doable ?

Thanks!

Matt

2015-08-25 10:42 GMT+02:00 Dridi Boukelmoune <dridi at varni.sh>:
> Hi Matt,
>
> I believe that's what my example does. It is a test case, you can run
> it with varnishtest.
>
> Attached is the same test case, with a second client requesting the
> other domain.
>
> Dridi
>
> On Mon, Aug 24, 2015 at 11:27 AM, Matt . <yamakasi.014 at gmail.com> wrote:
>> Hi Dridi,
>>
>> I'm not 100% sure about the example.
>>
>> The idea is that when I have
>>
>> domain1.tld
>> domain2.tld
>>
>> acceces with subdomainX.domain1.tld or subdomainY.domain2.tld or
>> whatever.domain1.tld I set it in my vcl to
>>
>> TheVisitedSubdomain.mymaindomain.tld
>>
>> And use that one in my VLC...
>>
>> Thanks so far!
>>
>> Matt



From dridi at varni.sh  Tue Aug 25 13:58:09 2015
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Tue, 25 Aug 2015 15:58:09 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <CAPNQp04gDUDwMjzhzD6MXvU2Vji526F=9GamMz-DOhFdOxrpmw@mail.gmail.com>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
 <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>
 <CAPNQp05i4EkTPqKQ=4yXOYsneqUGvfYNzx3TJhU3NT5ZFmU9Yg@mail.gmail.com>
 <CABoVN9CBhaGojWP_Xih8rA-AMSR1RDSfwnCdijCjNuym+uNEcw@mail.gmail.com>
 <CAPNQp04gDUDwMjzhzD6MXvU2Vji526F=9GamMz-DOhFdOxrpmw@mail.gmail.com>
Message-ID: <CABoVN9C+d5+i0Kj5L-mAOOP37pLQ2wQJmDOfBbnqaoUDi4d4Lw@mail.gmail.com>

On Tue, Aug 25, 2015 at 2:55 PM, Matt . <yamakasi.014 at gmail.com> wrote:
> Hi Dridi,
>
> Indeed, that works, I never tested that was as I test real code on dev machines.
>
> That works! but now it really rewrites, but the best would be to
> redirect because of certificates that don't match.
>
> I would make sure domain one is accessed @ port 80, redirected to
> domain2 at 443 as it has a valid certificate.
>
> That should be doable ?

Yes, you can easily do that with Varnish, but you should get familiar
with VCL and Varnish first.

There are other considerations like making sure you don't generate an
infinite loop of redirections etc.

> Thanks!
>
> Matt
>
> 2015-08-25 10:42 GMT+02:00 Dridi Boukelmoune <dridi at varni.sh>:
>> Hi Matt,
>>
>> I believe that's what my example does. It is a test case, you can run
>> it with varnishtest.
>>
>> Attached is the same test case, with a second client requesting the
>> other domain.
>>
>> Dridi
>>
>> On Mon, Aug 24, 2015 at 11:27 AM, Matt . <yamakasi.014 at gmail.com> wrote:
>>> Hi Dridi,
>>>
>>> I'm not 100% sure about the example.
>>>
>>> The idea is that when I have
>>>
>>> domain1.tld
>>> domain2.tld
>>>
>>> acceces with subdomainX.domain1.tld or subdomainY.domain2.tld or
>>> whatever.domain1.tld I set it in my vcl to
>>>
>>> TheVisitedSubdomain.mymaindomain.tld
>>>
>>> And use that one in my VLC...
>>>
>>> Thanks so far!
>>>
>>> Matt



From yamakasi.014 at gmail.com  Tue Aug 25 14:39:54 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Tue, 25 Aug 2015 16:39:54 +0200
Subject: Rewrite domain and keep subdomain
In-Reply-To: <CABoVN9C+d5+i0Kj5L-mAOOP37pLQ2wQJmDOfBbnqaoUDi4d4Lw@mail.gmail.com>
References: <CAPNQp07K1CC7u55nn4HGbYXyYD0Ck6NOEtBN9OXP4ixEqKgKtQ@mail.gmail.com>
 <CABoVN9AjGyQPG-bW88LKbrhbg05Mu4w92o_G1pUV-NrjGUJa=Q@mail.gmail.com>
 <CAPNQp05i4EkTPqKQ=4yXOYsneqUGvfYNzx3TJhU3NT5ZFmU9Yg@mail.gmail.com>
 <CABoVN9CBhaGojWP_Xih8rA-AMSR1RDSfwnCdijCjNuym+uNEcw@mail.gmail.com>
 <CAPNQp04gDUDwMjzhzD6MXvU2Vji526F=9GamMz-DOhFdOxrpmw@mail.gmail.com>
 <CABoVN9C+d5+i0Kj5L-mAOOP37pLQ2wQJmDOfBbnqaoUDi4d4Lw@mail.gmail.com>
Message-ID: <CAPNQp06cgBF6Egc6qx1pJJrfy5yYz8-CCsVP6XuzmrUPB30zgg@mail.gmail.com>

Hi Dridi,

Yes, I'm using Varnish already this way but I'm now about to "change"
the url in the adres bar first by using a 302 in some way to domain2.

What you decribe is what I already do but I need to redirect first.

Like

A6.audi.com -> A6.car.com
325.bmw.com -> 325.car.com

etc etc.

So writing this thread I need a 302 that way before I go furter in my VCL.

Thanks!

Matt

2015-08-25 15:58 GMT+02:00 Dridi Boukelmoune <dridi at varni.sh>:
> On Tue, Aug 25, 2015 at 2:55 PM, Matt . <yamakasi.014 at gmail.com> wrote:
>> Hi Dridi,
>>
>> Indeed, that works, I never tested that was as I test real code on dev machines.
>>
>> That works! but now it really rewrites, but the best would be to
>> redirect because of certificates that don't match.
>>
>> I would make sure domain one is accessed @ port 80, redirected to
>> domain2 at 443 as it has a valid certificate.
>>
>> That should be doable ?
>
> Yes, you can easily do that with Varnish, but you should get familiar
> with VCL and Varnish first.
>
> There are other considerations like making sure you don't generate an
> infinite loop of redirections etc.
>
>> Thanks!
>>
>> Matt
>>
>> 2015-08-25 10:42 GMT+02:00 Dridi Boukelmoune <dridi at varni.sh>:
>>> Hi Matt,
>>>
>>> I believe that's what my example does. It is a test case, you can run
>>> it with varnishtest.
>>>
>>> Attached is the same test case, with a second client requesting the
>>> other domain.
>>>
>>> Dridi
>>>
>>> On Mon, Aug 24, 2015 at 11:27 AM, Matt . <yamakasi.014 at gmail.com> wrote:
>>>> Hi Dridi,
>>>>
>>>> I'm not 100% sure about the example.
>>>>
>>>> The idea is that when I have
>>>>
>>>> domain1.tld
>>>> domain2.tld
>>>>
>>>> acceces with subdomainX.domain1.tld or subdomainY.domain2.tld or
>>>> whatever.domain1.tld I set it in my vcl to
>>>>
>>>> TheVisitedSubdomain.mymaindomain.tld
>>>>
>>>> And use that one in my VLC...
>>>>
>>>> Thanks so far!
>>>>
>>>> Matt



From lists at redlab.be  Fri Aug 28 12:51:27 2015
From: lists at redlab.be (Balder VC)
Date: Fri, 28 Aug 2015 14:51:27 +0200
Subject: varnish panic investigation information
Message-ID: <55E0594F.8010303@redlab.be>

Hi

I'm requesting some help or info for a panic we had.

First a small token of appreciation and some background.
We had our varnish-cache (varnish-4.0.3 revision b8c4a34 )  filled with
around 4.6 billion ( 4.6KK ) page entries on a 90GB RAM machine.  !
Great stuff, still responding fast and furious! Thank you varnish!

Varnish has 7 back ends configured. At some point they started failing
till they all broke down. As expected varnish does the job it's need to
do. It responds with an error page if the requested entry is not in the
cache, serves a cache paged ( that is either in grace mode or just
cached ) when it can.

After a certain time the varnish child process panicked and restarted
and we could not serve the cached pages any more since the were gone.

Assert error in VDI_GetFd(), cache/cache_dir.c line 111:

  Condition((d) != NULL) not true.


see http://pastebin.com/H1ZdTyPd for the full panic.show log

Now, there is one url that is served differently. Via a pipe, I show
this because it seems from the panic log that this has something to do
with it.

    if (req.url ~ "^/content/" ) {

      return (pipe);

    }





I perhaps thought that this pipe might have caused the panic due to no
backends being available to pipe too ? And that this could be a bug?
Or if not, perhaps we are doing something wrong?

Let me know if you want more information.


Kind Regards

Balder VC





From c.hartlage at modenkg.de  Sat Aug 29 22:48:42 2015
From: c.hartlage at modenkg.de (Christian Hartlage)
Date: Sun, 30 Aug 2015 00:48:42 +0200
Subject: Varnish 3.0 slow backend fetches
Message-ID: <D4B2C357-BFE6-4E9F-B720-7E3E48891BCC@modenkg.de>

Hi guys,

we're troubleshooting with a problem with our varnish setup - it would be great if you could help us.

Our server setup is as follows:

NGINX (as ssl-terminator) > Varnish  > NGINX/php-fpm (multiple web-server instances) 

Cacheable URLs work fine, even with around 5 ESI per site,  we can deliver 1000 req/s in around 100ms.

When it comes to non-cacheable URLs our setup fails completely. We can only serve about 5 req/s through varnish. If we leave out varnish for these urls and ask the nginx backends directly, we?re able to serve about 25 req/s. Without varnish, nginx directs these 25 req/s to the php-fpm processes and they get some nice cpu load.
With Varnish the php-fpm processes have nearyl zero cpu load, because varnish gets only 5req/s to the nginx backend.

Testing these requests with curl showed that it takes nginx/php-fpm about 0.2sec for a single response. For us it seems like varnish can do only 1 fetch at once, so that the backend request are blocking. Is that possible?
During load testing the client_conn counter jumps to 200 almost instantly and then increases with about 5 units per second. the backend_conn counter does not have the initial jump to 200 but increases from zero with 5 units per second.
After the load test is completed the client_conn counter does not increase anymore (obviously), but the backend_conn counter will continue increasing slowly until it reaches the amount of client_conn. This also made us think that varnish processes the backend requests sequentially, even when the clients are not actually waiting for the content anymore.

Best Regards, 

Christian Hartlage




From apj at mutt.dk  Mon Aug 31 06:37:34 2015
From: apj at mutt.dk (Andreas Plesner)
Date: Mon, 31 Aug 2015 08:37:34 +0200
Subject: Varnish 3.0 slow backend fetches
In-Reply-To: <D4B2C357-BFE6-4E9F-B720-7E3E48891BCC@modenkg.de>
References: <D4B2C357-BFE6-4E9F-B720-7E3E48891BCC@modenkg.de>
Message-ID: <20150831063734.GG2656@nerd.dk>

On Sun, Aug 30, 2015 at 12:48:42AM +0200, Christian Hartlage wrote:
> 
> Testing these requests with curl showed that it takes nginx/php-fpm about
> 0.2sec for a single response. For us it seems like varnish can do only 1
> fetch at once, so that the backend request are blocking. Is that possible?

Sounds like you're doing return(hit_for_pass) in vcl_fetch without setting a
TTL, thus effectively serializing all requests with the same hash.

If you set a TTL, varnish will cache that it can't cache that particular URL
and not attempt to coalesce requests in the future.

-- 
Andreas



From c.hartlage at modenkg.de  Mon Aug 31 15:01:25 2015
From: c.hartlage at modenkg.de (Christian Hartlage)
Date: Mon, 31 Aug 2015 17:01:25 +0200
Subject: Varnish 3.0 slow backend fetches
In-Reply-To: <20150831063734.GG2656@nerd.dk>
References: <D4B2C357-BFE6-4E9F-B720-7E3E48891BCC@modenkg.de>
 <20150831063734.GG2656@nerd.dk>
Message-ID: <C945A8CB-C759-4121-9264-9FA6253DDC85@modenkg.de>

Hi Andreas,

thanks for your response. You were completely right. Right before we returned hit_for_pass, the beresp.ttl was set to 0.
We thought this would be the correct way to do this, we should have read the manual better, since this behaviour is explicitly lined out there [1].

Best Regards,

Christian Hartlage


[1] https://www.varnish-software.com/book/3/VCL_Basics.html#vcl-vcl-fetch (red box)


> Am 31.08.2015 um 08:37 schrieb Andreas Plesner <apj at mutt.dk>:
> 
> On Sun, Aug 30, 2015 at 12:48:42AM +0200, Christian Hartlage wrote:
>> 
>> Testing these requests with curl showed that it takes nginx/php-fpm about
>> 0.2sec for a single response. For us it seems like varnish can do only 1
>> fetch at once, so that the backend request are blocking. Is that possible?
> 
> Sounds like you're doing return(hit_for_pass) in vcl_fetch without setting a
> TTL, thus effectively serializing all requests with the same hash.
> 
> If you set a TTL, varnish will cache that it can't cache that particular URL
> and not attempt to coalesce requests in the future.
> 
> -- 
> Andreas
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc