From tdevelioglu at ebuddy.com  Mon Nov  1 17:02:18 2010
From: tdevelioglu at ebuddy.com (Taylan Develioglu)
Date: Mon, 1 Nov 2010 18:02:18 +0100
Subject: varnish 2.1.4 swap and increase in objects:heads.
In-Reply-To: <20101030232242.GA53813@fupp.net>
References: <1288352241.5722.134.camel@oasis>
	<29097274.10.1288353418406.JavaMail.rbenzaquen@rbenzaquen4310>
	<AANLkTik2Mi31M2jWcjemXzOwhjN1E=Jx-9idnQUxSSKT@mail.gmail.com>
	<1288369190.18624.14.camel@oasis>  <20101030232242.GA53813@fupp.net>
Message-ID: <1288630938.2168.26.camel@oasis>

Hi Anders,

malloc had its own set of problems (ticket #784).

Judging by the object:header ratio, it looks like varnish is creating
too many objects. Unlikely that changing storage backend will help with
that.

I'm clueless and have no time to debug. I'm going to reduce the cache
size and hold out until squid 3.2 goes GA.

On Sun, 2010-10-31 at 01:22 +0200, Anders Nordby wrote:
> Hi,
> 
> No. I had issues with having a too large storage size, and havn't tried
> increasing it since.
> 
> But Varnish with -s malloc in FreeBSD does seem to run OK with 60 GB
> storage size and having 51 GB in the swap on disk at one of our sites. I
> have 32 GB of physical RAM.
> 
> Regads,
> Anders.
> 
> On Fri, Oct 29, 2010 at 06:19:50PM +0200, Taylan Develioglu wrote:
> > More of a general question:
> > 
> > Is there anyone that uses varnish with a storage size of 800GB+ (single
> > instance, file or malloc) ?
> > 
> > I'm just wondering because we've had so much problems with varnish
> > (2.1.2 to 2.1.4) that I'm starting to think that we're an edge case.
> > 
> > On Fri, 2010-10-29 at 14:26 +0200, Per Buer wrote:
> > > Hi.
> > > 
> > > The issues are actually a bit different. One uses "-s file" which
> > > really makes the kernel handle the memory management implicitly in
> > > comparison to you Rodrigo, which uses -s malloc, where you clearly
> > > state that Varnish should allocate this much memory.
> > > 
> > > Regarding -s malloc we recommend that on Linux one should not use more
> > > then 75% of the physical memory for -s malloc. In addition to the
> > > actual object store Varnish uses quite some memory for it's internal
> > > data structures. If you go above 75% chances are that Linux will start
> > > swapping a bit which might have a negative effect on overall
> > > performance. Tollef is getting back to you outside the list regarding
> > > the issue you've raised with us.
> > > 
> > > Per.
> > > 
> > > On Fri, Oct 29, 2010 at 1:59 PM, Rodrigo Benzaquen
> > > <rodrigo at mercadolibre.com> wrote:
> > > > Hi Taylan, exactly the same is happening to us, we already open a ticket
> > > > with Varnish software.
> > > >
> > > > Please let me know if you find a solution.
> > > >
> > > > Thanks
> > > > Rodrigo
> > > >
> > > >
> > > >
> > > >
> > > > ------------------
> > > >
> > > > Rodrigo Benzaquen
> > > > @rbenzaquen
> > > >
> > > >
> > > >
> > > > ________________________________
> > > > From: "Taylan Develioglu" <tdevelioglu at ebuddy.com>
> > > > To: varnish-misc at varnish-cache.org
> > > > Sent: Viernes, 29 de Octubre 2010 8:37:21
> > > > Subject: varnish 2.1.4 swap and increase in objects:heads.
> > > >
> > > > Debian Lenny, Linux varnish0 2.6.33.7 #1 SMP Thu Sep 2 15:27:53 CEST
> > > > 2010 x86_64 GNU/Linux
> > > >
> > > > I have a strange situation on one of our varnish servers.
> > > >
> > > > We recently raised the expiry on previously uncached content a
> > > > (Cache-Control: max-age=0, to Cache-Control: max-age=5).
> > > >
> > > > The number of objects vs. heads increased and the system started
> > > > swapping (vm.swappiness=0).
> > > >
> > > > http://lcontent.ebuddy.com/varnish/memory-week.png
> > > > http://lcontent.ebuddy.com/varnish/varnish_objects-week.png
> > > > http://lcontent.ebuddy.com/varnish/varnish_expunge-week.png
> > > >
> > > >
> > > > I don't understand why this is happening, bug maybe ?
> > > >
> > > > DAEMON_OPTS="-a :27535 \
> > > >              -T localhost:6082 \
> > > >              -f /etc/varnish/default.vcl \
> > > >              -s file,/var/cache/varnish/sda/varnish_storage.bin,135G \
> > > >              -s file,/var/cache/varnish/sdb/varnish_storage.bin,135G \
> > > >              -s file,/var/cache/varnish/sdc/varnish_storage.bin,135G \
> > > >              -s file,/var/cache/varnish/sdd/varnish_storage.bin,135G \
> > > >              -s file,/var/cache/varnish/sde/varnish_storage.bin,135G \
> > > >              -s file,/var/cache/varnish/sdf/varnish_storage.bin,135G \
> > > >              -p thread_pools=4 \
> > > >              -p thread_pool_min=200 \
> > > >              -p thread_pool_max=2400 \
> > > >              -p thread_pool_add_delay=2 \
> > > >              -p lru_interval=20"
> > > >
> > > >
> > > >
> > > > ---vcl---
> > > > sub vcl_recv {
> > > >     unset req.http.cookie;
> > > >     unset req.http.Authorization;
> > > >
> > > >     if (req.http.host ~ "^.*host.domain.com.*") {
> > > >         set req.http.host = "host.domain.com";
> > > >     }
> > > >
> > > >     if (req.request == "HEAD") {
> > > >         return(pass);
> > > >     }
> > > >
> > > >     if (req.request == "PURGE") {
> > > >         if (!client.ip ~ purge) {
> > > >             error 405 "Not allowed.";
> > > >         }
> > > >         return(lookup);
> > > >     }
> > > >
> > > >     if (req.request == "GET" && req.url == "/monitor") {
> > > >         error 200 "OK.";
> > > >     }
> > > >
> > > >     if (req.backend.healthy) {
> > > >         set req.grace = 30s;
> > > >   } else {
> > > >         set req.grace = 48h;
> > > >     }
> > > >
> > > > }
> > > >
> > > >  sub vcl_fetch {
> > > >     if (beresp.status == 404 || beresp.status >= 300) {
> > > >          return (pass);
> > > >     }
> > > >     set beresp.grace = 48h;
> > > >
> > > >  }
> > > >
> > > > sub vcl_hit {
> > > >     if (req.request == "PURGE") {
> > > >         set obj.ttl = 0s;
> > > >         error 200 "Purged.";
> > > >     }
> > > > }
> > > >
> > > > sub vcl_miss {
> > > >     if (req.request == "PURGE") {
> > > >         error 404 "Not in cache.";
> > > >     }
> > > > }
> > > > ---EOF---
> > > >
> > > >
> > > > _______________________________________________
> > > > varnish-misc mailing list
> > > > varnish-misc at varnish-cache.org
> > > > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> > > >
> > > > _______________________________________________
> > > > varnish-misc mailing list
> > > > varnish-misc at varnish-cache.org
> > > > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> > > >
> > > 
> > > 
> > > 
> > 
> > 
> > 
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> 
> 





From m.schuhfuss at googlemail.com  Tue Nov  2 00:11:23 2010
From: m.schuhfuss at googlemail.com (Martin Schuhfuss)
Date: Tue, 2 Nov 2010 01:11:23 +0100
Subject: Very slow responses from backend-requests
Message-ID: <AANLkTim6TMBDg=dDaux_hnaba+eZcig22BU4_V0arQDB@mail.gmail.com>

Hi all,

I'm having some problems with a newly installed varnish-server.
Since I'm pretty new to varnish after all, it might be i'm missing something
i currently don't see.

My setup consists of a varnish-server (2.1.4 SVN) sitting in front of an
apache/mod_php-backend.

It seems that nearly any request that is passed on to the backend takes
(more or less) exactly 15sec to complete.
Anything coming from the cache is delivered in no time.
There is, however, no reason I can see since the backend-processing takes
only 82ms.

When an URL is called, it hits the backend immediately which in turn takes
only 82ms to deliver the content - but for some reason it seems to hang
around in varnish for about 15 sec.

I'm left with absolutely no idea where to look for a solution for this
problem and I hope someone here can give me a hint.

Regards, Martin


For reference, here's the VCL used and an example of such a request from the
varnishlog:

# cat /etc/varnish/default.vcl
backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

sub vcl_recv {
    if( req.url ~ "\.(png|jpg|gif|swf|flv|css|js|ico)$" ) {
       unset req.http.cookie;
    }

    if( req.url ~ "^/(downloads|css|js|images)" ) {
       unset req.http.cookie;
    }
}

sub vcl_fetch {
    # http://www.varnish-cache.org/trac/ticket/806
    if (req.request != "HEAD") {
        unset beresp.http.content-length;
    }

    if( req.url ~ "\.(png|jpg|gif|swf|flv|css|js|ico)$") {
        unset beresp.http.set-cookie;
        set beresp.ttl = 900s;
    }
}


# varnishlog -c -o rxurl 'foobar'
   22 SessionOpen  c 123.123.123.123 53337 :80
   22 ReqStart     c 123.123.123.123 53337 1717196682
   22 RxRequest    c GET
   22 RxURL        c /?foobar
   22 RxProtocol   c HTTP/1.1
   22 RxHeader     c Host: example.com
   22 RxHeader     c Connection: keep-alive
   22 RxHeader     c Cache-Control: max-age=0
   22 RxHeader     c Accept:
application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
   22 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US)
AppleWebKit/534.10 (KHTML, like Gecko) Ubuntu/10.04 Chromium/8.0.552.18
Chrome/8.0.552.18 Safari/534.10
   22 RxHeader     c Accept-Encoding: gzip,deflate,sdch
   22 RxHeader     c Accept-Language: en-US;q=0.6,en;q=0.4
   22 RxHeader     c Accept-Charset: utf-8;q=0.7,*;q=0.3
   22 RxHeader     c Cookie: foo=649cffb6cebc98dcb50121d68931e26c
   22 VCL_call     c recv pass
   22 VCL_call     c hash hash
   22 VCL_call     c pass pass
   22 Backend      c 26 default default
   22 TTL          c 1717196682 RFC 0 1288655727 1288655727 375007920 0 0
   22 VCL_call     c fetch pass
   22 ObjProtocol  c HTTP/1.1
   22 ObjStatus    c 200
   22 ObjResponse  c OK
   22 ObjHeader    c Date: Mon, 01 Nov 2010 23:55:27 GMT
   22 ObjHeader    c Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9 with
Suhosin-Patch
   22 ObjHeader    c X-Powered-By: PHP/5.2.6-1+lenny9
   22 ObjHeader    c Expires: Thu, 19 Nov 1981 08:52:00 GMT
   22 ObjHeader    c Pragma: no-cache
   22 ObjHeader    c X-Time: t=1288655727366215 D=82924
   22 ObjHeader    c Vary: Accept-Encoding
   22 ObjHeader    c Content-Encoding: gzip
   22 ObjHeader    c Content-Type: text/html; charset=utf-8
   22 VCL_call     c deliver deliver
   22 TxProtocol   c HTTP/1.1
   22 TxStatus     c 200
   22 TxResponse   c OK
   22 TxHeader     c Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9 with
Suhosin-Patch
   22 TxHeader     c X-Powered-By: PHP/5.2.6-1+lenny9
   22 TxHeader     c Expires: Thu, 19 Nov 1981 08:52:00 GMT
   22 TxHeader     c Pragma: no-cache
   22 TxHeader     c X-Time: t=1288655727366215 D=82924
   22 TxHeader     c Vary: Accept-Encoding
   22 TxHeader     c Content-Encoding: gzip
   22 TxHeader     c Content-Type: text/html; charset=utf-8
   22 TxHeader     c Content-Length: 6747
   22 TxHeader     c Date: Mon, 01 Nov 2010 23:55:42 GMT
   22 TxHeader     c X-Varnish: 1717196682
   22 TxHeader     c Age: 15
   22 TxHeader     c Via: 1.1 varnish
   22 TxHeader     c Connection: keep-alive
   22 Length       c 6747
   22 ReqEnd       c 1717196682 1288655727.366053343 1288655742.456236839
0.000039101 15.090134859 0.000048637
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101102/d40fb136/attachment.html>

From mark.stafford at gmail.com  Tue Nov  2 21:23:31 2010
From: mark.stafford at gmail.com (Mark C. Stafford)
Date: Tue, 2 Nov 2010 15:23:31 -0600
Subject: frontend links include backend port
Message-ID: <AANLkTik_Y9+H2fSKReT6EDV2j2csEzd3WJSGHzW53hOh@mail.gmail.com>

I've used squid as an accelerator in the past and would like to try
varnish. I don't understand why links to my site show up as
http://site:8100/query-etc right now.

How can I get the backend port out of my links? Should I expect to
have to rewrite them?

I'm using current packages for apache and varnish.

Thanks

Mark



# from /etc/init.d/varnish
# DAEMON_OPTS="-a 0.0.0.0:80 -T 0.0.0.0:8200 -f
/etc/varnish/varnish.vcl -S /etc/varnish/secret -s
file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"
DAEMON_OPTS="-T :8200 -f /etc/varnish/varnish.vcl -S
/etc/varnish/secret -s
file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"

# from ports.conf
NameVirtualHost *:8100
Listen 127.0.0.1:8100

# from varnish.vcl
backend default {
  .host = "127.0.0.1";
  .port = "8100";
}

# other info

kernel
  2.6.18.8-xen
distro
  debian/lenny
apache
  2.2.9-10+lenny8
varnish
  2.1.4-1~lenny1

# netstat -anp | grep varnish
tcp        0      0 0.0.0.0:8200            0.0.0.0:*
LISTEN      19911/varnishd
tcp        0      0 0.0.0.0:80              0.0.0.0:*
LISTEN      19912/varnishd
tcp        1      0 127.0.0.1:49060         127.0.0.1:8100
CLOSE_WAIT  19912/varnishd
tcp6       0      0 :::8200                 :::*
LISTEN      19911/varnishd
unix  2      [ ]         DGRAM                    687365   19911/varnishd

# netstat -anp | grep apache
tcp        0      0 127.0.0.1:8100          0.0.0.0:*
LISTEN      19987/apache2

# varnishlog -o

13 TxRequest    - GET
13 TxURL        - /
13 TxProtocol   - HTTP/1.1
13 TxHeader     - Host: *****
13 TxHeader     - Accept:
application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
13 TxHeader     - User-Agent: Mozilla/5.0 (X11; U; Linux x86_64;
en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.41
Safari/534.7
13 TxHeader     - Accept-Encoding: gzip,deflate,sdch
13 TxHeader     - Accept-Language: en-US,en;q=0.8
13 TxHeader     - Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
13 TxHeader     - Cookie: PHPSESSID=4b594099f859844319e122a3291c2d37
13 TxHeader     - X-Forwarded-For: ***.***.***.***
13 Length       - 3373
13 BackendReuse - default
12 SessionOpen  c ***.***.***.*** 32211 0.0.0.0:80
12 ReqStart     c ***.***.***.*** 32211 1997066746
12 RxRequest    c GET
12 RxURL        c /
12 RxProtocol   c HTTP/1.1
12 RxHeader     c Host: *****
12 RxHeader     c Connection: keep-alive
12 RxHeader     c Cache-Control: max-age=0
12 RxHeader     c Accept:
application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
12 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux x86_64;
en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.41
Safari/534.7
12 RxHeader     c Accept-Encoding: gzip,deflate,sdch
12 RxHeader     c Accept-Language: en-US,en;q=0.8
12 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
12 RxHeader     c Cookie: PHPSESSID=4b594099f859844319e122a3291c2d37
12 VCL_call     c recv pass
12 VCL_call     c hash hash
12 VCL_call     c pass pass
12 Backend      c 13 default default
12 TTL          c 1997066746 RFC 0 1288731432 1288731432 375007920 0 0
12 VCL_call     c fetch pass
12 ObjProtocol  c HTTP/1.1
12 ObjStatus    c 200
12 ObjResponse  c OK
12 ObjHeader    c Date: Tue, 02 Nov 2010 20:57:12 GMT
12 ObjHeader    c Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9
with Suhosin-Patch
12 ObjHeader    c X-Powered-By: PHP/5.2.6-1+lenny9
12 ObjHeader    c Expires: Thu, 19 Nov 1981 08:52:00 GMT
12 ObjHeader    c Pragma: no-cache
12 ObjHeader    c X-action: display_home
12 ObjHeader    c Vary: Accept-Encoding
12 ObjHeader    c Content-Encoding: gzip
12 ObjHeader    c Content-Length: 3373
12 ObjHeader    c Content-Type: text/html;charset=utf-8
12 VCL_call     c deliver deliver
12 TxProtocol   c HTTP/1.1
12 TxStatus     c 200
12 TxResponse   c OK
12 TxHeader     c Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9
with Suhosin-Patch
12 TxHeader     c X-Powered-By: PHP/5.2.6-1+lenny9
12 TxHeader     c Expires: Thu, 19 Nov 1981 08:52:00 GMT
12 TxHeader     c Pragma: no-cache
12 TxHeader     c X-action: display_home
12 TxHeader     c Vary: Accept-Encoding
12 TxHeader     c Content-Encoding: gzip
12 TxHeader     c Content-Length: 3373
12 TxHeader     c Content-Type: text/html;charset=utf-8
12 TxHeader     c Content-Length: 3373
12 TxHeader     c Date: Tue, 02 Nov 2010 20:57:12 GMT
12 TxHeader     c X-Varnish: 1997066746
12 TxHeader     c Age: 0
12 TxHeader     c Via: 1.1 varnish
12 TxHeader     c Connection: keep-alive
12 Length       c 3373
12 ReqEnd       c 1997066746 1288731432.496306896 1288731432.680691957
0.000109911 0.184325218 0.000059843



From perbu at varnish-software.com  Tue Nov  2 21:47:59 2010
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 2 Nov 2010 22:47:59 +0100
Subject: frontend links include backend port
In-Reply-To: <AANLkTik_Y9+H2fSKReT6EDV2j2csEzd3WJSGHzW53hOh@mail.gmail.com>
References: <AANLkTik_Y9+H2fSKReT6EDV2j2csEzd3WJSGHzW53hOh@mail.gmail.com>
Message-ID: <AANLkTimBT2Mc1ysdSXux6i6Sc3n+=yRNyvYP9+PsH-i_@mail.gmail.com>

Hi Mark, list.

On Tue, Nov 2, 2010 at 10:23 PM, Mark C. Stafford
<mark.stafford at gmail.com>wrote:

> I've used squid as an accelerator in the past and would like to try
> varnish. I don't understand why links to my site show up as
> http://site:8100/query-etc right now.
>


Your web- or application server listens on port 8100. It knows about this
and produces links in the HTML code that link directly to its content. You
have to look at how you configure either your application server or the
application that runs on top of it to produce links that won't contain the
:8100 part. This is not really relevant for Varnish and Varnish can't do
anything about it.

>
>
-- 
Per Buer,  Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101102/d3efad9a/attachment.html>

From jan-varnish at buksy.de  Tue Nov  2 22:16:32 2010
From: jan-varnish at buksy.de (Jan Weiher)
Date: Tue, 02 Nov 2010 23:16:32 +0100
Subject: frontend links include backend port
In-Reply-To: <AANLkTik_Y9+H2fSKReT6EDV2j2csEzd3WJSGHzW53hOh@mail.gmail.com>
References: <AANLkTik_Y9+H2fSKReT6EDV2j2csEzd3WJSGHzW53hOh@mail.gmail.com>
Message-ID: <4CD08DC0.5070302@buksy.de>

Hi,

is it possible that your webapplication is adding the portnumber to the
links? I've been using varnish in a similar configuration for a while
now and did not see such a behaviour.

best regards,
Jan



Am 02.11.2010 22:23, schrieb Mark C. Stafford:
> I've used squid as an accelerator in the past and would like to try
> varnish. I don't understand why links to my site show up as
> http://site:8100/query-etc right now.
> 
> How can I get the backend port out of my links? Should I expect to
> have to rewrite them?
> 
> I'm using current packages for apache and varnish.
> 
> Thanks
> 
> Mark
> 
> 
> 
> # from /etc/init.d/varnish
> # DAEMON_OPTS="-a 0.0.0.0:80 -T 0.0.0.0:8200 -f
> /etc/varnish/varnish.vcl -S /etc/varnish/secret -s
> file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"
> DAEMON_OPTS="-T :8200 -f /etc/varnish/varnish.vcl -S
> /etc/varnish/secret -s
> file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"
> 
> # from ports.conf
> NameVirtualHost *:8100
> Listen 127.0.0.1:8100
> 
> # from varnish.vcl
> backend default {
>   .host = "127.0.0.1";
>   .port = "8100";
> }
> 
> # other info
> 
> kernel
>   2.6.18.8-xen
> distro
>   debian/lenny
> apache
>   2.2.9-10+lenny8
> varnish
>   2.1.4-1~lenny1
> 
> # netstat -anp | grep varnish
> tcp        0      0 0.0.0.0:8200            0.0.0.0:*
> LISTEN      19911/varnishd
> tcp        0      0 0.0.0.0:80              0.0.0.0:*
> LISTEN      19912/varnishd
> tcp        1      0 127.0.0.1:49060         127.0.0.1:8100
> CLOSE_WAIT  19912/varnishd
> tcp6       0      0 :::8200                 :::*
> LISTEN      19911/varnishd
> unix  2      [ ]         DGRAM                    687365   19911/varnishd
> 
> # netstat -anp | grep apache
> tcp        0      0 127.0.0.1:8100          0.0.0.0:*
> LISTEN      19987/apache2
> 
> # varnishlog -o
> 
> 13 TxRequest    - GET
> 13 TxURL        - /
> 13 TxProtocol   - HTTP/1.1
> 13 TxHeader     - Host: *****
> 13 TxHeader     - Accept:
> application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
> 13 TxHeader     - User-Agent: Mozilla/5.0 (X11; U; Linux x86_64;
> en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.41
> Safari/534.7
> 13 TxHeader     - Accept-Encoding: gzip,deflate,sdch
> 13 TxHeader     - Accept-Language: en-US,en;q=0.8
> 13 TxHeader     - Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
> 13 TxHeader     - Cookie: PHPSESSID=4b594099f859844319e122a3291c2d37
> 13 TxHeader     - X-Forwarded-For: ***.***.***.***
> 13 Length       - 3373
> 13 BackendReuse - default
> 12 SessionOpen  c ***.***.***.*** 32211 0.0.0.0:80
> 12 ReqStart     c ***.***.***.*** 32211 1997066746
> 12 RxRequest    c GET
> 12 RxURL        c /
> 12 RxProtocol   c HTTP/1.1
> 12 RxHeader     c Host: *****
> 12 RxHeader     c Connection: keep-alive
> 12 RxHeader     c Cache-Control: max-age=0
> 12 RxHeader     c Accept:
> application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
> 12 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux x86_64;
> en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.41
> Safari/534.7
> 12 RxHeader     c Accept-Encoding: gzip,deflate,sdch
> 12 RxHeader     c Accept-Language: en-US,en;q=0.8
> 12 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
> 12 RxHeader     c Cookie: PHPSESSID=4b594099f859844319e122a3291c2d37
> 12 VCL_call     c recv pass
> 12 VCL_call     c hash hash
> 12 VCL_call     c pass pass
> 12 Backend      c 13 default default
> 12 TTL          c 1997066746 RFC 0 1288731432 1288731432 375007920 0 0
> 12 VCL_call     c fetch pass
> 12 ObjProtocol  c HTTP/1.1
> 12 ObjStatus    c 200
> 12 ObjResponse  c OK
> 12 ObjHeader    c Date: Tue, 02 Nov 2010 20:57:12 GMT
> 12 ObjHeader    c Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9
> with Suhosin-Patch
> 12 ObjHeader    c X-Powered-By: PHP/5.2.6-1+lenny9
> 12 ObjHeader    c Expires: Thu, 19 Nov 1981 08:52:00 GMT
> 12 ObjHeader    c Pragma: no-cache
> 12 ObjHeader    c X-action: display_home
> 12 ObjHeader    c Vary: Accept-Encoding
> 12 ObjHeader    c Content-Encoding: gzip
> 12 ObjHeader    c Content-Length: 3373
> 12 ObjHeader    c Content-Type: text/html;charset=utf-8
> 12 VCL_call     c deliver deliver
> 12 TxProtocol   c HTTP/1.1
> 12 TxStatus     c 200
> 12 TxResponse   c OK
> 12 TxHeader     c Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9
> with Suhosin-Patch
> 12 TxHeader     c X-Powered-By: PHP/5.2.6-1+lenny9
> 12 TxHeader     c Expires: Thu, 19 Nov 1981 08:52:00 GMT
> 12 TxHeader     c Pragma: no-cache
> 12 TxHeader     c X-action: display_home
> 12 TxHeader     c Vary: Accept-Encoding
> 12 TxHeader     c Content-Encoding: gzip
> 12 TxHeader     c Content-Length: 3373
> 12 TxHeader     c Content-Type: text/html;charset=utf-8
> 12 TxHeader     c Content-Length: 3373
> 12 TxHeader     c Date: Tue, 02 Nov 2010 20:57:12 GMT
> 12 TxHeader     c X-Varnish: 1997066746
> 12 TxHeader     c Age: 0
> 12 TxHeader     c Via: 1.1 varnish
> 12 TxHeader     c Connection: keep-alive
> 12 Length       c 3373
> 12 ReqEnd       c 1997066746 1288731432.496306896 1288731432.680691957
> 0.000109911 0.184325218 0.000059843
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc



From varnish at ds.schledermann.net  Tue Nov  2 22:51:01 2010
From: varnish at ds.schledermann.net (Daniel Schledermann (Varnish mail list))
Date: Tue, 02 Nov 2010 23:51:01 +0100
Subject: frontend links include backend port
In-Reply-To: <AANLkTik_Y9+H2fSKReT6EDV2j2csEzd3WJSGHzW53hOh@mail.gmail.com>
References: <AANLkTik_Y9+H2fSKReT6EDV2j2csEzd3WJSGHzW53hOh@mail.gmail.com>
Message-ID: <1a71a1ee1ee6be9a1543481fd1bf776b@localhost>

On Tue, 2 Nov 2010 15:23:31 -0600, "Mark C. Stafford"
<mark.stafford at gmail.com> wrote:
> I've used squid as an accelerator in the past and would like to try
> varnish. I don't understand why links to my site show up as
> http://site:8100/query-etc right now.
> 
> How can I get the backend port out of my links? Should I expect to
> have to rewrite them?


Well that would depend on your backend, but in your setup it should be
entirely possible for both varnish and apache both to listen on port
8200. 

varnish -> site:8200
apache -> localhost:8200

/D



From tfheen at varnish-software.com  Wed Nov  3 07:30:06 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 03 Nov 2010 08:30:06 +0100
Subject: Dealing with large unwanted objects
In-Reply-To: <AANLkTi=SSBCF+_EGa+TYZmEarvR7hmTW4jp4L8p0iPEv@mail.gmail.com>
	(Mark Moseley's message of "Thu, 28 Oct 2010 16:04:14 -0700")
References: <AANLkTi=SSBCF+_EGa+TYZmEarvR7hmTW4jp4L8p0iPEv@mail.gmail.com>
Message-ID: <874obyzwxt.fsf@qurzaw.linpro.no>

]] Mark Moseley 

| I'm doing a half-way job of it by calling 'pass' in vcl_fetch, i.e.,

As Per's pointed out, that won't actually do what you want, which is to
pipe.  However, we can use a neat trick: restart, and the fact that we
can check headers in vcl_fetch, we haven't grabbed the body yet.

Something like:

sub vcl_recv {
  if (req.http.x-pipe && req.restarts > 0) {
    return(pipe);
  }
}

sub vcl_fetch {
  # This is the rule to knock out big files
  if ( beresp.http.Content-Length ~ "[0-9]{8,}" ) {
     set req.http.x-pipe = "1";
     restart;
}

I believe this should work.  It's completely untested and feedback is
very welcome.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From tfheen at varnish-software.com  Wed Nov  3 07:47:31 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 03 Nov 2010 08:47:31 +0100
Subject: How to know the current cache size using VCL
In-Reply-To: <4CC9A3D2.80304@smartjog.com> (David Donchez's message of "Thu,
	28 Oct 2010 18:24:50 +0200")
References: <4CC9A3D2.80304@smartjog.com>
Message-ID: <87zktqyhkc.fsf@qurzaw.linpro.no>

]] David Donchez 

| I want to know what's happen when the cache is full. Is the first object
| cached be deleted ?

The least recently used item is discarded, there's no way to customise
which object is discarded.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From tfheen at varnish-software.com  Wed Nov  3 09:11:21 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 03 Nov 2010 10:11:21 +0100
Subject: Very slow responses from backend-requests
In-Reply-To: <AANLkTim6TMBDg=dDaux_hnaba+eZcig22BU4_V0arQDB@mail.gmail.com>
	(Martin Schuhfuss's message of "Tue, 2 Nov 2010 01:11:23 +0100")
References: <AANLkTim6TMBDg=dDaux_hnaba+eZcig22BU4_V0arQDB@mail.gmail.com>
Message-ID: <87vd4eydom.fsf@qurzaw.linpro.no>

]] Martin Schuhfuss 

| It seems that nearly any request that is passed on to the backend takes
| (more or less) exactly 15sec to complete.
| Anything coming from the cache is delivered in no time.
| There is, however, no reason I can see since the backend-processing takes
| only 82ms.

I believe this is now fixed in the 2.1 branch in SVN.  Testing would be
much appreciated.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From ITLJ at gyldendal.dk  Wed Nov  3 12:10:37 2010
From: ITLJ at gyldendal.dk (=?iso-8859-1?Q?Lars_J=F8rgensen?=)
Date: Wed, 3 Nov 2010 13:10:37 +0100
Subject: LRU moved object but still memory to spare
Message-ID: <311386E60ED7E74C8716A6573A7740A01869808BAD@s-exch01-v.gyldendal.local>

Hi,

varnishstat says

      257738          .            .   N LRU moved objects
  1924911104          .            .   bytes allocated
126924107776          .            .   bytes free

The LRU counter is increasing steadily, but the server has ample memory to spare. Why do LRU objects gets thrown out of the cache? Or am I misunderstanding that counter?


-- 
Lars


From wido at widodh.nl  Wed Nov  3 13:51:14 2010
From: wido at widodh.nl (Wido den Hollander)
Date: Wed, 03 Nov 2010 14:51:14 +0100
Subject: Logging to syslog via Embedded C and http-range support
Message-ID: <1288792274.2210.23.camel@wido-desktop>

Hi all,

In our current setup (without Varnish) we have a cluster of Apache
webservers which all log to syslog.

Via a remote-syslog machine we collect all the logs and parse them and
do with them what we want.

We use a specific logformat, so using varnishncsa is not possible (yet).

Right now I've written the following code:

sub vcl_deliver {
    C{
    time_t rawtime;
    struct tm * timeinfo;
    char timebuffer[80];

    time(&rawtime);
    timeinfo = localtime(&rawtime);
    strftime(timebuffer, 80, "[%d/%b/%Y:%T %z]", timeinfo);

    char *referer = VRT_GetHdr(sp, HDR_REQ, "\010Referer:");
    if (referer == NULL) {
        referer = "-";
    }

    char *useragent = VRT_GetHdr(sp, HDR_REQ, "\013User-Agent:");
    if (useragent == NULL) {
        useragent = "-";
    }

    syslog(LOG_INFO,
        "%s %s %s - - \"%s %s %s\" %d %s \"%s\" \"%s\"",
        VRT_GetHdr(sp, HDR_REQ, "\005Host:"),
        VRT_IP_string(sp, VRT_r_client_ip(sp)),
        timebuffer,
        VRT_r_req_request(sp),
        VRT_r_req_url(sp),
        VRT_r_req_proto(sp),
        VRT_r_obj_status(sp),
        VRT_GetHdr(sp, HDR_RESP, "\017Content-Length:"),
        referer,
        useragent);
    }C
}

This works fine, I get the logs in the format I want it to be, but when
enabling http_range_support I get strange behaviour.

$ curl -i -r 0-500 <url>

When doing that, I get a 206 response from Varnish with a Content-Length
of 501 bytes, but my logfile shows a obj_status of 200 and a
Content-Length of almost 10kb (the actual size of the object).

It seems to me that the actual headers to the client are somewhere
overwritten when using http_range_support.

Is this because of the experimental http_range_support? Or is it
possible to collect the correct data somewhere so I can log it?

Thank you,

Wido




From A.Hongens at netmatch.nl  Wed Nov  3 14:10:17 2010
From: A.Hongens at netmatch.nl (=?utf-8?B?QW5nZWxvIEjDtm5nZW5z?=)
Date: Wed, 3 Nov 2010 14:10:17 +0000
Subject: response time in varnishncsa
In-Reply-To: <6A7ABA19243F1E4EADD8BB1563CDDCCB0BC4C0@TIL-EXCH-05.netmatch.local>
References: <717687862.6179.1280994948003.JavaMail.root@imap1b>
	<1970065075.273993.1286537149850.JavaMail.root@imap1b>
	<6A7ABA19243F1E4EADD8BB1563CDDCCB083920@TIL-EXCH-05.netmatch.local>
	<6A7ABA19243F1E4EADD8BB1563CDDCCB0BC4C0@TIL-EXCH-05.netmatch.local>
Message-ID: <6A7ABA19243F1E4EADD8BB1563CDDCCB0D614C@TIL-EXCH-05.netmatch.local>

>> -----Original Message-----
>> From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-
>> bounces at varnish-cache.org] On Behalf Of Angelo H?ngens
>> Sent: vrijdag 8 oktober 2010 13:35
>> To: 'Varnish-misc'
>> Subject: RE: response time in varnishncsa
>> 
>> Havent seen anything on the list about this. I would also really like
>> to see the time-taken field in the varnishnca log. And what I would
>> want even more, is a field indicating hit/miss! I might even convince
>> my boss to pay for the hours taken.
>
> -----Original Message-----
> From: Angelo H?ngens
> Sent: zaterdag 23 oktober 2010 17:41
> To: 'Poul-Henning Kamp (phk at phk.freebsd.dk)'
> Subject: RE: response time in varnishncsa
> 
> Poul-Henning,
> 
> Just a quick question off-list: I really want the extra feature below
> (2 extra fields in varnishncsa output).
> 
> Would you be willing to do this for pay? I'm pretty sure I can get my
> employer to pay for this feature. Don't know if you have any time, or
> how many hours it would take you, but I was thinking about some fixed
> price based on an initial estimate of your hours?
> 
> Please let me know..


Since PHK is not responding, is there anyone else that's interested in making a buck on this?


-- 

 
With kind regards,
 
 
Angelo H?ngens
 
Systems Administrator
 
------------------------------------------
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:A.Hongens at netmatch.nl
http://www.netmatch.nl
------------------------------------------




From Glen.Kelly at fire.tas.gov.au  Thu Nov  4 05:42:27 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Thu, 4 Nov 2010 16:42:27 +1100
Subject: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86
Message-ID: <674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>

Hi

I am receiving the same errors as described here when installing on Centos 5.5 x86.

error: rpmts_HdrFromFdno: Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb

Is there a known solution?

Also will the 2.1.4 rpm's be eventually available from http://sourceforge.net/projects/varnish/files/?

Regards Glen

[cid:tasfire_wakeup_small440d.jpg]<http://www.fire.tas.gov.au/mysite/Show?pageId=colDayLightSavings>
P Do you need to print this? Consider the environment, prevent paper waste.
CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101104/cdb5a6ff/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tasfire_wakeup_small440d.jpg
Type: image/jpeg
Size: 27459 bytes
Desc: tasfire_wakeup_small440d.jpg
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101104/cdb5a6ff/attachment.jpg>

From tfheen at varnish-software.com  Thu Nov  4 07:43:38 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Thu, 04 Nov 2010 08:43:38 +0100
Subject: LRU moved object but still memory to spare
In-Reply-To: <311386E60ED7E74C8716A6573A7740A01869808BAD@s-exch01-v.gyldendal.local>
	("Lars =?utf-8?Q?J=C3=B8rgensen=22's?= message of "Wed,
	3 Nov 2010 13:10:37 +0100")
References: <311386E60ED7E74C8716A6573A7740A01869808BAD@s-exch01-v.gyldendal.local>
Message-ID: <871v71y1n9.fsf@qurzaw.linpro.no>

]] Lars J?rgensen 

Hi,

|       257738          .            .   N LRU moved objects
|   1924911104          .            .   bytes allocated
| 126924107776          .            .   bytes free
| 
| The LRU counter is increasing steadily, but the server has ample
| memory to spare. Why do LRU objects gets thrown out of the cache? Or
| am I misunderstanding that counter?

LRU moved just means the LRU list has been updated.  LRU nuked means
they're removed from the cache, so yes, you're misunderstanding the
counter.

Best regards,
-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From ITLJ at gyldendal.dk  Thu Nov  4 09:28:03 2010
From: ITLJ at gyldendal.dk (=?utf-8?B?TGFycyBKw7hyZ2Vuc2Vu?=)
Date: Thu, 4 Nov 2010 10:28:03 +0100
Subject: SV: LRU moved object but still memory to spare
In-Reply-To: <871v71y1n9.fsf@qurzaw.linpro.no>
References: <311386E60ED7E74C8716A6573A7740A01869808BAD@s-exch01-v.gyldendal.local>
	<871v71y1n9.fsf@qurzaw.linpro.no>
Message-ID: <311386E60ED7E74C8716A6573A7740A0186D14B349@s-exch01-v.gyldendal.local>

Hi Tollef,

Sorry for topquoting, but using Outlook at work, so..

Anyways, thanks for your reply. Since I have no LRU nuked-entries in varnishstat, things seems to be working as intended.


Lars

-----Oprindelig meddelelse-----
Fra: Tollef Fog Heen [mailto:tfheen at varnish-software.com] 
Sendt: 4. november 2010 08:44
Til: Lars J?rgensen
Cc: 'varnish-misc at varnish-cache.org'
Emne: Re: LRU moved object but still memory to spare

]] Lars J?rgensen 

Hi,

|       257738          .            .   N LRU moved objects
|   1924911104          .            .   bytes allocated
| 126924107776          .            .   bytes free
| 
| The LRU counter is increasing steadily, but the server has ample
| memory to spare. Why do LRU objects gets thrown out of the cache? Or
| am I misunderstanding that counter?

LRU moved just means the LRU list has been updated.  LRU nuked means
they're removed from the cache, so yes, you're misunderstanding the
counter.

Best regards,
-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64

From moseleymark at gmail.com  Thu Nov  4 18:42:44 2010
From: moseleymark at gmail.com (Mark Moseley)
Date: Thu, 4 Nov 2010 11:42:44 -0700
Subject: Dealing with large unwanted objects
In-Reply-To: <874obyzwxt.fsf@qurzaw.linpro.no>
References: <AANLkTi=SSBCF+_EGa+TYZmEarvR7hmTW4jp4L8p0iPEv@mail.gmail.com>
	<874obyzwxt.fsf@qurzaw.linpro.no>
Message-ID: <AANLkTinFKoYCon0mWqdCjMk=DSh9VuMGGeHyPcrFG-Uu@mail.gmail.com>

On Wed, Nov 3, 2010 at 12:30 AM, Tollef Fog Heen
<tfheen at varnish-software.com> wrote:
> ]] Mark Moseley
>
> | I'm doing a half-way job of it by calling 'pass' in vcl_fetch, i.e.,
>
> As Per's pointed out, that won't actually do what you want, which is to
> pipe. ?However, we can use a neat trick: restart, and the fact that we
> can check headers in vcl_fetch, we haven't grabbed the body yet.
>
> Something like:
>
> sub vcl_recv {
> ?if (req.http.x-pipe && req.restarts > 0) {
> ? ?return(pipe);
> ?}
> }
>
> sub vcl_fetch {
> ?# This is the rule to knock out big files
> ?if ( beresp.http.Content-Length ~ "[0-9]{8,}" ) {
> ? ? set req.http.x-pipe = "1";
> ? ? restart;
> }
>
> I believe this should work. ?It's completely untested and feedback is
> very welcome.


Wow, that's cool. I figured that the body had also been completely
fetched by the time vcl_fetch is invoked, so that changes things
completely. I'll definitely give this a try.

BTW, is there any "cast" trick to be able to use arithmetic operators
like ">" with beresp.http.Content-Length (or other numeric headers)?



From Glen.Kelly at fire.tas.gov.au  Thu Nov  4 23:21:09 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Fri, 5 Nov 2010 10:21:09 +1100
Subject: Varnish 2.1.4 rpm's - bad key error when installing on Centos
	5.5 x86
In-Reply-To: <AANLkTi=1vJ6kC9mJYpwmub0hsRoxCenxP0WzaawUR3+O@mail.gmail.com>
References: <Act74xB84v0tjXmARRurjFpm8yFobA==>
	<674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>
	<AANLkTi=1vJ6kC9mJYpwmub0hsRoxCenxP0WzaawUR3+O@mail.gmail.com>
Message-ID: <674B23B1CEE7B140B4868996CE484506109C8A0761@hoexchange1.fire.tas.gov.au>

Hi Ruslan

That sort of worked, I had to use the -nosignature flag instead. Also because of dependencies it would not let me upgrade, I had to uninstall and reinstall.

But big negative is that it breaks the rpm -q option. Until these rpm's are fixed I cannot use them in production. I assume the developers read this mailing list but in case they don't how do I go about raising a bug report?

(root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-libs-2.1.4-1.el5.x86_64.rpm
error: Failed dependencies:
        varnish-libs = 2.1.3-1.el5 is needed by (installed) varnish-2.1.3-1.el5.x86_64

(root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-2.1.4-1.el5.x86_64.rpm
error: Failed dependencies:
        varnish-libs = 2.1.4-1.el5 is needed by varnish-2.1.4-1.el5.x86_64

 (root at bolrpdev1:/tmp)# rpm -qa | grep varn
varnish-libs-2.1.3-1.el5
varnish-2.1.3-1.el5

(root at bolrpdev1:/tmp)# rpm -e varnish-2.1.3-1.el5
warning: /etc/varnish/default.vcl saved as /etc/varnish/default.vcl.rpmsave
warning: /etc/sysconfig/varnish saved as /etc/sysconfig/varnish.rpmsave
warning: /etc/logrotate.d/varnish saved as /etc/logrotate.d/varnish.rpmsave

(root at bolrpdev1:/tmp)# rpm -e varnish-libs-2.1.3-1.el5

(root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-libs-2.1.4-1.el5.x86_64.rpm

(root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-2.1.4-1.el5.x86_64.rpm

(root at bolrpdev1:/tmp)# rpm -qa | grep varn
error: rpmdbNextIterator: skipping h#     395 Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
error: rpmdbNextIterator: skipping h#     394 Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb

(root at bolrpdev1:/tmp)# service varnish start
Starting varnish HTTP accelerator:                         [  OK  ]

(root at bolrpdev1:/tmp)# service varnishlog start
Starting varnish logging daemon:                           [  OK  ]

(root at bolrpdev1:/tmp)# varnishd -V
varnishd (varnish-2.1.4 SVN 5447M)
Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS

Regards Glen

[cid:tasfire_wakeup_small4cad.jpg]<http://www.fire.tas.gov.au/mysite/Show?pageId=colDayLightSavings>
P Do you need to print this? Consider the environment, prevent paper waste.
CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.

From: Ruslan Sivak [mailto:russ at vshift.com]
Sent: Friday, 5 November 2010 12:47 AM
To: Glen Kelly
Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86


The workaround is to download the 2 rpms manually and run

rpm --signature -Uvh varnish*

I hope whoever is generating the rpms fixes this soon as well as generates some x32 rpms.

Is this possibly the wrong list to ask for this?  Should it be on the  dev list, or should I file an issue in the bug tracker?

Russ
On Nov 4, 2010 1:42 AM, "Glen Kelly" <Glen.Kelly at fire.tas.gov.au<mailto:Glen.Kelly at fire.tas.gov.au>> wrote:
> Hi
>
> I am receiving the same errors as described here when installing on Centos 5.5 x86.
>
> error: rpmts_HdrFromFdno: Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
>
> Is there a known solution?
>
> Also will the 2.1.4 rpm's be eventually available from http://sourceforge.net/projects/varnish/files/?
>
> Regards Glen
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101105/d6a9e761/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tasfire_wakeup_small4cad.jpg
Type: image/jpeg
Size: 27459 bytes
Desc: tasfire_wakeup_small4cad.jpg
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101105/d6a9e761/attachment.jpg>

From lists at rtty.us  Thu Nov  4 23:21:29 2010
From: lists at rtty.us (Bob Camp)
Date: Thu, 4 Nov 2010 19:21:29 -0400
Subject: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5
	x86
In-Reply-To: <674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>
References: <674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>
Message-ID: <376F6E14-4F86-45E7-BA5D-159D494993F1@rtty.us>

Hi

It's a known issue that is being worked on. Hopefully there will be a solution soon.

Bob


On Nov 4, 2010, at 1:42 AM, Glen Kelly wrote:

> Hi
>  
> I am receiving the same errors as described here when installing on Centos 5.5 x86.
>  
> error: rpmts_HdrFromFdno: Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
>  
> Is there a known solution?
>  
> Also will the 2.1.4 rpm?s be eventually available from http://sourceforge.net/projects/varnish/files/?
>  
> Regards Glen
>  
> <tasfire_wakeup_small440d.jpg>
> P Do you need to print this? Consider the environment, prevent paper waste.
> CONFIDENTIALITY NOTICE AND DISCLAIMER
> The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.
> 
> <tasfire_wakeup_small440d.jpg>_______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101104/02c60a0b/attachment.html>

From Glen.Kelly at fire.tas.gov.au  Thu Nov  4 23:28:16 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Fri, 5 Nov 2010 10:28:16 +1100
Subject: Varnish 2.1.4 rpm's - bad key error when installing on Centos
	5.5 x86
In-Reply-To: <376F6E14-4F86-45E7-BA5D-159D494993F1@rtty.us>
References: <674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>
	<376F6E14-4F86-45E7-BA5D-159D494993F1@rtty.us>
Message-ID: <674B23B1CEE7B140B4868996CE484506109C8A0764@hoexchange1.fire.tas.gov.au>

Hi Bob

Good. Thank you for confirming that it is a known issue and it is being worked on.

Glen

From: Bob Camp [mailto:lists at rtty.us]
Sent: Friday, 5 November 2010 10:21 AM
To: Glen Kelly
Cc: varnish-misc at varnish-cache.org
Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86

Hi

It's a known issue that is being worked on. Hopefully there will be a solution soon.

Bob


On Nov 4, 2010, at 1:42 AM, Glen Kelly wrote:


Hi

I am receiving the same errors as described here when installing on Centos 5.5 x86.

error: rpmts_HdrFromFdno: Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb

Is there a known solution?

Also will the 2.1.4 rpm's be eventually available from http://sourceforge.net/projects/varnish/files/?

Regards Glen

<tasfire_wakeup_small440d.jpg><http://www.fire.tas.gov.au/mysite/Show?pageId=colDayLightSavings>

P Do you need to print this? Consider the environment, prevent paper waste.

CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.


<tasfire_wakeup_small440d.jpg>_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101105/12a76b87/attachment.html>

From lists at rtty.us  Fri Nov  5 00:17:42 2010
From: lists at rtty.us (Bob Camp)
Date: Thu, 4 Nov 2010 20:17:42 -0400
Subject: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5
	x86
In-Reply-To: <674B23B1CEE7B140B4868996CE484506109C8A0761@hoexchange1.fire.tas.gov.au>
References: <Act74xB84v0tjXmARRurjFpm8yFobA==>
	<674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>
	<AANLkTi=1vJ6kC9mJYpwmub0hsRoxCenxP0WzaawUR3+O@mail.gmail.com>
	<674B23B1CEE7B140B4868996CE484506109C8A0761@hoexchange1.fire.tas.gov.au>
Message-ID: <1FE6E036-49E5-452D-8E94-997A83973C3B@rtty.us>

Hi

If you have both of the files present you can do a rpm --nosignature -Uvh varnish*

That will get rid of the failed dependency issue.

Bob

On Nov 4, 2010, at 7:21 PM, Glen Kelly wrote:

> Hi Ruslan
>  
> That sort of worked, I had to use the ?nosignature flag instead. Also because of dependencies it would not let me upgrade, I had to uninstall and reinstall.
>  
> But big negative is that it breaks the rpm -q option. Until these rpm?s are fixed I cannot use them in production. I assume the developers read this mailing list but in case they don?t how do I go about raising a bug report?
>  
> (root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-libs-2.1.4-1.el5.x86_64.rpm
> error: Failed dependencies:
>         varnish-libs = 2.1.3-1.el5 is needed by (installed) varnish-2.1.3-1.el5.x86_64
>  
> (root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-2.1.4-1.el5.x86_64.rpm
> error: Failed dependencies:
>         varnish-libs = 2.1.4-1.el5 is needed by varnish-2.1.4-1.el5.x86_64
>  
>  (root at bolrpdev1:/tmp)# rpm -qa | grep varn
> varnish-libs-2.1.3-1.el5
> varnish-2.1.3-1.el5
>  
> (root at bolrpdev1:/tmp)# rpm -e varnish-2.1.3-1.el5
> warning: /etc/varnish/default.vcl saved as /etc/varnish/default.vcl.rpmsave
> warning: /etc/sysconfig/varnish saved as /etc/sysconfig/varnish.rpmsave
> warning: /etc/logrotate.d/varnish saved as /etc/logrotate.d/varnish.rpmsave
>  
> (root at bolrpdev1:/tmp)# rpm -e varnish-libs-2.1.3-1.el5
>  
> (root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-libs-2.1.4-1.el5.x86_64.rpm
>  
> (root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-2.1.4-1.el5.x86_64.rpm
>  
> (root at bolrpdev1:/tmp)# rpm -qa | grep varn
> error: rpmdbNextIterator: skipping h#     395 Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
> error: rpmdbNextIterator: skipping h#     394 Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
>  
> (root at bolrpdev1:/tmp)# service varnish start
> Starting varnish HTTP accelerator:                         [  OK  ]
>  
> (root at bolrpdev1:/tmp)# service varnishlog start
> Starting varnish logging daemon:                           [  OK  ]
>  
> (root at bolrpdev1:/tmp)# varnishd -V
> varnishd (varnish-2.1.4 SVN 5447M)
> Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS
>  
> Regards Glen
>  
> <tasfire_wakeup_small4cad.jpg>
> P Do you need to print this? Consider the environment, prevent paper waste.
> CONFIDENTIALITY NOTICE AND DISCLAIMER
> The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.
> 
> From: Ruslan Sivak [mailto:russ at vshift.com] 
> Sent: Friday, 5 November 2010 12:47 AM
> To: Glen Kelly
> Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86
>  
> The workaround is to download the 2 rpms manually and run
> 
> rpm --signature -Uvh varnish*
> 
> I hope whoever is generating the rpms fixes this soon as well as generates some x32 rpms.
> 
> Is this possibly the wrong list to ask for this?  Should it be on the  dev list, or should I file an issue in the bug tracker?
> 
> Russ
> 
> On Nov 4, 2010 1:42 AM, "Glen Kelly" <Glen.Kelly at fire.tas.gov.au> wrote:
> > Hi
> > 
> > I am receiving the same errors as described here when installing on Centos 5.5 x86.
> > 
> > error: rpmts_HdrFromFdno: Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
> > 
> > Is there a known solution?
> > 
> > Also will the 2.1.4 rpm's be eventually available from http://sourceforge.net/projects/varnish/files/?
> > 
> > Regards Glen
> >
> <tasfire_wakeup_small4cad.jpg>_______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101104/01cb4732/attachment.html>

From Glen.Kelly at fire.tas.gov.au  Fri Nov  5 00:28:53 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Fri, 5 Nov 2010 11:28:53 +1100
Subject: Varnish 2.1.4 rpm's - bad key error when installing on Centos
	5.5 x86
In-Reply-To: <1FE6E036-49E5-452D-8E94-997A83973C3B@rtty.us>
References: <Act74xB84v0tjXmARRurjFpm8yFobA==>
	<674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>
	<AANLkTi=1vJ6kC9mJYpwmub0hsRoxCenxP0WzaawUR3+O@mail.gmail.com>
	<674B23B1CEE7B140B4868996CE484506109C8A0761@hoexchange1.fire.tas.gov.au>
	<1FE6E036-49E5-452D-8E94-997A83973C3B@rtty.us>
Message-ID: <674B23B1CEE7B140B4868996CE484506109C8A0793@hoexchange1.fire.tas.gov.au>

Hi Bob

Ah, I understand now.

I assumed the * in Ruslan's example was just for me to change and insert the relevant filename, I see now he actually meant it as a literal '*'. Oops.

That is interesting to know the rpm can resolve interlinking dependencies like that, an upgrade will be considerably faster if I don't need to uninstall, reinstall and then reconfigure.

Thank you for the clarification.

Regards Glen

From: Bob Camp [mailto:lists at rtty.us]
Sent: Friday, 5 November 2010 11:18 AM
To: Glen Kelly
Cc: varnish-misc at varnish-cache.org
Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86

Hi

If you have both of the files present you can do a rpm --nosignature -Uvh varnish*

That will get rid of the failed dependency issue.

Bob

On Nov 4, 2010, at 7:21 PM, Glen Kelly wrote:


Hi Ruslan

That sort of worked, I had to use the -nosignature flag instead. Also because of dependencies it would not let me upgrade, I had to uninstall and reinstall.

But big negative is that it breaks the rpm -q option. Until these rpm's are fixed I cannot use them in production. I assume the developers read this mailing list but in case they don't how do I go about raising a bug report?

(root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-libs-2.1.4-1.el5.x86_64.rpm
error: Failed dependencies:
        varnish-libs = 2.1.3-1.el5 is needed by (installed) varnish-2.1.3-1.el5.x86_64

(root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-2.1.4-1.el5.x86_64.rpm
error: Failed dependencies:
        varnish-libs = 2.1.4-1.el5 is needed by varnish-2.1.4-1.el5.x86_64

 (root at bolrpdev1:/tmp)# rpm -qa | grep varn
varnish-libs-2.1.3-1.el5
varnish-2.1.3-1.el5

(root at bolrpdev1:/tmp)# rpm -e varnish-2.1.3-1.el5
warning: /etc/varnish/default.vcl saved as /etc/varnish/default.vcl.rpmsave
warning: /etc/sysconfig/varnish saved as /etc/sysconfig/varnish.rpmsave
warning: /etc/logrotate.d/varnish saved as /etc/logrotate.d/varnish.rpmsave

(root at bolrpdev1:/tmp)# rpm -e varnish-libs-2.1.3-1.el5

(root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-libs-2.1.4-1.el5.x86_64.rpm

(root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-2.1.4-1.el5.x86_64.rpm

(root at bolrpdev1:/tmp)# rpm -qa | grep varn
error: rpmdbNextIterator: skipping h#     395 Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
error: rpmdbNextIterator: skipping h#     394 Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb

(root at bolrpdev1:/tmp)# service varnish start
Starting varnish HTTP accelerator:                         [  OK  ]

(root at bolrpdev1:/tmp)# service varnishlog start
Starting varnish logging daemon:                           [  OK  ]

(root at bolrpdev1:/tmp)# varnishd -V
varnishd (varnish-2.1.4 SVN 5447M)
Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS

Regards Glen

<tasfire_wakeup_small4cad.jpg><http://www.fire.tas.gov.au/mysite/Show?pageId=colDayLightSavings>

P Do you need to print this? Consider the environment, prevent paper waste.

CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.


From: Ruslan Sivak [mailto:russ at vshift.com]
Sent: Friday, 5 November 2010 12:47 AM
To: Glen Kelly
Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86


The workaround is to download the 2 rpms manually and run

rpm --signature -Uvh varnish*

I hope whoever is generating the rpms fixes this soon as well as generates some x32 rpms.

Is this possibly the wrong list to ask for this?  Should it be on the  dev list, or should I file an issue in the bug tracker?

Russ
On Nov 4, 2010 1:42 AM, "Glen Kelly" <Glen.Kelly at fire.tas.gov.au<mailto:Glen.Kelly at fire.tas.gov.au>> wrote:
> Hi
>
> I am receiving the same errors as described here when installing on Centos 5.5 x86.
>
> error: rpmts_HdrFromFdno: Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
>
> Is there a known solution?
>
> Also will the 2.1.4 rpm's be eventually available from http://sourceforge.net/projects/varnish/files/?
>
> Regards Glen
>
<tasfire_wakeup_small4cad.jpg>_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101105/7afa3a81/attachment.html>

From lists at rtty.us  Fri Nov  5 00:29:45 2010
From: lists at rtty.us (Bob Camp)
Date: Thu, 4 Nov 2010 20:29:45 -0400
Subject: Wiki hash_always_miss
Message-ID: <F07C3B8C-641D-4E7E-88F3-D2CF8BFB965A@rtty.us>

Hi

The wiki shows:

# Command to refresh cached front page
$ GET -H 'X-REFRESH: DOIT' http://my.site.com/

# VCL
sub vcl_recv {
    if (req.http.X-REFRESH) {
        # Force a cache miss
        
set req.hash_always_miss
 = 1;
    }
}

That does not work. 

The varnishtest test suite shows:

  sub vcl_recv {
	                if (req.http.x-missit == "1") {
	                        set req.hash_always_miss = true;
	                }
	        }

Setting it to true rather than 1 does indeed work.

Bob



From tfheen at varnish-software.com  Fri Nov  5 08:10:08 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Fri, 05 Nov 2010 09:10:08 +0100
Subject: Dealing with large unwanted objects
In-Reply-To: <AANLkTinFKoYCon0mWqdCjMk=DSh9VuMGGeHyPcrFG-Uu@mail.gmail.com>
	(Mark Moseley's message of "Thu, 4 Nov 2010 11:42:44 -0700")
References: <AANLkTi=SSBCF+_EGa+TYZmEarvR7hmTW4jp4L8p0iPEv@mail.gmail.com>
	<874obyzwxt.fsf@qurzaw.linpro.no>
	<AANLkTinFKoYCon0mWqdCjMk=DSh9VuMGGeHyPcrFG-Uu@mail.gmail.com>
Message-ID: <877hgsur6n.fsf@qurzaw.linpro.no>

]] Mark Moseley 

| Wow, that's cool. I figured that the body had also been completely
| fetched by the time vcl_fetch is invoked, so that changes things
| completely. I'll definitely give this a try.

It is in 2.0, but not in 2.1.

| BTW, is there any "cast" trick to be able to use arithmetic operators
| like ">" with beresp.http.Content-Length (or other numeric headers)?

There'll be a way to do that in 3.0, yes.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From bob at heavyh2o.com  Fri Nov  5 15:35:19 2010
From: bob at heavyh2o.com (Robert Brewin)
Date: Fri, 5 Nov 2010 08:35:19 -0700
Subject: Cache hits / misses
Message-ID: <1ef401cb7cff$0e3f8660$2abe9320$@heavyh2o.com>

Question . for statistics and histograms, does the following VCL result
statistically as a "miss" ?

 

sub vcl_recv {

        if ( req.http.host ~ "^(www.)?example.com" ) {

 

                return(pass);     # OR . "return(pipe);"

        }

        ..

}

 

I've used both "pass" and "pipe" and trying to narrow down what seems like
mostly misses ( at least when checking *stat and *hist ) . unfortunately, I
have both highly variable web service calls and invocations against (mostly
static) web sites through the same varnish instance and I'm trying to
eliminate the above as skewing the statistics (they should just be passed
through and ignored by varnish).  

 

Rather check with folks on this list before substantially changing the
architecture or handling the service invocations without going through the
same "gateway" into the web/service cluster.

 

And yes, I've unset all cookies and it looks like at least on the web side
of the equation, it's caching properly . it's the service calls which seem
to be registering as "misses".  Note that I realize this may be expected
behavior . but if not, is there a way of having certain calls ignored and
passed through entirely .

 

Cheers,

 

Bob

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101105/3181a320/attachment.html>

From hijinks at gmail.com  Fri Nov  5 18:13:12 2010
From: hijinks at gmail.com (Mike Zupan)
Date: Fri, 5 Nov 2010 14:13:12 -0400
Subject: VarnishSpy
Message-ID: <AANLkTikR5NpMYmY-vgR+Fv3fBxHFMdAvVpPDZjE_RN=1@mail.gmail.com>

I liked

http://team-soundcloud.no.de/

Right now my company doesn't cache a lot of pages so varnishtop via a
webpage was almost useless to look at. So that is where varnishspy
comes in

https://github.com/mzupan/varnishspy

It uses varnishncsa via node.js to talk to a webclient to get what
traffic is happening as you load the page.

Hope it helps some people who want a real time look at what urls are popular.

Mike



From moseleymark at gmail.com  Fri Nov  5 18:54:37 2010
From: moseleymark at gmail.com (Mark Moseley)
Date: Fri, 5 Nov 2010 11:54:37 -0700
Subject: Dealing with large unwanted objects
In-Reply-To: <877hgsur6n.fsf@qurzaw.linpro.no>
References: <AANLkTi=SSBCF+_EGa+TYZmEarvR7hmTW4jp4L8p0iPEv@mail.gmail.com>
	<874obyzwxt.fsf@qurzaw.linpro.no>
	<AANLkTinFKoYCon0mWqdCjMk=DSh9VuMGGeHyPcrFG-Uu@mail.gmail.com>
	<877hgsur6n.fsf@qurzaw.linpro.no>
Message-ID: <AANLkTimmDJeD1k6raZ=zyqsSKHMNMe-e4N74PHG25-QM@mail.gmail.com>

On Fri, Nov 5, 2010 at 1:10 AM, Tollef Fog Heen
<tfheen at varnish-software.com> wrote:
> ]] Mark Moseley
>
> | Wow, that's cool. I figured that the body had also been completely
> | fetched by the time vcl_fetch is invoked, so that changes things
> | completely. I'll definitely give this a try.
>
> It is in 2.0, but not in 2.1.
>
> | BTW, is there any "cast" trick to be able to use arithmetic operators
> | like ">" with beresp.http.Content-Length (or other numeric headers)?
>
> There'll be a way to do that in 3.0, yes.

So if it's not fetched the complete body yet, once 'restart' is
called, does varnish just dump the body? When I tried out your recipe
(which works like a charm, btw), I noticed that varnish still reads
the entirety of the first (i.e. the restarted one) request's response
body, before proceeding to fetch the restarted request. All this
happens over localhost, so it's quite fast, but in the interest of
efficiency, is there something I can set or call so that it closes
that first connection almost immediately? Having to refetch a 800meg
file off of NFS might hurt -- even if a good chunk of it is still in
the OS block cache.

To test, I set up Apache so that it'd return a 403 if the 'restart'
header is on the request, so that the first restarted request gets
fetched completely but the second one is basically empty. Given a 200
meg file, on a newly started varnish, the "SMA bytes allocated"
immediately jumped to 224265008. Once the first request finished, only
then did 'SMA bytes free' show up, also with 224265008. That would
make it appear that is storing the response. Or is it allocating the
memory in name only -- i.e. not actually doing anything -- and
possibly not causing varnish to knock things off the LRU?

BTW, this is 64-bit 2.1.4 on Debian Lenny with "-s malloc,4g".



From karel.minarik at gmail.com  Sat Nov  6 10:18:00 2010
From: karel.minarik at gmail.com (=?UTF-8?Q?Karel_Mina=C5=99=C3=ADk?=)
Date: Sat, 6 Nov 2010 11:18:00 +0100
Subject: Using Varnish to accelerate CouchDB
Message-ID: <C564B65C-396C-47BF-A78E-2F1B03A5F955@gmail.com>

Hello,

I've spent the last couple of days trying to figure out how to  
accelerate CouchDB [http://couchdb.org/] with Varnish. I got kinda  
lost and would like to ask for some help.

CouchDB sends an ETag header in the request:

   $ curl -I http://localhost:5984/test/abc1
   ...
   Etag: "3-6585b511acdd9a73040e673329369ff6"
   Cache-Control: must-revalidate

My original thought was that I could use this in Varnish to speed up  
responses from Couch, because Varnish could "accumulate" requests to  
the backend in certain time frame, and read cached object from memory,  
as opposed from disk. (CouchDB view or CouchDB-Lucene queries send  
ETag in their responses as well, so this would work very well accross  
the whole database.)

However, I can't find any definite information about if and how  
Varnish uses ETags? Is this a bad/impossible approach in Varnish?

In my tests, I got some *huge* speedup (with a default setup) when  
querying Couch via Varnish: the mean response time dropped around  
half, sometimes more (from ~ 2-3msec to ~0.5-1msec), and the requests  
per second went up almost ten times (measured by simple ApacheBench  
tests).

I couldn't, though, make the cached response to expire by changing the  
document in the database and thus changing Etag for its response.  
(Which I consider strange given the "must-revalidate" header?)

In the end, I suspect this is because Varnish has some default TTL  
(120sec?) and that is what the acceleration is based on. Is that so?  
(Again, I couldn't find any definite info about this in docs, or in  
the default.vcl.)

What would you consider to be the most effective Varnish configuration  
for CouchDB? Is time-based caching (eg. setting the Expires or Last- 
Modified header) the only option?

Thanks for any help in advance!

Karel



From michal.taborsky at nrholding.com  Sat Nov  6 10:34:24 2010
From: michal.taborsky at nrholding.com (Michal Taborsky - Netretail Holding)
Date: Sat, 06 Nov 2010 11:34:24 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <C564B65C-396C-47BF-A78E-2F1B03A5F955@gmail.com>
References: <C564B65C-396C-47BF-A78E-2F1B03A5F955@gmail.com>
Message-ID: <4CD52F30.1060404@nrholding.com>

Hi Karel,

you will need some kind of cache invalidation. When you change the data 
in CouchDB, your application will have to notify Varnish the document 
has changed and should purge it form the cache. Next request for the 
same object will fetch the object from CouchDB and store the new version 
in Varnish. You do not need ETags for this. You only need to set correct 
caching headers that will tell Varnish to cache the object forever. But 
you definitely need the cache invalidation in your app.

Or, if you don't care Varnish will serve stale version of the document, 
you can cache it for some short period without the invalidation.

Michal

Dne 6.11.2010 11:18, Karel Mina??k napsal(a):
> Hello,
>
> I've spent the last couple of days trying to figure out how to 
> accelerate CouchDB [http://couchdb.org/] with Varnish. I got kinda 
> lost and would like to ask for some help.
>
> CouchDB sends an ETag header in the request:
>
>   $ curl -I http://localhost:5984/test/abc1
>   ...
>   Etag: "3-6585b511acdd9a73040e673329369ff6"
>   Cache-Control: must-revalidate
>
> My original thought was that I could use this in Varnish to speed up 
> responses from Couch, because Varnish could "accumulate" requests to 
> the backend in certain time frame, and read cached object from memory, 
> as opposed from disk. (CouchDB view or CouchDB-Lucene queries send 
> ETag in their responses as well, so this would work very well accross 
> the whole database.)
>
> However, I can't find any definite information about if and how 
> Varnish uses ETags? Is this a bad/impossible approach in Varnish?
>
> In my tests, I got some *huge* speedup (with a default setup) when 
> querying Couch via Varnish: the mean response time dropped around 
> half, sometimes more (from ~ 2-3msec to ~0.5-1msec), and the requests 
> per second went up almost ten times (measured by simple ApacheBench 
> tests).
>
> I couldn't, though, make the cached response to expire by changing the 
> document in the database and thus changing Etag for its response. 
> (Which I consider strange given the "must-revalidate" header?)
>
> In the end, I suspect this is because Varnish has some default TTL 
> (120sec?) and that is what the acceleration is based on. Is that so? 
> (Again, I couldn't find any definite info about this in docs, or in 
> the default.vcl.)
>
> What would you consider to be the most effective Varnish configuration 
> for CouchDB? Is time-based caching (eg. setting the Expires or 
> Last-Modified header) the only option?
>
> Thanks for any help in advance!
>
> Karel
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>

-- 
Michal T?borsk?
chief systems architect
Netretail Holding, B.V.
http://www.nrholding.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101106/0a4ea9a7/attachment.html>

From karel.minarik at gmail.com  Sat Nov  6 11:32:11 2010
From: karel.minarik at gmail.com (=?UTF-8?Q?Karel_Mina=C5=99=C3=ADk?=)
Date: Sat, 6 Nov 2010 12:32:11 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
Message-ID: <EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>

Hello, Michal,

> you will need some kind of cache invalidation. When you change the  
> data
> in CouchDB, your application will have to notify Varnish the document
> has changed and should purge it form the cache.

well, that's certainly doable, but I would rather not mess with cache  
invalidation -- that'd bring a whole different layer in the stack.

> Or, if you don't care Varnish will serve stale version of the  
> document,
> you can cache it for some short period without the invalidation.


Yes, that's an approach we could use for certain resources, but not  
ideal for most.

I am still curious if Varnish use ETag based caching at all?

It seems that Squid also does not work with ETags (http://wiki.squid-cache.org/Features/EtagSupport 
). Can someone please clarify this?

The only info I've found is this old thread: http://lists.varnish-cache.org/pipermail/varnish-misc/2008-November/002184.html

Thanks,

Karel




From michal.taborsky at nrholding.com  Sat Nov  6 15:09:58 2010
From: michal.taborsky at nrholding.com (Michal Taborsky - Netretail Holding)
Date: Sat, 06 Nov 2010 16:09:58 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
Message-ID: <4CD56FC6.60303@nrholding.com>

Karel,

> I am still curious if Varnish use ETag based caching at all?

Think of ETag as hash of the document. It can be used to identify 
whether the document is changed. But in order to compare the documents, 
you must check the ETag on client side against ETag on the server. So, 
in order to use ETags, Varnish would have to get the document (or at 
least it's ETag) from backend on each request.

ETags are used in client/server scenario like this:

Request 1:
client: I want /foo
server: (computes the ETag of document) here is the document, also ETag 
of /foo is bar

Request 2:
client: I want /foo, I have cached copy with ETag=bar
server: (computes ETag, it is still bar) you are OK, the document has 
not changed

-document changes-

Request 3:
client: I want /foo, I have cached copy with ETag=bar
server: (computes ETag, it is now baz) here is a new version, it has 
ETag baz

So, you see, in order to use ETags, there has to be direct communication 
between client and the server. If there is a cache in between, ETags are 
useless.

You either have to implement invalidation, or be OK with serving stale 
content. You can contact me off the list (in Czech if you wish :) ) if 
it's still not clear.

Michal

-- 
Michal T?borsk?
chief systems architect
Netretail Holding, B.V.
http://www.nrholding.com




From karel.minarik at gmail.com  Sat Nov  6 15:44:40 2010
From: karel.minarik at gmail.com (=?ISO-8859-2?Q?Karel_Mina=F8=EDk?=)
Date: Sat, 6 Nov 2010 16:44:40 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <4CD56FC6.60303@nrholding.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
	<4CD56FC6.60303@nrholding.com>
Message-ID: <473C4BDE-E73C-40BF-8B22-E7A8E001FCAA@gmail.com>

Hello,

> So, in order to use ETags, Varnish would have to get the document  
> (or at least it's ETag) from backend on each request.
> ...
> So, you see, in order to use ETags, there has to be direct  
> communication between client and the server. If there is a cache in  
> between, ETags are useless.

thanks, but I don't think this is neccessarily true, if:

* The cache can "accumulate" requests to a certain resource for a  
certain (configurable?) period of time (1 second, 1 minute, ...) and  
ask the backend less often -- accelerating througput.

* The cache can return "possibly stale" content immediately and check  
with the backend afterwards (on the background, when n-th next request  
comes, ...) -- accelerating response time.

I am quite sure there are other ways I am not thinking of. And I think  
these or other strategies are doable with Varnish, that time-based  
acceleration is not the only option. Of course its possible/probable I  
am absolutely mistaken :) That's why I am asking if anyone has any  
reccommendations for accelerating Couch with Varnish.

Best,

Karel





From perbu at varnish-software.com  Sat Nov  6 18:48:05 2010
From: perbu at varnish-software.com (Per Buer)
Date: Sat, 6 Nov 2010 19:48:05 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
Message-ID: <AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>

On Sat, Nov 6, 2010 at 12:32 PM, Karel Mina??k <karel.minarik at gmail.com> wrote:
> Hello, Michal,
>
>> you will need some kind of cache invalidation. When you change the data
>> in CouchDB, your application will have to notify Varnish the document
>> has changed and should purge it form the cache.
>
> well, that's certainly doable, but I would rather not mess with cache
> invalidation -- that'd bring a whole different layer in the stack.

I haven't really ever touched Couch, but from what I overheard I think
you can embed custom code into the database, meaning it should not be
to hard to wactually have the database call out to Varnish and
invalidate the content itself.

You might have more luck asking on the Couch list, though.

-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From joe at joetify.com  Sat Nov  6 23:00:25 2010
From: joe at joetify.com (Joe Williams)
Date: Sat, 6 Nov 2010 16:00:25 -0700
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
	<AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>
Message-ID: <32F9E898-589A-4255-A0EB-3BA0C9FC4B77@joetify.com>


On Nov 6, 2010, at 11:48 AM, Per Buer wrote:

> On Sat, Nov 6, 2010 at 12:32 PM, Karel Mina??k <karel.minarik at gmail.com> wrote:
>> Hello, Michal,
>> 
>>> you will need some kind of cache invalidation. When you change the data
>>> in CouchDB, your application will have to notify Varnish the document
>>> has changed and should purge it form the cache.
>> 
>> well, that's certainly doable, but I would rather not mess with cache
>> invalidation -- that'd bring a whole different layer in the stack.
> 
> I haven't really ever touched Couch, but from what I overheard I think
> you can embed custom code into the database, meaning it should not be
> to hard to wactually have the database call out to Varnish and
> invalidate the content itself.
> 
> You might have more luck asking on the Couch list, though.


We (Cloudant) have actually discussed implementing exactly that in CouchDB. I agree with Per that it wouldn't be difficult to do, we just haven't gotten around to it.   TTLs are good enough in some cases but cache invalidation is the only way to go in many.

-Joe


Name: Joseph A. Williams
Email: joe at joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe




From karel.minarik at gmail.com  Sun Nov  7 09:00:52 2010
From: karel.minarik at gmail.com (=?UTF-8?Q?Karel_Mina=C5=99=C3=ADk?=)
Date: Sun, 7 Nov 2010 10:00:52 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
	<AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>
Message-ID: <B1122262-3897-45A1-BA3A-59527ABC47FE@gmail.com>

Hello,

> I haven't really ever touched Couch, but from what I overheard I think
> you can embed custom code into the database, meaning it should not be
> to hard to wactually have the database call out to Varnish and
> invalidate the content itself.

invalidating the cache could be done most conveniently via the  
_changes feed or via the "update_notification" mechanism [http://wiki.apache.org/couchdb/Regenerating_views_on_update 
, probably. Since not only single documents, but also aggregated view  
results or fulltext queries would get cached, it wouldn't be exactly  
trivial to expire everything. "There are two hard things in computer  
science ..."

I am sorry to ask the same dumb questions over and over, but does it  
mean, that Varnish does not work in ETags, ie. does not support  
"Validation" type of caching as opposed to "Expiration" caching in the  
sense of Ryan Tomayko's article [http://tomayko.com/writings/things-caches-do 
] ?

I am well aware that "dumb caching" based on ETags would not  
accelerate anything when it would make 1 "check" request to backend  
for every 1 request to the cache. It was my impression that Varnish  
could be configured to eg. deal with concurrent 100 requests to some  
resource and "accumulate" them, eg. not send them all to the backend.

The reason I am asking is that in a backend such as a Ruby On Rails,  
computing an ETag could be much cheaper then computing and returning  
whole response. So, once again and for the last time, sorry :) --  
there is no way to use Varnish like this? Only time-based (Expires,  
Last-Modified, etc) caching is possible?

> You might have more luck asking on the Couch list, though.

Yes, definitely, I'll ask there for any experience. There seem to be  
rather little information available -- while I am quite certain that  
lots of people is Using Varnish to accelerate Couch.

Thanks!,

Karel



From perbu at varnish-software.com  Sun Nov  7 09:12:04 2010
From: perbu at varnish-software.com (Per Buer)
Date: Sun, 7 Nov 2010 10:12:04 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <B1122262-3897-45A1-BA3A-59527ABC47FE@gmail.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
	<AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>
	<B1122262-3897-45A1-BA3A-59527ABC47FE@gmail.com>
Message-ID: <AANLkTindqdOQ1Fb71pzyeHs+J2P+NT35E3CsVXRrpE70@mail.gmail.com>

On Sun, Nov 7, 2010 at 10:00 AM, Karel Mina??k <karel.minarik at gmail.com> wrote:
>
> I am sorry to ask the same dumb questions over and over, but does it mean,
> that Varnish does not work in ETags, ie. does not support "Validation" type
> of caching as opposed to "Expiration" caching in the sense of Ryan Tomayko's
> article [http://tomayko.com/writings/things-caches-do] ?

Varnish only supports ETags "in front" that is, only when a client
sends us an ETag will we respons correctly. Towards the backend Etags
are not supported although I think there are patches floating around
that might address that. If I recall correctly Slink and some
Rackspace people where eager to see ETags in Varnish.


-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From karel.minarik at gmail.com  Sun Nov  7 09:55:47 2010
From: karel.minarik at gmail.com (=?UTF-8?Q?Karel_Mina=C5=99=C3=ADk?=)
Date: Sun, 7 Nov 2010 10:55:47 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <32F9E898-589A-4255-A0EB-3BA0C9FC4B77@joetify.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
	<AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>
	<32F9E898-589A-4255-A0EB-3BA0C9FC4B77@joetify.com>
Message-ID: <2BF856DF-2D83-4BFF-B729-C42D356CF90D@gmail.com>

Hello,

> We (Cloudant) have actually discussed implementing exactly that in  
> CouchDB. I agree with Per that it wouldn't be difficult to do, we  
> just haven't gotten around to it.

I've noticed some native caching is coming for Couch 1.1.0 [http://www.apacheserver.net/memrioa-management-at1047105.htm 
], but know no details.

> TTLs are good enough in some cases but cache invalidation is the  
> only way to go in many.

Yes, in many cases "expiration" based caching makes sense.  
"Validation" based caching would be really handy, though. But maybe as  
non-trivial as cache invalidation of view results, fulltext queries,  
etc based on changes of individual docs...

Karel



From karel.minarik at gmail.com  Sun Nov  7 10:08:33 2010
From: karel.minarik at gmail.com (=?UTF-8?Q?Karel_Mina=C5=99=C3=ADk?=)
Date: Sun, 7 Nov 2010 11:08:33 +0100
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <AANLkTindqdOQ1Fb71pzyeHs+J2P+NT35E3CsVXRrpE70@mail.gmail.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>
	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>
	<AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>
	<B1122262-3897-45A1-BA3A-59527ABC47FE@gmail.com>
	<AANLkTindqdOQ1Fb71pzyeHs+J2P+NT35E3CsVXRrpE70@mail.gmail.com>
Message-ID: <9C6EC871-AE07-45CF-A8C7-CD3F20D3D8AD@gmail.com>

Hello,

> Varnish only supports ETags "in front" that is, only when a client
> sends us an ETag will we respons correctly. Towards the backend Etags
> are not supported (...)

thanks for clarification, Per!

> If I recall correctly Slink and some
> Rackspace people where eager to see ETags in Varnish.

Yes, that makes sense because specificially in Rails, it could be much  
cheaper to compute an ETag and return 304 than to assemble and return  
the whole response. It would be awesome to make use of that in Varnish.

Karel



From rtshilston at gmail.com  Sun Nov  7 10:28:55 2010
From: rtshilston at gmail.com (Rob S)
Date: Sun, 07 Nov 2010 10:28:55 +0000
Subject: Using Varnish to accelerate CouchDB
In-Reply-To: <B1122262-3897-45A1-BA3A-59527ABC47FE@gmail.com>
References: <mailman.290.1289039670.2113.varnish-misc@varnish-cache.org>	<EFEE03F4-AD88-408B-BABE-1D94A728D86C@gmail.com>	<AANLkTimsMfBYQ8CxK_muwuZGyUZSvOFQOKUReYAu3Sw_@mail.gmail.com>
	<B1122262-3897-45A1-BA3A-59527ABC47FE@gmail.com>
Message-ID: <4CD67F67.7010308@gmail.com>

On 07/11/2010 09:00, Karel Mina??k wrote:
>  It was my impression that Varnish could be configured to eg. deal 
> with concurrent 100 requests to some resource and "accumulate" them, 
> eg. not send them all to the backend.
>
Yes - I believe that's right.  If a request is received for a slow 
object which is not in cache, and whilst that's being generated by the 
backend additional clients request the same object, then all requests 
are aggregated.

I think you'll need to look at the documentation around the 'grace' 
values to tune this to behave exactly as you want.

Rob



From klinux at gmail.com  Sun Nov  7 19:26:26 2010
From: klinux at gmail.com (Kleber Rocha)
Date: Sun, 7 Nov 2010 17:26:26 -0200
Subject: Varnish 2.1.4
Message-ID: <AANLkTi=DfnuZPQVD_Cn4BuUbpcf_G8yPmNY2YKLw_Fuw@mail.gmail.com>

I upgraded varnish 2.1.3 to 2.1.4 and my server is responding very slow, I
went back version to 2.1.3 and  all worked fine.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101107/40de5062/attachment.html>

From racemd at verizon.net  Mon Nov  8 15:50:00 2010
From: racemd at verizon.net (Roland Rebstock)
Date: Mon, 08 Nov 2010 10:50:00 -0500
Subject: Attempt to make Varnish on RHEL5 64bit error during make  you need
	rst2man installed to make dist
Message-ID: <000c01cb7f5c$9a7dde10$cf799a30$@net>

All, can someone tell me how to get around this error during make 

 

Making all in bin

make[2]: Entering directory `/tmp/2.1/varnish-cache/bin'

Making all in varnishadm

make[3]: Entering directory `/tmp/2.1/varnish-cache/bin/varnishadm'

========================================

You need rst2man installed to make dist

========================================

make[3]: *** [varnishadm.1] Error 1

make[3]: Leaving directory `/tmp/2.1/varnish-cache/bin/varnishadm'

make[2]: *** [all-recursive] Error 1

make[2]: Leaving directory `/tmp/2.1/varnish-cache/bin'

make[1]: *** [all-recursive] Error 1

make[1]: Leaving directory `/tmp/2.1/varnish-cache'

make: *** [all] Error 2

 

 

 

[root at new-host varnish-cache]# ./configure --bindir=/export/home/varnish/bin
--sbindir=/export/home/varnish/sbin
--libexecdir=/export/home/varnish/libexec
--datadir=/export/home/varnish/share --sysconfdir=/export/home/varnish/etc
--sharedstatedir=/export/home/varnish/com --libdir=/export/home/varnish/lib
-localstatedir=/export/home/varnish/var
--includedir=/export/home/varnish/include
--infodir=/export/home/varnish/info --prefix=/export/home/vanrish

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101108/c4330f21/attachment.html>

From perbu at varnish-software.com  Mon Nov  8 17:15:01 2010
From: perbu at varnish-software.com (Per Buer)
Date: Mon, 8 Nov 2010 18:15:01 +0100
Subject: Attempt to make Varnish on RHEL5 64bit error during make you need
	rst2man installed to make dist
In-Reply-To: <000c01cb7f5c$9a7dde10$cf799a30$@net>
References: <000c01cb7f5c$9a7dde10$cf799a30$@net>
Message-ID: <AANLkTimqQgsaZO0XeKmFz3n1uvXeZStAYPG=K0tDsNi+@mail.gmail.com>

Hi Roland.

Have you tried to install rst2man?

Please note - you normally don't need to do "make dist" - the man
pages are already generated in the tar-ball you've downloaded.

Per.

On Mon, Nov 8, 2010 at 4:50 PM, Roland Rebstock <racemd at verizon.net> wrote:
> All, can someone tell me how to get around this error during make
>
>
>
> Making all in bin
>
> make[2]: Entering directory `/tmp/2.1/varnish-cache/bin'
>
> Making all in varnishadm
>
> make[3]: Entering directory `/tmp/2.1/varnish-cache/bin/varnishadm'
>
> ========================================
>
> You need rst2man installed to make dist
>
> ========================================
>
> make[3]: *** [varnishadm.1] Error 1
>
> make[3]: Leaving directory `/tmp/2.1/varnish-cache/bin/varnishadm'
>
> make[2]: *** [all-recursive] Error 1
>
> make[2]: Leaving directory `/tmp/2.1/varnish-cache/bin'
>
> make[1]: *** [all-recursive] Error 1
>
> make[1]: Leaving directory `/tmp/2.1/varnish-cache'
>
> make: *** [all] Error 2
>
>
>
>
>
>
>
> [root at new-host varnish-cache]# ./configure --bindir=/export/home/varnish/bin
> --sbindir=/export/home/varnish/sbin
> --libexecdir=/export/home/varnish/libexec
> --datadir=/export/home/varnish/share --sysconfdir=/export/home/varnish/etc
> --sharedstatedir=/export/home/varnish/com --libdir=/export/home/varnish/lib
> -localstatedir=/export/home/varnish/var
> --includedir=/export/home/varnish/include
> --infodir=/export/home/varnish/info --prefix=/export/home/vanrish
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From scaunter at topscms.com  Mon Nov  8 17:59:00 2010
From: scaunter at topscms.com (Caunter, Stefan)
Date: Mon, 8 Nov 2010 12:59:00 -0500
Subject: Varnish 2.1.4
In-Reply-To: <AANLkTi=DfnuZPQVD_Cn4BuUbpcf_G8yPmNY2YKLw_Fuw@mail.gmail.com>
References: <AANLkTi=DfnuZPQVD_Cn4BuUbpcf_G8yPmNY2YKLw_Fuw@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902C77D561@TMG-EVS02.torstar.net>

Which server? The backend or varnish? You will have lost all cache so
that will hit the backend for a while.

 

Stefan Caunter :: Senior Systems Administrator :: TOPS

e: scaunter at topscms.com  ::  m: (416) 561-4871

www.thestar.com www.topscms.com

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Kleber
Rocha
Sent: November-07-10 2:26 PM
To: varnish-misc at varnish-cache.org
Subject: Varnish 2.1.4

 

I upgraded varnish 2.1.3 to 2.1.4 and my server is responding very slow,
I went back version to 2.1.3 and  all worked fine.

Thanks

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101108/ea7f89a2/attachment.html>

From klinux at gmail.com  Mon Nov  8 18:35:37 2010
From: klinux at gmail.com (Kleber Rocha)
Date: Mon, 8 Nov 2010 16:35:37 -0200
Subject: Varnish 2.1.4
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902C77D561@TMG-EVS02.torstar.net>
References: <AANLkTi=DfnuZPQVD_Cn4BuUbpcf_G8yPmNY2YKLw_Fuw@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902C77D561@TMG-EVS02.torstar.net>
Message-ID: <AANLkTikGNvX7bm04oVy77V09NTEYtzfELeSCJhP3VVrX@mail.gmail.com>

The varnish server, the time response it's very long, the backends are
responding very quickly, when accessed directly. other thing that I can see,
varnishncsa don't log anything. this is a bug or I comited some error?

Thanks

2010/11/8 Caunter, Stefan <scaunter at topscms.com>

>  Which server? The backend or varnish? You will have lost all cache so
> that will hit the backend for a while.
>
>
>
> Stefan Caunter :: Senior Systems Administrator :: TOPS
>
> e: scaunter at topscms.com  ::  m: (416) 561-4871
>
> www.thestar.com www.topscms.com
>
>
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Kleber Rocha
> *Sent:* November-07-10 2:26 PM
> *To:* varnish-misc at varnish-cache.org
> *Subject:* Varnish 2.1.4
>
>
>
> I upgraded varnish 2.1.3 to 2.1.4 and my server is responding very slow, I
> went back version to 2.1.3 and  all worked fine.
>
> Thanks
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101108/6dcfe0eb/attachment.html>

From racemd at verizon.net  Mon Nov  8 20:02:30 2010
From: racemd at verizon.net (Roland Rebstock)
Date: Mon, 08 Nov 2010 15:02:30 -0500
Subject: Attempt to make Varnish on RHEL5 64bit error during make you need
	rst2man installed to make dist
In-Reply-To: <AANLkTimqQgsaZO0XeKmFz3n1uvXeZStAYPG=K0tDsNi+@mail.gmail.com>
References: <000c01cb7f5c$9a7dde10$cf799a30$@net>
	<AANLkTimqQgsaZO0XeKmFz3n1uvXeZStAYPG=K0tDsNi+@mail.gmail.com>
Message-ID: <002e01cb7f7f$e093d530$a1bb7f90$@net>

Is that a RPM for RHEL 5?  I do not see it on rhn.com. for the 2nd part
after the configure, are you saying I don?t have to run "make"?  I never ran
make dist, just "make".



-----Original Message-----
From: Per Buer [mailto:perbu at varnish-software.com] 
Sent: Monday, November 08, 2010 12:15 PM
To: Roland Rebstock
Cc: varnish-misc at varnish-cache.org
Subject: Re: Attempt to make Varnish on RHEL5 64bit error during make you
need rst2man installed to make dist

Hi Roland.

Have you tried to install rst2man?

Please note - you normally don't need to do "make dist" - the man
pages are already generated in the tar-ball you've downloaded.

Per.

On Mon, Nov 8, 2010 at 4:50 PM, Roland Rebstock <racemd at verizon.net> wrote:
> All, can someone tell me how to get around this error during make
>
>
>
> Making all in bin
>
> make[2]: Entering directory `/tmp/2.1/varnish-cache/bin'
>
> Making all in varnishadm
>
> make[3]: Entering directory `/tmp/2.1/varnish-cache/bin/varnishadm'
>
> ========================================
>
> You need rst2man installed to make dist
>
> ========================================
>
> make[3]: *** [varnishadm.1] Error 1
>
> make[3]: Leaving directory `/tmp/2.1/varnish-cache/bin/varnishadm'
>
> make[2]: *** [all-recursive] Error 1
>
> make[2]: Leaving directory `/tmp/2.1/varnish-cache/bin'
>
> make[1]: *** [all-recursive] Error 1
>
> make[1]: Leaving directory `/tmp/2.1/varnish-cache'
>
> make: *** [all] Error 2
>
>
>
>
>
>
>
> [root at new-host varnish-cache]# ./configure
--bindir=/export/home/varnish/bin
> --sbindir=/export/home/varnish/sbin
> --libexecdir=/export/home/varnish/libexec
> --datadir=/export/home/varnish/share --sysconfdir=/export/home/varnish/etc
> --sharedstatedir=/export/home/varnish/com
--libdir=/export/home/varnish/lib
> -localstatedir=/export/home/varnish/var
> --includedir=/export/home/varnish/include
> --infodir=/export/home/varnish/info --prefix=/export/home/vanrish
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer




From perbu at varnish-software.com  Mon Nov  8 20:09:59 2010
From: perbu at varnish-software.com (Per Buer)
Date: Mon, 8 Nov 2010 21:09:59 +0100
Subject: Attempt to make Varnish on RHEL5 64bit error during make you need
	rst2man installed to make dist
In-Reply-To: <002e01cb7f7f$e093d530$a1bb7f90$@net>
References: <000c01cb7f5c$9a7dde10$cf799a30$@net>
	<AANLkTimqQgsaZO0XeKmFz3n1uvXeZStAYPG=K0tDsNi+@mail.gmail.com>
	<002e01cb7f7f$e093d530$a1bb7f90$@net>
Message-ID: <AANLkTinnndi8FyeZwycT23Ppq=-+Oed0dR-QRX-x+FQJ@mail.gmail.com>

Right.

If so it would be a build bug on RHEL for 2.1.4. Looks like it won't
build without rst2man present (which should't be needed because the
man pages are pregenerated in the tar-ball). You didn't do a clean
after unpacking or something?

There are packages availble for RHEL - see varnish-cache.org - but
there are alleged problems with the signatures, I think.

Per.

On Mon, Nov 8, 2010 at 9:02 PM, Roland Rebstock <racemd at verizon.net> wrote:
> Is that a RPM for RHEL 5? ?I do not see it on rhn.com. for the 2nd part
> after the configure, are you saying I don?t have to run "make"? ?I never ran
> make dist, just "make".
>
>
>
> -----Original Message-----
> From: Per Buer [mailto:perbu at varnish-software.com]
> Sent: Monday, November 08, 2010 12:15 PM
> To: Roland Rebstock
> Cc: varnish-misc at varnish-cache.org
> Subject: Re: Attempt to make Varnish on RHEL5 64bit error during make you
> need rst2man installed to make dist
>
> Hi Roland.
>
> Have you tried to install rst2man?
>
> Please note - you normally don't need to do "make dist" - the man
> pages are already generated in the tar-ball you've downloaded.
>
> Per.
>
> On Mon, Nov 8, 2010 at 4:50 PM, Roland Rebstock <racemd at verizon.net> wrote:
>> All, can someone tell me how to get around this error during make
>>
>>
>>
>> Making all in bin
>>
>> make[2]: Entering directory `/tmp/2.1/varnish-cache/bin'
>>
>> Making all in varnishadm
>>
>> make[3]: Entering directory `/tmp/2.1/varnish-cache/bin/varnishadm'
>>
>> ========================================
>>
>> You need rst2man installed to make dist
>>
>> ========================================
>>
>> make[3]: *** [varnishadm.1] Error 1
>>
>> make[3]: Leaving directory `/tmp/2.1/varnish-cache/bin/varnishadm'
>>
>> make[2]: *** [all-recursive] Error 1
>>
>> make[2]: Leaving directory `/tmp/2.1/varnish-cache/bin'
>>
>> make[1]: *** [all-recursive] Error 1
>>
>> make[1]: Leaving directory `/tmp/2.1/varnish-cache'
>>
>> make: *** [all] Error 2
>>
>>
>>
>>
>>
>>
>>
>> [root at new-host varnish-cache]# ./configure
> --bindir=/export/home/varnish/bin
>> --sbindir=/export/home/varnish/sbin
>> --libexecdir=/export/home/varnish/libexec
>> --datadir=/export/home/varnish/share --sysconfdir=/export/home/varnish/etc
>> --sharedstatedir=/export/home/varnish/com
> --libdir=/export/home/varnish/lib
>> -localstatedir=/export/home/varnish/var
>> --includedir=/export/home/varnish/include
>> --infodir=/export/home/varnish/info --prefix=/export/home/vanrish
>>
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>>
>
>
>
> --
> Per Buer,? Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
>
>



-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From alan.g.dixon at gmail.com  Tue Nov  9 18:05:50 2010
From: alan.g.dixon at gmail.com (Alan Dixon)
Date: Tue, 9 Nov 2010 13:05:50 -0500
Subject: 503 due to content-length error? microwave issue?
Message-ID: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>

I have nicely functioning varnish instance in front of a
drupal/civicrm website, running on debian/lenny (varnish-2.1.3 SVN ).

I have one user who is unable to POST (though maybe by AJAX . not sure
...?). When she tries, she gets the varnish 503, which doesn't help
too much.

When I look at the apache logs, I see that the response to her POST is
a 200 with a reasonable amount of content, whereas I get a 302,
correctly redirecting me after a successful post.  So it might be that
she's getting some kind of form invalid type of response, but varnish
isn't displaying it for some reason?

But here's another clue: she's using a microwave tower for her ISP.
I'm thinking that maybe this is doing something funny with the
packets, and so the content-length of the submission or reply from
apache is wrong and apache is more forgiving than varnish. Does that
make sense?

The varnish log shows identical behaviour with my sucessful post up
until the end of the TxHeaders to the apache backend. At that point,
she gets

   16 FetchError   c backend write error: 11
   17 BackendClose b default

whereas I get the answer back to varnish who then sends it on to me
properly. In both cases, varnish is correctly pass'ing the request to
the apache backend, and the headers look almost identical (we're both
running chrome - she's on windows and I'm on linux). Her content
length is almost the same, with the same content-type/boundary.

Any clues?


-- 
Alan Dixon, Web Developer

Drupal and CiviCRM for Canadian nonprofits.
http://blackflysolutions.ca/



From mloftis at wgops.com  Tue Nov  9 18:12:51 2010
From: mloftis at wgops.com (Michael Loftis)
Date: Tue, 9 Nov 2010 11:12:51 -0700
Subject: 503 due to content-length error? microwave issue?
In-Reply-To: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>
References: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>
Message-ID: <AANLkTikGTxg6fVorJChJM9dwMx2ABV8EwDKUyT7_Uu-X@mail.gmail.com>

On Tue, Nov 9, 2010 at 11:05 AM, Alan Dixon <alan.g.dixon at gmail.com> wrote:
> whereas I get the answer back to varnish who then sends it on to me
> properly. In both cases, varnish is correctly pass'ing the request to
> the apache backend, and the headers look almost identical (we're both
> running chrome - she's on windows and I'm on linux). Her content
> length is almost the same, with the same content-type/boundary.
>
> Any clues?


Where I've personally seen this is with lighttpd+php-fcgi where the
FastCGI binary was actually crashing (in my case we're hitting a core
dump bug in PHP) after setting content headers, and varnish received
less, or more, content than indicated by Content-Length.  Varnish
doesn't handle misbehaving backends very well in my experience.  If
(for whatever reason) your PHP app is crashing, or sending incorrect
content-length headers varnish may interpret it this way.  May or may
be the ISP mangling HTTP traffic.



From bre at pagekite.net  Tue Nov  9 19:22:58 2010
From: bre at pagekite.net (=?ISO-8859-1?Q?Bjarni_R=FAnar_Einarsson?=)
Date: Tue, 9 Nov 2010 19:22:58 +0000
Subject: Is Varnish useless with slow/remote back-ends & large files?
Message-ID: <AANLkTi=62CaPU74rNxRNbf+K7kxO8QzbQU5JOhcS5w94@mail.gmail.com>

Hi!

I am attempting to use Varnish in a somewhat unusual fashion, and before I
give up and drop it for something else, I figured I'd describe my problem
here and see if there's something I've missed.

What I am doing is implementing a service (http://pagekite.net/) to provide
a "remote HTTP front-end" for people who want to run webservers off a
personal computing device (possibly even a mobile device). I've got a nice
proxy/tunnel solution up and running, and want to add some caching in the
front-end to lighten the load on the back-end server and more importantly
the tunnel to the back-end, which may be quite bandwidth constrained.

At the moment it seems Varnish is effectively useless for this, because it
can't both cache and stream at the same time - it does one or the other,
right?

This means when a client downloads a large file, things just hang while the
whole thing gets transferred (over a slow link) to Varnish, likely even
timing out. I can't automatically stream it anyway, as the only trick I've
seen to switch to "pipe" mode for large content is to "restart" based on the
content-length header, which means the object will be sent twice - doubling
the load on exactly the link I'm trying to spare. The closest I can get is
to automatically switch to "pipe" for certain paths based on URL regular
expressions, which avoids the sloth and timeouts but provides no caching
benefits and will still do the wrong thing for some large files.

Varnish is quite an awesome piece of software, but I'm getting the feeling
that it just wasn't designed for my (admittedly unusual) environment. I'd
love to be proven wrong!

Thanks... :-)

-- 
Bjarni R. Einarsson
The Beanstalks Project ehf.

Making personal web-pages fly: http://pagekite.net/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101109/cb41b973/attachment.html>

From lars at oddbit.com  Tue Nov  9 21:23:31 2010
From: lars at oddbit.com (Lars Kellogg-Stedman)
Date: Tue, 9 Nov 2010 16:23:31 -0500
Subject: Yet another Ganglia module for Varnish
Message-ID: <AANLkTimKHVLWF4fhKBb3C1hh36fZRz28ATGeuGn+CPjS@mail.gmail.com>

I was looking for a metrics module for gmond that used the Python
module interface, rather than calling gmetric (because the module
interface can provide some additional metadata about metrics), and
didn't find anything...so I wrote my own.  It's available here, if
anyone finds it useful:

  https://github.com/larsks/ganglia-plugin-varnish

The good:

- It will generate a configuration for you automatically.
- It tries to identify the metric types (count vs. rate) automatically

The bad:

- Not much in the way of error checking right now.
- Only used in a very small environment.



From scaunter at topscms.com  Wed Nov 10 14:48:17 2010
From: scaunter at topscms.com (Caunter, Stefan)
Date: Wed, 10 Nov 2010 09:48:17 -0500
Subject: Is Varnish useless with slow/remote back-ends & large files?
In-Reply-To: <AANLkTi=62CaPU74rNxRNbf+K7kxO8QzbQU5JOhcS5w94@mail.gmail.com>
References: <AANLkTi=62CaPU74rNxRNbf+K7kxO8QzbQU5JOhcS5w94@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902C838FF2@TMG-EVS02.torstar.net>

Sorry for the top post (outlook at work). Sounds like you need to prime the cache with a script so your clients don't get "first request" syndrome. 

 

Stefan Caunter :: Senior Systems Administrator :: TOPS

e: scaunter at topscms.com  ::  m: (416) 561-4871

www.thestar.com www.topscms.com

 

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Bjarni R?nar Einarsson
Sent: November-09-10 2:23 PM
To: varnish-misc at varnish-cache.org
Subject: Is Varnish useless with slow/remote back-ends & large files?

 

Hi!

I am attempting to use Varnish in a somewhat unusual fashion, and before I give up and drop it for something else, I figured I'd describe my problem here and see if there's something I've missed.

What I am doing is implementing a service (http://pagekite.net/) to provide a "remote HTTP front-end" for people who want to run webservers off a personal computing device (possibly even a mobile device). I've got a nice proxy/tunnel solution up and running, and want to add some caching in the front-end to lighten the load on the back-end server and more importantly the tunnel to the back-end, which may be quite bandwidth constrained.

At the moment it seems Varnish is effectively useless for this, because it can't both cache and stream at the same time - it does one or the other, right?

This means when a client downloads a large file, things just hang while the whole thing gets transferred (over a slow link) to Varnish, likely even timing out. I can't automatically stream it anyway, as the only trick I've seen to switch to "pipe" mode for large content is to "restart" based on the content-length header, which means the object will be sent twice - doubling the load on exactly the link I'm trying to spare. The closest I can get is to automatically switch to "pipe" for certain paths based on URL regular expressions, which avoids the sloth and timeouts but provides no caching benefits and will still do the wrong thing for some large files.

Varnish is quite an awesome piece of software, but I'm getting the feeling that it just wasn't designed for my (admittedly unusual) environment. I'd love to be proven wrong!

Thanks... :-)

-- 
Bjarni R. Einarsson
The Beanstalks Project ehf.

Making personal web-pages fly: http://pagekite.net/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101110/2b4c5d4f/attachment.html>

From alan.g.dixon at gmail.com  Wed Nov 10 14:55:43 2010
From: alan.g.dixon at gmail.com (Alan Dixon)
Date: Wed, 10 Nov 2010 09:55:43 -0500
Subject: 503 due to content-length error? microwave issue?
In-Reply-To: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>
References: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>
Message-ID: <AANLkTinwf0sCa6kJMwxWTGRKJ8AoSDEicMjDSi=fDAmY@mail.gmail.com>

Okay, here's the answer: the problem was solved by the user's ISP:
they just lowered the MTU.

Conclusion: adding varnish can create subtle network problems [not
unique to varnish - also true of any front end proxy or ssl].

Not varnish's fault or problem, but good to know, and would be even
better if it were easier to debug.

 - Alan

On Tue, Nov 9, 2010 at 1:05 PM, Alan Dixon <alan.g.dixon at gmail.com> wrote:
> I have nicely functioning varnish instance in front of a
> drupal/civicrm website, running on debian/lenny (varnish-2.1.3 SVN ).
>
> I have one user who is unable to POST (though maybe by AJAX . not sure
> ...?). When she tries, she gets the varnish 503, which doesn't help
> too much.
>
> When I look at the apache logs, I see that the response to her POST is
> a 200 with a reasonable amount of content, whereas I get a 302,
> correctly redirecting me after a successful post. ?So it might be that
> she's getting some kind of form invalid type of response, but varnish
> isn't displaying it for some reason?
>
> But here's another clue: she's using a microwave tower for her ISP.
> I'm thinking that maybe this is doing something funny with the
> packets, and so the content-length of the submission or reply from
> apache is wrong and apache is more forgiving than varnish. Does that
> make sense?
>
> The varnish log shows identical behaviour with my sucessful post up
> until the end of the TxHeaders to the apache backend. At that point,
> she gets
>
> ? 16 FetchError ? c backend write error: 11
> ? 17 BackendClose b default
>
> whereas I get the answer back to varnish who then sends it on to me
> properly. In both cases, varnish is correctly pass'ing the request to
> the apache backend, and the headers look almost identical (we're both
> running chrome - she's on windows and I'm on linux). Her content
> length is almost the same, with the same content-type/boundary.
>
> Any clues?
>
>
> --
> Alan Dixon, Web Developer
>
> Drupal and CiviCRM for Canadian nonprofits.
> http://blackflysolutions.ca/
>



-- 
Alan Dixon, Web Developer

Drupal and CiviCRM for Canadian nonprofits.
http://blackflysolutions.ca/



From scaunter at topscms.com  Wed Nov 10 15:00:55 2010
From: scaunter at topscms.com (Caunter, Stefan)
Date: Wed, 10 Nov 2010 10:00:55 -0500
Subject: 503 due to content-length error? microwave issue?
In-Reply-To: <AANLkTinwf0sCa6kJMwxWTGRKJ8AoSDEicMjDSi=fDAmY@mail.gmail.com>
References: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>
	<AANLkTinwf0sCa6kJMwxWTGRKJ8AoSDEicMjDSi=fDAmY@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902C838FFF@TMG-EVS02.torstar.net>

Irrelevant. Varnish (or any front end proxy) knows nothing about layer
2, 3 or 4 in the network stack. It's an application, handling requests
and responses at layer 7 using the HTTP protocol.

Stefan Caunter :: Senior Systems Administrator :: TOPS
e: scaunter at topscms.com  ::  m: (416) 561-4871
www.thestar.com www.topscms.com


-----Original Message-----
From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Alan Dixon
Sent: November-10-10 9:56 AM
To: varnish-misc at varnish-cache.org
Subject: Re: 503 due to content-length error? microwave issue?

Okay, here's the answer: the problem was solved by the user's ISP:
they just lowered the MTU.

Conclusion: adding varnish can create subtle network problems [not
unique to varnish - also true of any front end proxy or ssl].

Not varnish's fault or problem, but good to know, and would be even
better if it were easier to debug.

 - Alan



From alan.g.dixon at gmail.com  Wed Nov 10 15:07:47 2010
From: alan.g.dixon at gmail.com (Alan Dixon)
Date: Wed, 10 Nov 2010 10:07:47 -0500
Subject: 503 due to content-length error? microwave issue?
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902C838FFF@TMG-EVS02.torstar.net>
References: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>
	<AANLkTinwf0sCa6kJMwxWTGRKJ8AoSDEicMjDSi=fDAmY@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902C838FFF@TMG-EVS02.torstar.net>
Message-ID: <AANLkTikMPJN3acEoo7=1gDxfFjKHAWRVJW-+Ven7TxBS@mail.gmail.com>

Theoretically perhaps, but:

1. Adding varnish triggered the problem.
2. Changing the MTU solved the problem.
3. The system administrators of the ISP had seen a similar
problem/solution in the past.

My limited understanding is that varnish's role in the problem was
just being more demanding ("standards sensitive"), whereas Apache was
able to handle the bad data better.

  - Alan

On Wed, Nov 10, 2010 at 10:00 AM, Caunter, Stefan <scaunter at topscms.com> wrote:
> Irrelevant. Varnish (or any front end proxy) knows nothing about layer
> 2, 3 or 4 in the network stack. It's an application, handling requests
> and responses at layer 7 using the HTTP protocol.
>
> Stefan Caunter :: Senior Systems Administrator :: TOPS
> e: scaunter at topscms.com ?:: ?m: (416) 561-4871
> www.thestar.com www.topscms.com
>
>
> -----Original Message-----
> From: varnish-misc-bounces at varnish-cache.org
> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Alan Dixon
> Sent: November-10-10 9:56 AM
> To: varnish-misc at varnish-cache.org
> Subject: Re: 503 due to content-length error? microwave issue?
>
> Okay, here's the answer: the problem was solved by the user's ISP:
> they just lowered the MTU.
>
> Conclusion: adding varnish can create subtle network problems [not
> unique to varnish - also true of any front end proxy or ssl].
>
> Not varnish's fault or problem, but good to know, and would be even
> better if it were easier to debug.
>
> ?- Alan
>



-- 
Alan Dixon, Web Developer

Drupal and CiviCRM for Canadian nonprofits.
http://blackflysolutions.ca/



From vburshteyn at broadway.com  Wed Nov 10 16:56:43 2010
From: vburshteyn at broadway.com (Vitaly Burshteyn)
Date: Wed, 10 Nov 2010 11:56:43 -0500
Subject: varnish regex
Message-ID: <0F69574F9901D4459C6B75C9FF64FBC40518FE77@mxfl01.hollywoodmedia.corp>

Hi folks,

 

I have a set of url's that I need to make sure that varnish does cache.
The url patern looks like this:

 

Siteurl/r/anynumber

 

if (req.url ~ "/r/*/") {

        return(pass);

    }

 

 

Or 

 

 

if (req.url ~ "^/r/\d+/$") {

        return(pass);

    }

 

 

Sorry about this question, but I am not sure how varnish processes regex
values.

 

Vitaly Burshteyn

Senior Network Engineer

Broadway.com, Theatre Direct International

729  7th Avenue

New York, New York 10019

Phone: 212.817.9117

Cell# 917-701-5732

 


____________________________________
The information contained in this transmission may contain privileged 
and confidential information.  It is intended only for the use of the 
person(s) named above. If you are not the intended recipient,  you are 
hereby notified that any review, dissemination, distribution or 
duplication of this communication is strictly prohibited. If you are 
not the intended recipient, please contact the sender by reply email 
and destroy all copies of the original message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101110/5f1f4d25/attachment.html>

From mloftis at wgops.com  Wed Nov 10 17:20:57 2010
From: mloftis at wgops.com (Michael Loftis)
Date: Wed, 10 Nov 2010 10:20:57 -0700
Subject: varnish regex
In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC40518FE77@mxfl01.hollywoodmedia.corp>
References: <0F69574F9901D4459C6B75C9FF64FBC40518FE77@mxfl01.hollywoodmedia.corp>
Message-ID: <AANLkTimXdxyL=Kg-1yFhFACw1a7x9WUD6XnWH_p2jTU=@mail.gmail.com>

On Wed, Nov 10, 2010 at 9:56 AM, Vitaly Burshteyn
<vburshteyn at broadway.com> wrote:
> Hi folks,
>
>
>
> I have a set of url?s that I need to make sure that varnish does cache.? The
> url patern looks like this:
>
>
>
> Siteurl/r/anynumber
>
>
>
> if (req.url ~ "/r/*/") {
>
> ??????? return(pass);
>
> ??? }
>
>
>
>
>
> Or
>
>
>
>
>
> if (req.url ~ "^/r/\d+/$") {
>
> ??????? return(pass);
>
> ??? }
>
>
>
>
>
> Sorry about this question, but I am not sure how varnish processes regex
> values.


First, return (lookup) not pass.  Lookup pushes it into the cache
hash/lookup routine for a fetch/hit.  Secondly if there's no trailing
/ in your requested URLs, then make sure it doesn't appear in your
pattern either.  The regular expression flavor is PCRE/libpcre.

Even if you return lookup you're not guarenteed to cache.  POSTs,
things with cookies don't get cached.
http://www.varnish-cache.org/docs/2.1/reference/vcl.html for VCL
reference and the default VCL.



From vburshteyn at broadway.com  Wed Nov 10 17:23:23 2010
From: vburshteyn at broadway.com (Vitaly Burshteyn)
Date: Wed, 10 Nov 2010 12:23:23 -0500
Subject: varnish regex
In-Reply-To: <AANLkTimXdxyL=Kg-1yFhFACw1a7x9WUD6XnWH_p2jTU=@mail.gmail.com>
References: <0F69574F9901D4459C6B75C9FF64FBC40518FE77@mxfl01.hollywoodmedia.corp>
	<AANLkTimXdxyL=Kg-1yFhFACw1a7x9WUD6XnWH_p2jTU=@mail.gmail.com>
Message-ID: <0F69574F9901D4459C6B75C9FF64FBC40518FEB2@mxfl01.hollywoodmedia.corp>

Sorry 

I somehow forgot the NOT part.

I need it not to cache that url.

Vitaly Burshteyn
Senior Network Engineer
Broadway.com, Theatre Direct International
729? 7th Avenue
New York, New York 10019
Phone: 212.817.9117
Cell# 917-701-5732


-----Original Message-----
From: Michael Loftis [mailto:mloftis at wgops.com] 
Sent: Wednesday, November 10, 2010 12:21 PM
To: Vitaly Burshteyn
Cc: varnish-misc
Subject: Re: varnish regex

On Wed, Nov 10, 2010 at 9:56 AM, Vitaly Burshteyn
<vburshteyn at broadway.com> wrote:
> Hi folks,
>
>
>
> I have a set of url's that I need to make sure that varnish does cache.? The
> url patern looks like this:
>
>
>
> Siteurl/r/anynumber
>
>
>
> if (req.url ~ "/r/*/") {
>
> ??????? return(pass);
>
> ??? }
>
>
>
>
>
> Or
>
>
>
>
>
> if (req.url ~ "^/r/\d+/$") {
>
> ??????? return(pass);
>
> ??? }
>
>
>
>
>
> Sorry about this question, but I am not sure how varnish processes regex
> values.


First, return (lookup) not pass.  Lookup pushes it into the cache
hash/lookup routine for a fetch/hit.  Secondly if there's no trailing
/ in your requested URLs, then make sure it doesn't appear in your
pattern either.  The regular expression flavor is PCRE/libpcre.

Even if you return lookup you're not guarenteed to cache.  POSTs,
things with cookies don't get cached.
http://www.varnish-cache.org/docs/2.1/reference/vcl.html for VCL
reference and the default VCL.

____________________________________
The information contained in this transmission may contain privileged 
and confidential information.  It is intended only for the use of the 
person(s) named above. If you are not the intended recipient,  you are 
hereby notified that any review, dissemination, distribution or 
duplication of this communication is strictly prohibited. If you are 
not the intended recipient, please contact the sender by reply email 
and destroy all copies of the original message.



From mloftis at wgops.com  Wed Nov 10 17:31:22 2010
From: mloftis at wgops.com (Michael Loftis)
Date: Wed, 10 Nov 2010 10:31:22 -0700
Subject: varnish regex
In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC40518FEB2@mxfl01.hollywoodmedia.corp>
References: <0F69574F9901D4459C6B75C9FF64FBC40518FE77@mxfl01.hollywoodmedia.corp>
	<AANLkTimXdxyL=Kg-1yFhFACw1a7x9WUD6XnWH_p2jTU=@mail.gmail.com>
	<0F69574F9901D4459C6B75C9FF64FBC40518FEB2@mxfl01.hollywoodmedia.corp>
Message-ID: <AANLkTi=kBj-BxQy7ttGLrbb53_UW2AFPYSN_M3Ujpo7e@mail.gmail.com>

On Wed, Nov 10, 2010 at 10:23 AM, Vitaly Burshteyn
<vburshteyn at broadway.com> wrote:
> Sorry
>
> I somehow forgot the NOT part.
>
> I need it not to cache that url.

Then yes you can return pass in any number of places depending on your
needs, I usually do it in vcl_recv.
http://www.varnish-cache.org/trac/wiki/VCLExampleDefault might be
helpful to understand the flow/what happens when you return X from sub
Y.

>
> Vitaly Burshteyn
> Senior Network Engineer
> Broadway.com, Theatre Direct International
> 729? 7th Avenue
> New York, New York 10019
> Phone: 212.817.9117
> Cell# 917-701-5732
>
>
> -----Original Message-----
> From: Michael Loftis [mailto:mloftis at wgops.com]
> Sent: Wednesday, November 10, 2010 12:21 PM
> To: Vitaly Burshteyn
> Cc: varnish-misc
> Subject: Re: varnish regex
>
> On Wed, Nov 10, 2010 at 9:56 AM, Vitaly Burshteyn
> <vburshteyn at broadway.com> wrote:
>> Hi folks,
>>
>>
>>
>> I have a set of url's that I need to make sure that varnish does cache.? The
>> url patern looks like this:
>>
>>
>>
>> Siteurl/r/anynumber
>>
>>
>>
>> if (req.url ~ "/r/*/") {
>>
>> ??????? return(pass);
>>
>> ??? }
>>
>>
>>
>>
>>
>> Or
>>
>>
>>
>>
>>
>> if (req.url ~ "^/r/\d+/$") {
>>
>> ??????? return(pass);
>>
>> ??? }
>>
>>
>>
>>
>>
>> Sorry about this question, but I am not sure how varnish processes regex
>> values.
>
>
> First, return (lookup) not pass. ?Lookup pushes it into the cache
> hash/lookup routine for a fetch/hit. ?Secondly if there's no trailing
> / in your requested URLs, then make sure it doesn't appear in your
> pattern either. ?The regular expression flavor is PCRE/libpcre.
>
> Even if you return lookup you're not guarenteed to cache. ?POSTs,
> things with cookies don't get cached.
> http://www.varnish-cache.org/docs/2.1/reference/vcl.html for VCL
> reference and the default VCL.
>
> ____________________________________
> The information contained in this transmission may contain privileged
> and confidential information. ?It is intended only for the use of the
> person(s) named above. If you are not the intended recipient, ?you are
> hereby notified that any review, dissemination, distribution or
> duplication of this communication is strictly prohibited. If you are
> not the intended recipient, please contact the sender by reply email
> and destroy all copies of the original message.
>



From scaunter at topscms.com  Wed Nov 10 17:54:53 2010
From: scaunter at topscms.com (Caunter, Stefan)
Date: Wed, 10 Nov 2010 12:54:53 -0500
Subject: 503 due to content-length error? microwave issue?
In-Reply-To: <AANLkTikMPJN3acEoo7=1gDxfFjKHAWRVJW-+Ven7TxBS@mail.gmail.com>
References: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com><AANLkTinwf0sCa6kJMwxWTGRKJ8AoSDEicMjDSi=fDAmY@mail.gmail.com><7F0AA702B8A85A4A967C4C8EBAD6902C838FFF@TMG-EVS02.torstar.net>
	<AANLkTikMPJN3acEoo7=1gDxfFjKHAWRVJW-+Ven7TxBS@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902C8390B0@TMG-EVS02.torstar.net>

There's nothing theoretical. Varnish talks HTTP. HTTP knows nothing, and needs to know nothing, about TCP/IP or MTU; it is a client and server protocol with various implementations of the relevant RFCs.

You cannot troubleshoot network or transport layer issues at the application layer. 

A more accurate conclusion would be that adding Varnish exposed existing TCP/IP (network/transport layer) issues. They were there before adding Varnish, and would still have been there had Varnish been removed. Changing the MTU corrected the issue.

Stefan Caunter :: Senior Systems Administrator :: TOPS
e: scaunter at topscms.com  ::  m: (416) 561-4871
www.thestar.com www.topscms.com


-----Original Message-----
From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Alan Dixon
Sent: November-10-10 10:08 AM
To: varnish-misc at varnish-cache.org
Subject: Re: 503 due to content-length error? microwave issue?

Theoretically perhaps, but:

1. Adding varnish triggered the problem.
2. Changing the MTU solved the problem.
3. The system administrators of the ISP had seen a similar
problem/solution in the past.

My limited understanding is that varnish's role in the problem was
just being more demanding ("standards sensitive"), whereas Apache was
able to handle the bad data better.

  - Alan

On Wed, Nov 10, 2010 at 10:00 AM, Caunter, Stefan <scaunter at topscms.com> wrote:
> Irrelevant. Varnish (or any front end proxy) knows nothing about layer
> 2, 3 or 4 in the network stack. It's an application, handling requests
> and responses at layer 7 using the HTTP protocol.
>
> Stefan Caunter :: Senior Systems Administrator :: TOPS
> e: scaunter at topscms.com ?:: ?m: (416) 561-4871
> www.thestar.com www.topscms.com
>
>
> -----Original Message-----
> From: varnish-misc-bounces at varnish-cache.org
> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Alan Dixon
> Sent: November-10-10 9:56 AM
> To: varnish-misc at varnish-cache.org
> Subject: Re: 503 due to content-length error? microwave issue?
>
> Okay, here's the answer: the problem was solved by the user's ISP:
> they just lowered the MTU.
>
> Conclusion: adding varnish can create subtle network problems [not
> unique to varnish - also true of any front end proxy or ssl].
>
> Not varnish's fault or problem, but good to know, and would be even
> better if it were easier to debug.
>
> ?- Alan
>



-- 
Alan Dixon, Web Developer

Drupal and CiviCRM for Canadian nonprofits.
http://blackflysolutions.ca/

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://lists.varnish-cache.org/mailman/listinfo/varnish-misc



From alan.g.dixon at gmail.com  Wed Nov 10 20:32:58 2010
From: alan.g.dixon at gmail.com (Alan Dixon)
Date: Wed, 10 Nov 2010 15:32:58 -0500
Subject: 503 due to content-length error? microwave issue?
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902C8390B0@TMG-EVS02.torstar.net>
References: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>
	<AANLkTinwf0sCa6kJMwxWTGRKJ8AoSDEicMjDSi=fDAmY@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902C838FFF@TMG-EVS02.torstar.net>
	<AANLkTikMPJN3acEoo7=1gDxfFjKHAWRVJW-+Ven7TxBS@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902C8390B0@TMG-EVS02.torstar.net>
Message-ID: <AANLkTimZ4=QphNgU=vQt-c_KO4SCyJwO0+AduT4zXO6A@mail.gmail.com>

On Wed, Nov 10, 2010 at 12:54 PM, Caunter, Stefan <scaunter at topscms.com> wrote:
> There's nothing theoretical. Varnish talks HTTP. HTTP knows nothing, and needs to know nothing, about TCP/IP or MTU; it is a client and server protocol with various implementations of the relevant RFCs.
>
> You cannot troubleshoot network or transport layer issues at the application layer.
>
> A more accurate conclusion would be that adding Varnish exposed existing TCP/IP (network/transport layer) issues. They were there before adding Varnish, and would still have been there had Varnish been removed. Changing the MTU corrected the issue.

Yes, expose rather than create is more accurate.

But at the point of discovering the symptom, it's not clear which it
is so I'm trying to provide helpful information for others with a
similar problem.

>From a symptom point of view, the problems were invisible to the user
before varnish, so her experience was that varnish 'caused' the
problem.



-- 
Alan Dixon, Web Developer

Drupal and CiviCRM for Canadian nonprofits.
http://blackflysolutions.ca/



From varnish at mm.quex.org  Thu Nov 11 03:05:36 2010
From: varnish at mm.quex.org (Michael Alger)
Date: Thu, 11 Nov 2010 11:05:36 +0800
Subject: varnish regex
In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC40518FE77@mxfl01.hollywoodmedia.corp>
References: <0F69574F9901D4459C6B75C9FF64FBC40518FE77@mxfl01.hollywoodmedia.corp>
Message-ID: <20101111030536.GA29639@grum.quex.org>

On Wed, Nov 10, 2010 at 11:56:43AM -0500, Vitaly Burshteyn wrote:
> 
> I have a set of url's that I need to make sure that varnish does
> [not] cache.  The url patern looks like this:
> 
> Siteurl/r/anynumber
> 
> if (req.url ~ "/r/*/") {
> 
>         return(pass);
> 
>     }

That looks more like a glob/wildcard match than a regex, and won't
do what you want.

> Or 
> 
> if (req.url ~ "^/r/\d+/$") {
> 
>         return(pass);
> 
>     }

The above is what you want, assuming nothing else can come after the
"anynumber", but there's _always_ a slash after it. If the slash
actually isn't _ever_ present, then just use

  if (req.url ~ "^/r/\d+$") {

If the slash may or may not be present, then

  if (req.url ~ "^/r/\d+/?$") {

may be better. If there's also the possibility of additional things
after the number, such as a query string, and you want to avoid caching
those as well, then I'd just drop the $ at the end:

  if (req.url ~ "^/r/\d+") {

> Sorry about this question, but I am not sure how varnish processes
> regex values.

Varnish switched to pcre in 2.1.x I think; prior to that, regular
expressions were case-insensitive by default. So you should check
what version you're running if that matters to you. Current releases
are case-sensitive by default, so if your backend isn't then you can
use the (?i) syntax to make the expression case-insensitive:

  if (req.url ~ "(?i)^/r/\d+/?$") {




From Glen.Kelly at fire.tas.gov.au  Thu Nov 11 23:52:14 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Fri, 12 Nov 2010 10:52:14 +1100
Subject: Vcl command restart and multiple backends
Message-ID: <674B23B1CEE7B140B4868996CE484506109CB20EA2@hoexchange1.fire.tas.gov.au>

How does varnish handle request to multiple backend when the vcl "restart" command is run.

Does it use the same backend as the initial request or does it use another backend in accordance to the configured director.

i.e.

two backends b1 and b2, restart command is in vcl_error

request for object from b1 fails, vcl_error runs and the restart command is executed, on the next retry what backend will varnish use, b1 again or what will it act in accordance with the director (if the director was round robin would it use b2)?

I know you can explicitly script this in the vcl like in the example provided at http://northernmost.org/blog/some-trickery-or-resilience-with-varnish/ but I currently have our backends using the round robin director and I don't want to explicitly configure specific backends in a preference order.

Regards Glen

[cid:tasfire_wakeup_small7eb7.jpg]<http://www.fire.tas.gov.au/Show?pageId=colDayLightSavings>
P Do you need to print this? Consider the environment, prevent paper waste.
CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101112/13dd02c1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tasfire_wakeup_small7eb7.jpg
Type: image/jpeg
Size: 27459 bytes
Desc: tasfire_wakeup_small7eb7.jpg
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101112/13dd02c1/attachment.jpg>

From perbu at varnish-software.com  Fri Nov 12 09:48:34 2010
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 12 Nov 2010 10:48:34 +0100
Subject: Vcl command restart and multiple backends
In-Reply-To: <674B23B1CEE7B140B4868996CE484506109CB20EA2@hoexchange1.fire.tas.gov.au>
References: <674B23B1CEE7B140B4868996CE484506109CB20EA2@hoexchange1.fire.tas.gov.au>
Message-ID: <AANLkTi=ZuKjGGZFqXw9TT20pGq3z_GSwN-mEDHXGb-sa@mail.gmail.com>

On Fri, Nov 12, 2010 at 12:52 AM, Glen Kelly <Glen.Kelly at fire.tas.gov.au>wrote:

> How does varnish handle request to multiple backend when the vcl ?restart?
> command is run.
>
>
>
> Does it use the same backend as the initial request or does it use another
> backend in accordance to the configured director.
>

You should look at "Saint Mode" -
http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html#saint-mode

When enabled it will blacklist failing objects from their backends.



-- 
Per Buer,  Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer

Varnish user? Take our survey:
http://www.varnish-cache.org/user-survey-november-2010
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101112/2c509b94/attachment.html>

From simon at darkmere.gen.nz  Fri Nov 12 23:49:34 2010
From: simon at darkmere.gen.nz (Simon Lyall)
Date: Sat, 13 Nov 2010 12:49:34 +1300 (NZDT)
Subject: Offtopic Bikeshedding: deflate compression
Message-ID: <alpine.DEB.2.00.1011131218170.5692@green.darkmere.gen.nz>

This bugs me when I see the sample code for Compression here 
http://www.varnish-cache.org/trac/wiki/FAQ/Compression

The bit that bugs me is that "deflate" compression doesn't work with IE 
since Microsoft implimented it differently from everybody else.

This doesn't actually affect 99.99% of http queries since they are really 
using gzip but if you happen to send a deflate compressed page to IE ( 
versions 6,7,8 or 9) that is generated by a Unix box then you will get an 
error (something like "connot connect to remote site") on IE.

This edge case sometimes affects me (not with varnish but with my current 
web cache) and I suspect others will get bitten sometimes so I was 
wondering if we could remove deflate from the example code so that people 
won't also be caught.

Scarily enough this is very lightly documented on the Internet but here 
are a some links:

http://stackoverflow.com/questions/883841/why-do-real-world-servers-prefer-gzip-over-deflate-encoding
http://www.vervestudios.co/projects/compression-tests/results
http://www.subbu.org/blog/2008/03/ie7-deflate-or-not

-- 
Simon Lyall  |  Very Busy  |  Web: http://www.darkmere.gen.nz/
"To stay awake all night adds a day to your life" - Stilgar | eMT.




From reuben at wikihow.com  Sat Nov 13 00:55:35 2010
From: reuben at wikihow.com (Reuben S.)
Date: Fri, 12 Nov 2010 16:55:35 -0800
Subject: Varnish panic message on purge / synthetic
Message-ID: <AANLkTimLyQnV0Mrj8Zf_5fOhaD53PcK7o+tjW_+z1Nea@mail.gmail.com>

I am getting an Assert error which I think is related to a synthetic {...}
call I make in the vcl on PURGE commands from the back-end.  Is this is a
known issue?  Should I be doing something differently?

Here's an abbreviated version of the vcl:

acl purge {
    "127.0.0.1";
    "192.168.[...]"/[...];
}

sub vcl_recv {
    set req.backend = dir;

    if (req.request == "PURGE") {
        # if IP is not in "purge" acl, don't allow
        if (!client.ip ~ purge) {
            error 405 "Not allowed";
        } else {
            purge("req.url ~ " req.url);
            synthetic {"Purged"};
            return (lookup);
        }
    }
}

I'm running Varnish 2.1.4, which I compiled myself.  This happened with
varnish 2.1.3 too.  I'm running Red Hat Enterprise Linux Server release 5.5
with 64-bit cpus.

Here's the error from the syslog:

Nov 12 22:49:15 squid1 varnishd[2433]: Child (13846) Panic message: Assert
error in VRT_synth_page(), cache_vrt.c line 989:   Condition((sp->obj) !=
NULL) not true. thread = (cache-worker) ident =
Linux,2.6.18-194.17.4.el5,x86_64,-smalloc,-sfile,-hcritbit,epoll Backtrace:
  0x423996: pan_ic+b6   0x42b153: VRT_synth_page+1e3   0x2ac3b2c1e31a:
_end+2ac3b25aad52   0x4289a3: VCL_recv_method+43   0x4143fa: CNT_Session+5ca
  0x425d58: wrk_do_cnt_sess+b8   0x42505e: wrk_thread_real+32e
0x32fa00673d: _end+32f9993175   0x32f90d3f6d: _end+32f8a609a5 sp =
0x2ac47b38e008 {   fd = 64, id = 64, xid = 1187807240,   client =
192.168.[...] 53762,   step = STP_RECV,   handling = deliver,   restarts =
0, esis = 0   ws = 0x2ac47b38e080 {      id = "sess",     {s,f,r,e} =
{0x2ac47b38ecd8,+184,(nil),+65536},   },   http[req] = {     ws =
0x2ac47b38e080[sess]       "PURGE",       "
http://www.wikihow.com/Document-Employee-Performance",       "HTTP/1.0",
  "Connection: Keep-Alive",   },   worker = 0x2ac6fe7bbe60 {     ws = 0x2

Thanks,

Reuben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101112/d87bf5b3/attachment.html>

From phk at phk.freebsd.dk  Sat Nov 13 01:12:40 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sat, 13 Nov 2010 01:12:40 +0000
Subject: Varnish panic message on purge / synthetic 
In-Reply-To: Your message of "Fri, 12 Nov 2010 16:55:35 PST."
	<AANLkTimLyQnV0Mrj8Zf_5fOhaD53PcK7o+tjW_+z1Nea@mail.gmail.com> 
Message-ID: <27954.1289610760@critter.freebsd.dk>

In message <AANLkTimLyQnV0Mrj8Zf_5fOhaD53PcK7o+tjW_+z1Nea at mail.gmail.com>, "Reu
ben S." writes:

>I am getting an Assert error which I think is related to a synthetic {...}
>call I make in the vcl on PURGE commands from the back-end.  Is this is a
>known issue?  Should I be doing something differently?

yes, at present synthetic{} can only be used in vcl_error{}.

You can do an error(755), and test for that status in vcl_error{ } and
do the synthetic there.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From fookill at gmail.com  Sat Nov 13 12:56:48 2010
From: fookill at gmail.com (Leonid Vasilyev)
Date: Sat, 13 Nov 2010 17:56:48 +0500
Subject: Question about implementation of logging to shm.
Message-ID: <AANLkTinLjzU8kdOneArbk419WAgmDgLZ5E4KKX-9De25@mail.gmail.com>

Hello,

Is there any information about implementation of logging to shared
memory available?

I've run varnishd with strace, but can't see any shmget/shmat syscalls
neither can i find any calls of shm_open().


Best regards
--
Leonid



From mloftis at wgops.com  Sat Nov 13 22:22:00 2010
From: mloftis at wgops.com (Michael Loftis)
Date: Sat, 13 Nov 2010 15:22:00 -0700
Subject: Question about implementation of logging to shm.
In-Reply-To: <AANLkTinLjzU8kdOneArbk419WAgmDgLZ5E4KKX-9De25@mail.gmail.com>
References: <AANLkTinLjzU8kdOneArbk419WAgmDgLZ5E4KKX-9De25@mail.gmail.com>
Message-ID: <AANLkTin8E43FADrn7VYFD5SfWD0z44Q3FhL9YsLmgXeu@mail.gmail.com>

I *think* varnish uses shared memory via mmap, not System V shm*.
I've never looked closely into it.

On Sat, Nov 13, 2010 at 5:56 AM, Leonid Vasilyev <fookill at gmail.com> wrote:
> Hello,
>
> Is there any information about implementation of logging to shared
> memory available?
>
> I've run varnishd with strace, but can't see any shmget/shmat syscalls
> neither can i find any calls of shm_open().
>
>
> Best regards
> --
> Leonid
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



From phk at phk.freebsd.dk  Sun Nov 14 01:57:44 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sun, 14 Nov 2010 01:57:44 +0000
Subject: Question about implementation of logging to shm. 
In-Reply-To: Your message of "Sat, 13 Nov 2010 17:56:48 +0500."
	<AANLkTinLjzU8kdOneArbk419WAgmDgLZ5E4KKX-9De25@mail.gmail.com> 
Message-ID: <34011.1289699864@critter.freebsd.dk>

In message <AANLkTinLjzU8kdOneArbk419WAgmDgLZ5E4KKX-9De25 at mail.gmail.com>, Leon
id Vasilyev writes:
>Hello,
>
>Is there any information about implementation of logging to shared
>memory available?
>
>I've run varnishd with strace, but can't see any shmget/shmat syscalls
>neither can i find any calls of shm_open().

We don't use SVID SHM, we simply mmap a file and write it there.

Look for "VSL"

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From cooltechemail at gmail.com  Sun Nov 14 03:18:45 2010
From: cooltechemail at gmail.com (Vincent)
Date: Sat, 13 Nov 2010 19:18:45 -0800
Subject: Grace Mode Questions
Message-ID: <AANLkTimcxdUpF7hO1=76wCJGng9pEZ99_mVrZwdMUgb1@mail.gmail.com>

Hi All,

I am using the grace mode and I am wondering how the old objects are stored
and used in varnish. For example, if I have the following setup:

sub vcl_recv {
  set req.grace = 30m;
}
sub vcl_fetch {
    set beresp.ttl =1m;
    set beresp.grace = 1h;
        if (beresp.status != 200 && beresp.status != 403 && beresp.status !=
404 && beresp.status != 301 && beresp.status != 302) {
            set beresp.saintmode = 30s;
            restart;
        }
}

My questions are:
1) Since all objects have a ttl of 1m, and the req.grace is 30m. Does this
mean varnish will keep all expired objects in the past 30 minutes, which
lead to 30 expired objects for each object. Or, Varnish only keeps the most
recent expired object?

2) in vcl_recv, can varnish check if the object is in saintmode so we can
set the grace longer? For example, can we do something like:
if(req.saintmode){
  set req.grace = 1h;
}
else{
  set req.grace = 30s;
}
(I tried the above code and it didn't work)



Thanks,

Vincent
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101113/95c68f95/attachment.html>

From cooltechemail at gmail.com  Sun Nov 14 06:37:31 2010
From: cooltechemail at gmail.com (Vincent)
Date: Sat, 13 Nov 2010 22:37:31 -0800
Subject: issues with purge via varnishadm ran by crontab
Message-ID: <AANLkTin=6WH2Z+4PuYBK7WVqzPp9p9zrgmgZy_XxbN+-@mail.gmail.com>

Hi All,

I am using 2.1.3 and I am having issue with purge via the varnishadm
command. Basically if I put the following command in crontab, the system
will run the command as defined in crontab but it seems that varnish will
ignore the command:

varnishadm  -T 127.0.0.1:8000 purge req.http.host == "www.example.com"

However, if I manually type the above command in the command line, there is
no problem. I've checked that www.example.com is not in purge.list when the
command is ran using crontab but it will be included in purge.list when I
manually type the command.

Is this a bug?

Thanks,

Vincent
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101113/d6d9ed50/attachment.html>

From justin at idontwatch.tv  Sun Nov 14 08:25:48 2010
From: justin at idontwatch.tv (Justin)
Date: Sun, 14 Nov 2010 01:25:48 -0700
Subject: www.varnish-cache.org/installation/freebsd error
Message-ID: <4CDF9D0C.7090905@idontwatch.tv>

Hey there,

I'd like to point out that on 
http://www.varnish-cache.org/installation/freebsd the line that reads:

|cd /usr/ports/varnish && make install clean

should read:

cd /usr/ports/www/varnish && make install clean

Trac didn't seem like the appropriate place to submit this to those in 
charge. If this mailing list isn't please let me know or forward it on 
to those in charge of these sorts of things.

Thanks!

-Justin

|
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101114/a8322e96/attachment.html>

From phk at phk.freebsd.dk  Sun Nov 14 12:17:49 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sun, 14 Nov 2010 12:17:49 +0000
Subject: Grace Mode Questions 
In-Reply-To: Your message of "Sat, 13 Nov 2010 19:18:45 PST."
	<AANLkTimcxdUpF7hO1=76wCJGng9pEZ99_mVrZwdMUgb1@mail.gmail.com> 
Message-ID: <36073.1289737069@critter.freebsd.dk>

In message <AANLkTimcxdUpF7hO1=76wCJGng9pEZ99_mVrZwdMUgb1 at mail.gmail.com>, Vinc
ent writes:

>I am using the grace mode and I am wondering how the old objects are stored
>and used in varnish. For example, if I have the following setup:

Until their TTL + their grace periode expires.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From thebog at gmail.com  Sun Nov 14 13:03:27 2010
From: thebog at gmail.com (thebog)
Date: Sun, 14 Nov 2010 14:03:27 +0100
Subject: issues with purge via varnishadm ran by crontab
In-Reply-To: <AANLkTin=6WH2Z+4PuYBK7WVqzPp9p9zrgmgZy_XxbN+-@mail.gmail.com>
References: <AANLkTin=6WH2Z+4PuYBK7WVqzPp9p9zrgmgZy_XxbN+-@mail.gmail.com>
Message-ID: <AANLkTimBvEmPP=R2XuvrXPKULvz2ViV-TaM6mgVx6r5r@mail.gmail.com>

Hi,

if you do not see it in the list. Then there is a big possibility that
the script never makes it into varnishadm. So I think it's safe to say
that the problem is in the script somehow, and not varnish.
Not that I am gonna debug your script, but take things into account
that cron is often run by /bin/sh which may have different env
variables than your normal shell (And therefore maybe a different
PATH).

Type /bin/sh and work from there.

YS
Anders Berg


On Sun, Nov 14, 2010 at 7:37 AM, Vincent <cooltechemail at gmail.com> wrote:
> Hi All,
> I am using 2.1.3 and I am having issue with purge via the?varnishadm
> command. Basically if I put the following command in crontab, the system
> will run the command as defined in crontab but it seems that varnish will
> ignore the command:
> varnishadm ?-T 127.0.0.1:8000 purge req.http.host == "www.example.com"
> However, if I manually type the above command in the command line, there is
> no problem. I've checked that www.example.com is not in purge.list when the
> command is ran using crontab but it will be included in purge.list when I
> manually type the command.
> Is this a bug?
> Thanks,
> Vincent
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



From cooltechemail at gmail.com  Sun Nov 14 15:08:58 2010
From: cooltechemail at gmail.com (Vincent Wells)
Date: Sun, 14 Nov 2010 07:08:58 -0800
Subject: Grace Mode Questions
In-Reply-To: <36073.1289737069@critter.freebsd.dk>
References: <AANLkTimcxdUpF7hO1=76wCJGng9pEZ99_mVrZwdMUgb1@mail.gmail.com>
	<36073.1289737069@critter.freebsd.dk>
Message-ID: <AANLkTinw8ZjDt2SsBvZPuC5O2bCBWPVpaXog7Du_joQ0@mail.gmail.com>

Hi Poul-Henning

Thank you for your reply. I know it is TTL+grace. The questions is if grace
is much longer than TTL, does it mean varnish will keep multiple expired
copies of the same objected instead of the most recent one? In case an old
object needs to be used (for example, saintmode), which expired copy will be
used? Will it be the most recent one?

I am asking this because I want to set grace longer so I can show the
end-user something in case of backend server error. However, I am worried
that:
1) a long grace will fill up the memory (will it?)
2) older objects will be used even if there are newer (but expired) objects
available. (how expired objects are used?)

Thanks,

Vincent

On Sun, Nov 14, 2010 at 4:17 AM, Poul-Henning Kamp <phk at phk.freebsd.dk>wrote:

> In message <AANLkTimcxdUpF7hO1=76wCJGng9pEZ99_mVrZwdMUgb1 at mail.gmail.com>,
> Vinc
> ent writes:
>
> >I am using the grace mode and I am wondering how the old objects are
> stored
> >and used in varnish. For example, if I have the following setup:
>
> Until their TTL + their grace periode expires.
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101114/a1ef17d4/attachment.html>

From cooltechemail at gmail.com  Sun Nov 14 15:14:56 2010
From: cooltechemail at gmail.com (Vincent Wells)
Date: Sun, 14 Nov 2010 07:14:56 -0800
Subject: issues with purge via varnishadm ran by crontab
In-Reply-To: <AANLkTimBvEmPP=R2XuvrXPKULvz2ViV-TaM6mgVx6r5r@mail.gmail.com>
References: <AANLkTin=6WH2Z+4PuYBK7WVqzPp9p9zrgmgZy_XxbN+-@mail.gmail.com>
	<AANLkTimBvEmPP=R2XuvrXPKULvz2ViV-TaM6mgVx6r5r@mail.gmail.com>
Message-ID: <AANLkTinwsQFDox5dnFRpi0HDVrXUPnCUcy5CfNVVLTf9@mail.gmail.com>

Hi Anders,

Thank you for your reply. I tried to start /bin/sh and type the varnishadm
purge command. It worked fine and I could see the purge in purge.list. I
also checked /var/log/cron and I am sure the varnishadm purge command was
executed by cron, but the purge does not appear in the purge.list. Any
ideas?

Vincent

On Sun, Nov 14, 2010 at 5:03 AM, thebog <thebog at gmail.com> wrote:

> Hi,
>
> if you do not see it in the list. Then there is a big possibility that
> the script never makes it into varnishadm. So I think it's safe to say
> that the problem is in the script somehow, and not varnish.
> Not that I am gonna debug your script, but take things into account
> that cron is often run by /bin/sh which may have different env
> variables than your normal shell (And therefore maybe a different
> PATH).
>
> Type /bin/sh and work from there.
>
> YS
> Anders Berg
>
>
> On Sun, Nov 14, 2010 at 7:37 AM, Vincent <cooltechemail at gmail.com> wrote:
> > Hi All,
> > I am using 2.1.3 and I am having issue with purge via the varnishadm
> > command. Basically if I put the following command in crontab, the system
> > will run the command as defined in crontab but it seems that varnish will
> > ignore the command:
> > varnishadm  -T 127.0.0.1:8000 purge req.http.host == "www.example.com"
> > However, if I manually type the above command in the command line, there
> is
> > no problem. I've checked that www.example.com is not in purge.list when
> the
> > command is ran using crontab but it will be included in purge.list when I
> > manually type the command.
> > Is this a bug?
> > Thanks,
> > Vincent
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101114/47e9623f/attachment.html>

From perbu at varnish-software.com  Sun Nov 14 18:31:11 2010
From: perbu at varnish-software.com (Per Buer)
Date: Sun, 14 Nov 2010 19:31:11 +0100
Subject: issues with purge via varnishadm ran by crontab
In-Reply-To: <AANLkTin=6WH2Z+4PuYBK7WVqzPp9p9zrgmgZy_XxbN+-@mail.gmail.com>
References: <AANLkTin=6WH2Z+4PuYBK7WVqzPp9p9zrgmgZy_XxbN+-@mail.gmail.com>
Message-ID: <AANLkTindesMOPEZp6S4NnZTcgfHCeLe5_Osjn=u-Uz1Y@mail.gmail.com>

On Sun, Nov 14, 2010 at 7:37 AM, Vincent <cooltechemail at gmail.com> wrote:
> Hi All,
> I am using 2.1.3 and I am having issue with purge via the?varnishadm
> command. Basically if I put the following command in crontab, the system
> will run the command as defined in crontab but it seems that varnish will
> ignore the command:
> varnishadm ?-T 127.0.0.1:8000 purge req.http.host == "www.example.com"
> However, if I manually type the above command in the command line, there is
> no problem. I've checked that www.example.com is not in purge.list when the
> command is ran using crontab but it will be included in purge.list when I
> manually type the command.
> Is this a bug?

Probably not. Putting actual commands with options in crontab can
sometimes lead to headaches with quotes, parameters and special
characters and addition to the problems Anders brought up. If I where
you I'd just wrap it in a simple shell script and just call the one
script from your crontab.

-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From perbu at varnish-software.com  Sun Nov 14 18:36:26 2010
From: perbu at varnish-software.com (Per Buer)
Date: Sun, 14 Nov 2010 19:36:26 +0100
Subject: www.varnish-cache.org/installation/freebsd error
In-Reply-To: <4CDF9D0C.7090905@idontwatch.tv>
References: <4CDF9D0C.7090905@idontwatch.tv>
Message-ID: <AANLkTi=QBqMD68UstPB26KCaTu0LGHsyB=u3FNCB-GV-@mail.gmail.com>

Thanks Justin.

People can direct these to the mailing list, kick me if nothing happens.


Per.

On Sun, Nov 14, 2010 at 9:25 AM, Justin <justin at idontwatch.tv> wrote:
> Hey there,
>
> I'd like to point out that on
> http://www.varnish-cache.org/installation/freebsd the line that reads:
>
> cd /usr/ports/varnish && make install clean
>
> should read:
>
> cd /usr/ports/www/varnish && make install clean
>
> Trac didn't seem like the appropriate place to submit this to those in
> charge. If this mailing list isn't please let me know or forward it on to
> those in charge of these sorts of things.
>
> Thanks!
>
> -Justin
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From phk at phk.freebsd.dk  Sun Nov 14 21:32:14 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sun, 14 Nov 2010 21:32:14 +0000
Subject: Grace Mode Questions 
In-Reply-To: Your message of "Sun, 14 Nov 2010 07:08:58 PST."
	<AANLkTinw8ZjDt2SsBvZPuC5O2bCBWPVpaXog7Du_joQ0@mail.gmail.com> 
Message-ID: <2965.1289770334@critter.freebsd.dk>

In message <AANLkTinw8ZjDt2SsBvZPuC5O2bCBWPVpaXog7Du_joQ0 at mail.gmail.com>, Vinc
ent Wells writes:

>Thank you for your reply. I know it is TTL+grace. The questions is if grace
>is much longer than TTL, does it mean varnish will keep multiple expired
>copies of the same objected instead of the most recent one?

Korrect:  There is no code to enforce only having one graced object.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From Glen.Kelly at fire.tas.gov.au  Sun Nov 14 22:20:13 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Mon, 15 Nov 2010 09:20:13 +1100
Subject: Vcl command restart and multiple backends
In-Reply-To: <AANLkTi=ZuKjGGZFqXw9TT20pGq3z_GSwN-mEDHXGb-sa@mail.gmail.com>
References: <674B23B1CEE7B140B4868996CE484506109CB20EA2@hoexchange1.fire.tas.gov.au>
	<AANLkTi=ZuKjGGZFqXw9TT20pGq3z_GSwN-mEDHXGb-sa@mail.gmail.com>
Message-ID: <674B23B1CEE7B140B4868996CE484506109CB20FF8@hoexchange1.fire.tas.gov.au>

Hi Per

Thanks for your advice. I had previously had saint mode set up as per the url.

After see another users vcl on the mailing list over the weekend I have changed my vcl to

                   #    if (beresp.status == 500) {
                   #        set beresp.saintmode = 30s;
                   #        restart;
                   #    }

                       if (beresp.status != 200 && beresp.status != 403 && beresp.status != 404 && beresp.status != 301 && beresp.status != 302) {
                           set beresp.saintmode = 30s;
                           restart;
                       }

Hopefully this will help.

Glen

[cid:tasfire_wakeup_small6b89.jpg]<http://www.fire.tas.gov.au/Show?pageId=colDayLightSavings>
P Do you need to print this? Consider the environment, prevent paper waste.
CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.

From: Per Buer [mailto:perbu at varnish-software.com]
Sent: Friday, 12 November 2010 8:49 PM
To: Glen Kelly
Cc: varnish-misc at varnish-cache.org
Subject: Re: Vcl command restart and multiple backends

On Fri, Nov 12, 2010 at 12:52 AM, Glen Kelly <Glen.Kelly at fire.tas.gov.au<mailto:Glen.Kelly at fire.tas.gov.au>> wrote:
How does varnish handle request to multiple backend when the vcl "restart" command is run.

Does it use the same backend as the initial request or does it use another backend in accordance to the configured director.

You should look at "Saint Mode" - http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html#saint-mode

When enabled it will blacklist failing objects from their backends.



--
Per Buer,  Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer

Varnish user? Take our survey: http://www.varnish-cache.org/user-survey-november-2010
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101115/be72eae0/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tasfire_wakeup_small6b89.jpg
Type: image/jpeg
Size: 27459 bytes
Desc: tasfire_wakeup_small6b89.jpg
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101115/be72eae0/attachment.jpg>

From rickytato at r2consulting.it  Mon Nov 15 10:35:15 2010
From: rickytato at r2consulting.it (rickytato rickytato)
Date: Mon, 15 Nov 2010 11:35:15 +0100
Subject: Objects and objects head
Message-ID: <AANLkTinepKqZaszAbQseeEayqGxWEvM4SHAbUF5e4jeE@mail.gmail.com>

I've two varnish server, balanced from DNS-RR, so the connection are pretty
same ; the configuration are the same.
But one server have a munin graph veri different from the other. In the
first server object and object head are the same, but the other no; in the
first server after one week the object start to expunge, but in the second
not.
Why?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101115/545529f1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Number of objects-1.png
Type: image/png
Size: 16751 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101115/545529f1/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Number of objects.png
Type: image/png
Size: 18081 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101115/545529f1/attachment-0001.png>

From pablort at gmail.com  Thu Nov 11 17:39:56 2010
From: pablort at gmail.com (Pablo Borges)
Date: Thu, 11 Nov 2010 15:39:56 -0200
Subject: Varnish 2.1.2 returning old version in cache
Message-ID: <AANLkTi=GDGGHV7fa8=1E_XYgTk21uJ4KDebWj0SQ7Vxp@mail.gmail.com>

Hey there,

I found out that one of my varnish clusters (still running 2.1.2) responded
an old version of an object in between hundreds of requests.

The expected behaviour is an error 500, but somehow the object (which has a
60s ttl, btw) was returned. I wonder if this is a known bug that has been
fixed.

Here's my wget output (cleaned out a bit) in sequence:

--2010-11-11 15:10:00--

HTTP request sent, awaiting response...
  HTTP/1.1 500 Internal Server Error
  Server: Varnish
  Retry-After: 0
  Content-Type: text/html; charset=utf-8
  Content-Length: 359
  Date: Thu, 11 Nov 2010 17:10:01 GMT
  Connection: close
  X-Cache: MISS
  X-Cache-Hits: 0
  X-Age: 1
2010-11-11 15:10:01 ERROR 500: Internal Server Error.


--2010-11-11 15:10:32--


HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Vary: Host,Accept-Encoding
  Expires: Thu, 11 Nov 2010 15:32:16 GMT
  Cache-Control: max-age=60, public
  Content-Type: text/html; charset=UTF-8
  Content-Length: 36294
  Date: Thu, 11 Nov 2010 17:10:32 GMT
  Connection: keep-alive
  X-Cache: HIT
  X-Cache-Hits: 4
  X-Age: 5956
Length: 36294 (35K) [text/html]
Saving to: `/dev/null'


--2010-11-11 15:10:40--

HTTP request sent, awaiting response...
  HTTP/1.1 500 Internal Server Error
  Server: Varnish
  Retry-After: 0
  Content-Type: text/html; charset=utf-8
  Content-Length: 359
  Date: Thu, 11 Nov 2010 17:10:41 GMT
  Connection: close
  X-Cache: MISS
  X-Cache-Hits: 0
  X-Age: 0
2010-11-11 15:10:41 ERROR 500: Internal Server Error.


--2010-11-11 15:10:42--

HTTP request sent, awaiting response...
  HTTP/1.1 500 Internal Server Error
  Server: Varnish
  Retry-After: 0
  Content-Type: text/html; charset=utf-8
  Content-Length: 359
  Date: Thu, 11 Nov 2010 17:10:43 GMT
  Connection: close
  X-Cache: MISS
  X-Cache-Hits: 0
  X-Age: 0
2010-11-11 15:10:43 ERROR 500: Internal Server Error.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101111/b23bdd68/attachment.html>

From tfheen at varnish-software.com  Mon Nov 15 12:27:30 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Mon, 15 Nov 2010 13:27:30 +0100
Subject: 503 due to content-length error? microwave issue?
In-Reply-To: <AANLkTikMPJN3acEoo7=1gDxfFjKHAWRVJW-+Ven7TxBS@mail.gmail.com>
	(Alan Dixon's message of "Wed, 10 Nov 2010 10:07:47 -0500")
References: <AANLkTinfhD36=S7icD6hDis2rP_1rugJDMf-E99=y=no@mail.gmail.com>
	<AANLkTinwf0sCa6kJMwxWTGRKJ8AoSDEicMjDSi=fDAmY@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902C838FFF@TMG-EVS02.torstar.net>
	<AANLkTikMPJN3acEoo7=1gDxfFjKHAWRVJW-+Ven7TxBS@mail.gmail.com>
Message-ID: <87zkta3h71.fsf@qurzaw.linpro.no>

]] Alan Dixon 

| 1. Adding varnish triggered the problem.
| 2. Changing the MTU solved the problem.
| 3. The system administrators of the ISP had seen a similar
| problem/solution in the past.
| 
| My limited understanding is that varnish's role in the problem was
| just being more demanding ("standards sensitive"), whereas Apache was
| able to handle the bad data better.

It looks like she was hitting timeouts in varnish.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From cooltechemail at gmail.com  Tue Nov 16 07:58:28 2010
From: cooltechemail at gmail.com (Vincent Wells)
Date: Mon, 15 Nov 2010 23:58:28 -0800
Subject: Grace Mode Questions
In-Reply-To: <2965.1289770334@critter.freebsd.dk>
References: <AANLkTinw8ZjDt2SsBvZPuC5O2bCBWPVpaXog7Du_joQ0@mail.gmail.com>
	<2965.1289770334@critter.freebsd.dk>
Message-ID: <AANLkTi=utsV2BuqkDA33+PSCV=Xpr0EdNNjDiEk6LPei@mail.gmail.com>

Hi Poul-Henning,

If there are multiple graced object, which one will be used in cases when
graced objects are needed? (for example, if the backend is sick). Will the
most recent one be used or varnish will randomly pick one?

Thanks,

Vincent

On Sun, Nov 14, 2010 at 1:32 PM, Poul-Henning Kamp <phk at phk.freebsd.dk>wrote:

> In message <AANLkTinw8ZjDt2SsBvZPuC5O2bCBWPVpaXog7Du_joQ0 at mail.gmail.com>,
> Vinc
> ent Wells writes:
>
> >Thank you for your reply. I know it is TTL+grace. The questions is if
> grace
> >is much longer than TTL, does it mean varnish will keep multiple expired
> >copies of the same objected instead of the most recent one?
>
> Korrect:  There is no code to enforce only having one graced object.
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101115/ee8bc57e/attachment.html>

From phk at phk.freebsd.dk  Tue Nov 16 08:31:13 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 16 Nov 2010 08:31:13 +0000
Subject: Grace Mode Questions 
In-Reply-To: Your message of "Mon, 15 Nov 2010 23:58:28 PST."
	<AANLkTi=utsV2BuqkDA33+PSCV=Xpr0EdNNjDiEk6LPei@mail.gmail.com> 
Message-ID: <98095.1289896273@critter.freebsd.dk>

In message <AANLkTi=utsV2BuqkDA33+PSCV=Xpr0EdNNjDiEk6LPei at mail.gmail.com>, Vinc
ent Wells writes:

>If there are multiple graced object, which one will be used in cases when
>graced objects are needed? (for example, if the backend is sick). Will the
>most recent one be used or varnish will randomly pick one?

That is a good question, I think "random" is an apt description right now.

It migh be a good idea to change that to take the "least expired" object.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From lassewesth at gmail.com  Tue Nov 16 12:20:42 2010
From: lassewesth at gmail.com (Lasse Westh-Nielsen)
Date: Tue, 16 Nov 2010 12:20:42 +0000
Subject: Concurrent cache misses
Message-ID: <AANLkTi=oj9PiBVmopVRcv+mx54g7x5+nhAmTfxcZJ9WJ@mail.gmail.com>

Hey,

Does Varnish have an equivalent of Squid's "collapsed forwarding" feature*?

Or, are there other ways to prevent a storm of requests landing on my
origin web servers when I introduce new content?

(We will be handling revalidation, this is just about preventing a
meltdown for content that has never been cached before)

 - Lasse




*: http://devel.squid-cache.org/collapsed_forwarding/
http://www.squid-cache.org/Versions/v2/2.7/cfgman/collapsed_forwarding.html



From perbu at varnish-software.com  Tue Nov 16 12:32:21 2010
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 16 Nov 2010 13:32:21 +0100
Subject: Concurrent cache misses
In-Reply-To: <AANLkTi=oj9PiBVmopVRcv+mx54g7x5+nhAmTfxcZJ9WJ@mail.gmail.com>
References: <AANLkTi=oj9PiBVmopVRcv+mx54g7x5+nhAmTfxcZJ9WJ@mail.gmail.com>
Message-ID: <AANLkTi=pK+b1rvp=4C+L-TBK-sZx+n4RwbAO3S=8rgiH@mail.gmail.com>

Hi,

On Tue, Nov 16, 2010 at 1:20 PM, Lasse Westh-Nielsen
<lassewesth at gmail.com> wrote:
> Hey,
>
> Does Varnish have an equivalent of Squid's "collapsed forwarding" feature*?

Yes. We've had that since 1.0.

-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From lassewesth at gmail.com  Tue Nov 16 12:47:54 2010
From: lassewesth at gmail.com (Lasse Westh-Nielsen)
Date: Tue, 16 Nov 2010 12:47:54 +0000
Subject: Concurrent cache misses
In-Reply-To: <AANLkTi=pK+b1rvp=4C+L-TBK-sZx+n4RwbAO3S=8rgiH@mail.gmail.com>
References: <AANLkTi=oj9PiBVmopVRcv+mx54g7x5+nhAmTfxcZJ9WJ@mail.gmail.com>
	<AANLkTi=pK+b1rvp=4C+L-TBK-sZx+n4RwbAO3S=8rgiH@mail.gmail.com>
Message-ID: <AANLkTimPBSxdD7dTLpU6-jUMn7g+qCSMx+8-_bXfM8gM@mail.gmail.com>

On Tue, Nov 16, 2010 at 12:32 PM, Per Buer <perbu at varnish-software.com> wrote:
>
> On Tue, Nov 16, 2010 at 1:20 PM, Lasse Westh-Nielsen
> <lassewesth at gmail.com> wrote:
>>
>> Does Varnish have an equivalent of Squid's "collapsed forwarding" feature*?
>
> Yes. We've had that since 1.0.

Thanks.

I couldn't find any documentation; do I take it that it is always on?

 - Lasse



From perbu at varnish-software.com  Tue Nov 16 13:13:36 2010
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 16 Nov 2010 14:13:36 +0100
Subject: Concurrent cache misses
In-Reply-To: <AANLkTimPBSxdD7dTLpU6-jUMn7g+qCSMx+8-_bXfM8gM@mail.gmail.com>
References: <AANLkTi=oj9PiBVmopVRcv+mx54g7x5+nhAmTfxcZJ9WJ@mail.gmail.com>
	<AANLkTi=pK+b1rvp=4C+L-TBK-sZx+n4RwbAO3S=8rgiH@mail.gmail.com>
	<AANLkTimPBSxdD7dTLpU6-jUMn7g+qCSMx+8-_bXfM8gM@mail.gmail.com>
Message-ID: <AANLkTinu0R2xgNhHcKJJOgr5iMpEx-qahvRrOYZe7TKZ@mail.gmail.com>

On Tue, Nov 16, 2010 at 1:47 PM, Lasse Westh-Nielsen
<lassewesth at gmail.com> wrote:
> On Tue, Nov 16, 2010 at 12:32 PM, Per Buer <perbu at varnish-software.com> wrote:
>>
>> On Tue, Nov 16, 2010 at 1:20 PM, Lasse Westh-Nielsen
>> <lassewesth at gmail.com> wrote:
>>>
>>> Does Varnish have an equivalent of Squid's "collapsed forwarding" feature*?
>>
>> Yes. We've had that since 1.0.
>
> Thanks.
>
> I couldn't find any documentation; do I take it that it is always on?

Yes.
-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From tdevelioglu at ebuddy.com  Tue Nov 16 13:24:01 2010
From: tdevelioglu at ebuddy.com (Taylan Develioglu)
Date: Tue, 16 Nov 2010 14:24:01 +0100
Subject: Objects and objects head
In-Reply-To: <AANLkTinepKqZaszAbQseeEayqGxWEvM4SHAbUF5e4jeE@mail.gmail.com>
References: <AANLkTinepKqZaszAbQseeEayqGxWEvM4SHAbUF5e4jeE@mail.gmail.com>
Message-ID: <1289913841.2863.107.camel@oasis>

On Mon, 2010-11-15 at 11:35 +0100, rickytato rickytato wrote:
> I've two varnish server, balanced from DNS-RR, so the connection are
> pretty same ;

I'm surprised they are.

>  the configuration are the same.
> But one server have a munin graph veri different from the other. In
> the first server object and object head are the same, but the other
> no; in the first server after one week the object start to expunge,
> but in the second not.
> Why?

Does your backend send any Vary: header ? Varnish will cache different
instances of an object based on the presence of it.

We have a similar situation where the object:head ratio is different on
each node. Not DNS-RR but rather url hash based balancing.
4 nodes, 3.2TB storage. All identical, except for the varnish stats and
various other kinds of general weirdness, crashes not excluded.

The discrepancy of head:object supposedly should have been fixed in
2.1.4 (and it was partially).

Honestly, I found varnish's stable status misleading. For us it's been
anything but.




From rickytato at r2consulting.it  Tue Nov 16 13:50:20 2010
From: rickytato at r2consulting.it (rickytato rickytato)
Date: Tue, 16 Nov 2010 14:50:20 +0100
Subject: Objects and objects head
In-Reply-To: <1289913841.2863.107.camel@oasis>
References: <AANLkTinepKqZaszAbQseeEayqGxWEvM4SHAbUF5e4jeE@mail.gmail.com>
	<1289913841.2863.107.camel@oasis>
Message-ID: <AANLkTin=NFjKJLcW3AOazSToaUqgnxdcOicEMMT7N-K_@mail.gmail.com>

2010/11/16 Taylan Develioglu <tdevelioglu at ebuddy.com>

> On Mon, 2010-11-15 at 11:35 +0100, rickytato rickytato wrote:
> > I've two varnish server, balanced from DNS-RR, so the connection are
> > pretty same ;
>
> I'm surprised they are.
>

The traffic's not much, but there are ~25r/s on servers.



> Does your backend send any Vary: header ? Varnish will cache different
> instances of an object based on the presence of it.
>

No, the backend are 2 Nginx servers which manage only static contents.
Varnish is 2.1.4 for all two servers.

I can't understand why server1 object and object head are ugual, while
aren't on server2.

The HTTP header you can see here:
http://1.citynews-milanotoday.stgy.it/pictures/20100903/polizia_5.jpeg

1.citynews-milanotoday.it is in DNS-RR on two IP address (server1 and
server2); no Vary header.


rr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101116/e3766710/attachment.html>

From superdupont at gmail.com  Tue Nov 16 16:16:48 2010
From: superdupont at gmail.com (Carlos Valiente)
Date: Tue, 16 Nov 2010 16:16:48 +0000
Subject: 503 responses with "backend write error: 11"
Message-ID: <AANLkTi=rCKBkX8DkNnSAzVuNUzRa_UfP0ica9R=CiejJ@mail.gmail.com>

Hi! I'm running varnish 2.1.13 on OpenSuSE 11.1 64-bit. From time to
time I'm seeing these kind of errors:

   22 ReqStart     c 10.0.0.1 2396 1481986113
   22 RxRequest    c POST
   22 RxURL        c /backend/statistics/
   22 RxProtocol   c HTTP/1.1
   22 RxHeader     c Host: www.example.com
   22 RxHeader     c User-Agent: Mozilla/5.0 (Windows; U; Windows NT
5.1; en-GB; rv:1.9.2.4) Gecko/20100527 Firefox/3.6.4
   22 RxHeader     c Accept: application/json, text/javascript, */*
   22 RxHeader     c Accept-Language: en-gb,en;q=0.5
   22 RxHeader     c Accept-Encoding: gzip,deflate
   22 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
   22 RxHeader     c Keep-Alive: 115
   22 RxHeader     c Connection: keep-alive
   22 RxHeader     c Content-Type: application/x-www-form-urlencoded;
charset=UTF-8
   22 RxHeader     c X-Requested-With: XMLHttpRequest
   22 RxHeader     c Referer: http://www.example.com/foo/
   22 RxHeader     c Content-Length: 29847
   22 RxHeader     c Cookie: foo=bar
   22 RxHeader     c Pragma: no-cache
   22 RxHeader     c Cache-Control: no-cache
   22 VCL_call     c recv
   22 VCL_return   c pass
   22 VCL_call     c hash
   22 VCL_return   c hash
   22 VCL_call     c pass
   22 VCL_return   c pass
   22 Backend      c 20 default backend3
   22 Backend      c 20 default backend3
   22 FetchError   c backend write error: 11
   22 VCL_call     c error
   22 VCL_return   c deliver
   22 VCL_call     c deliver
   22 VCL_return   c deliver
   22 TxProtocol   c HTTP/1.1
   22 TxStatus     c 503
   22 TxResponse   c Service Unavailable
   22 TxHeader     c Server: Varnish
   22 TxHeader     c Retry-After: 0
   22 TxHeader     c Content-Type: text/html; charset=utf-8
   22 TxHeader     c Content-Length: 419
   22 TxHeader     c Date: Tue, 16 Nov 2010 15:00:01 GMT
   22 TxHeader     c X-Varnish: 1481986113
   22 TxHeader     c Age: 5
   22 TxHeader     c Via: 1.1 varnish
   22 TxHeader     c Connection: close
   22 TxHeader     c X-Cache: MISS
   22 Length       c 419
   22 ReqEnd       c 1481986113 1289919596.007149696
1289919601.033036232 0.000028849 5.025856972 0.000029564
   22 SessionClose c error
   22 StatSess     c 10.0.0.1 2396 5 1 1 0 1 0 250 419

I have a couple of questions regarding this issue:

  1. It seems to me that Varnish fails when it tries to *write* the
request to the backend -- is this correct?

  2. The integer value '11' reported in "backend write error: 11" --
is it the value of ``errno`` for the failed call to ``write(2)``?

  3. If [2] above is correct: In the platform where Varnish is
running, 11 seems to correspond to EAGAIN:

        $ python -c 'import errno; print errno.EAGAIN'
        11
        $

     Is there any way of telling Varnish to try the write(2) again?

Cheers,

Carlos



From johannes.brandstetter at 1und1.de  Wed Nov 17 08:58:25 2010
From: johannes.brandstetter at 1und1.de (Johannes Brandstetter)
Date: Wed, 17 Nov 2010 09:58:25 +0100
Subject: Testing -s persistent, getting "Out of space in persistent silo"
In-Reply-To: <526F23AC-8ADA-4696-8D68-ABF8C3B02019@locaweb.com.br>
References: <c0feb4371003300852y400bdf6fr8be1467194c0843d@mail.gmail.com>
	<526F23AC-8ADA-4696-8D68-ABF8C3B02019@locaweb.com.br>
Message-ID: <4CE39931.4020100@1und1.de>

Hi,

I have the same issue here. But disk space seems not to be the issue there is 
plenty of free space.

Any other suggestions?

Cheers,

Johannes

On 10/01/2010 03:57 PM, Jo?o Gabriel wrote:
> df -h
>
> look your disk free space :D
>
> Jo?o Gabriel
> CT-Linux
>
>
>
>
> On Mar 30, 2010, at 12:52 PM, Maximilian Sch?fmann wrote:
>
>> Hi all,
>>
>> I'm currently testing 2.1 with -s
>> persistent,/var/lib/varnish/$INSTANCE/cache.bin,14G (on Ubuntu)
>> After a while, I start seeing these every few minutes
>>
>> varnishd[16181]: Child (7977) said Out of space in persistent silo
>> varnishd[16181]: Child (7977) said Committing suicide, restart will make space
>>
>> with the effect of the cache being empty after the child restarts (it
>> was far from filled before these crashes...).
>>
>> Best, Max
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



From idiaz at e-nnovva.com  Tue Nov 16 17:57:08 2010
From: idiaz at e-nnovva.com (Ivan Dario Diaz)
Date: Tue, 16 Nov 2010 12:57:08 -0500
Subject: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
Message-ID: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>

Hi all,

First i want to thank to all you for this great app. We use varnish in
all our website projects.

I have only one issue, we used to install varnish binaries on Debian
Lenny using backports.debian.org. But we are migrating to the official
varnish repo for obvious reasons. My new servers have addressed the
elastic IP's from 50.0.0.0 range. But i can't access your repositories
from 050/8 address.

 All my servers are deployed in amazon EC2, recently IANA has given the
Network 050/8 to ARIN (see
http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml ) and Amazon brought some on the IPv4 Space 50.0.0.0/8, for unknown reasons i cant access your IPv4 adresses; it seems that you have the IPv4 Network  87.238.36.96/27 or at least some IP's bettween 87.238.36.97  and 87.238.36.126.

Surfing on the net there are some references of system administrators
whose have blocked the 50.0.0.0/8 pool. Probably it was used to botnet
activities and/or IP Spoofing. But now there is no reason to block. IANA
assigned these addresses in 02/2010.

I have tested several IP's of Redpill Linpro AS (it seems like your
Hosting Provider) and mostly of them works, but varnish related (deb
trac planet www) IP's doesn't work. As you know there are a huge of
users in amazon EC2 using Varnish. It will be a problem to all of them.

I'm not saying that the blocking is in your servers. But all my tests
appoint to that. Please help us with that.

PD: Please forgive me if my English is not polite. Spanish is my natural
language.

Regards,

-- 


Ivan Dario Diaz Hoyos

____________________________________
Administrador de Sistemas
PBX: (57)(1) - 327 5888 Ext 2540
Email: idiaz at e-nnovva.com 

Este mensaje se dirige exclusivamente a su destinatario y puede contener
informaci?n privilegiada o confidencial. Si no es usted el destinatario
indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n
y/o copia sin autorizaci?n est? prohibida en virtud de la legislaci?n
vigente. Si ha recibido este mensaje por error, le rogamos que nos lo
comunique inmediatamente por esta misma v?a y proceda a su destrucci?n.
El correo electr?nico v?a Internet no permite asegurar la
confidencialidad de los mensajes que se transmiten ni su integridad o
correcta recepci?n. e-NNOVVA no asume ninguna responsabilidad por estas
circunstancias. 

This message is intended exclusively for its addresses and may contain
information that is CONFIDENTIAL and protected by a professional
privilege or whose disclosure is prohibited by law. If you are not the
intended recipient you are hereby notified that any read, dissemination,
copy or disclosure of this communication is strictly prohibited by law.
If this message has been received in error, please immediately notify us
via e-mail and delete it.  Internet e-mail neither guarantees the
confidentiality nor the integrity or proper receipt of the messages
sent. e-NNOVVA does not assume any liability for those circumstances.

________________________________________________________________________


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101116/83eb0f0b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Firma_Left.gif
Type: image/gif
Size: 3333 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101116/83eb0f0b/attachment.gif>

From idiaz at e-nnovva.com  Tue Nov 16 19:42:13 2010
From: idiaz at e-nnovva.com (Ivan Dario Diaz)
Date: Tue, 16 Nov 2010 14:42:13 -0500
Subject: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
In-Reply-To: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>
References: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>
Message-ID: <1289936533.2634.139.camel@sysadmin-linux.e-nnovva.com>

Hi,

Surfing in amazon EC2 forums that is the IPv4 pools of amazon EC2

50.16.0.0/15 (50.16.0.0 - 50.17.255.255)
50.18.0.0/18 (50.18.0.0 - 50.18.63.255) NEW

Details Here: https://forums.aws.amazon.com/ann.jspa?annID=830

Please unblock at least these ranges.

Regards,
-- 


Ivan Dario Diaz Hoyos

____________________________________
Administrador de Sistemas
PBX: (57)(1) - 327 5888 Ext 2540
Email: idiaz at e-nnovva.com 

Este mensaje se dirige exclusivamente a su destinatario y puede contener
informaci?n privilegiada o confidencial. Si no es usted el destinatario
indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n
y/o copia sin autorizaci?n est? prohibida en virtud de la legislaci?n
vigente. Si ha recibido este mensaje por error, le rogamos que nos lo
comunique inmediatamente por esta misma v?a y proceda a su destrucci?n.
El correo electr?nico v?a Internet no permite asegurar la
confidencialidad de los mensajes que se transmiten ni su integridad o
correcta recepci?n. e-NNOVVA no asume ninguna responsabilidad por estas
circunstancias. 

This message is intended exclusively for its addresses and may contain
information that is CONFIDENTIAL and protected by a professional
privilege or whose disclosure is prohibited by law. If you are not the
intended recipient you are hereby notified that any read, dissemination,
copy or disclosure of this communication is strictly prohibited by law.
If this message has been received in error, please immediately notify us
via e-mail and delete it.  Internet e-mail neither guarantees the
confidentiality nor the integrity or proper receipt of the messages
sent. e-NNOVVA does not assume any liability for those circumstances.

________________________________________________________________________


-----Mensaje original-----
De: Ivan Diaz <idiaz at e-nnovva.com>
Para: varnish-misc at varnish-cache.org <varnish-misc at varnish-cache.org>
Asunto: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
Fecha: Tue, 16 Nov 2010 12:57:08 -0500

Hi all,

First i want to thank to all you for this great app. We use varnish in
all our website projects.

I have only one issue, we used to install varnish binaries on Debian
Lenny using backports.debian.org. But we are migrating to the official
varnish repo for obvious reasons. My new servers have addressed the
elastic IP's from 50.0.0.0 range. But i can't access your repositories
from 050/8 address.

All my servers are deployed in amazon EC2, recently IANA has given the
Network 050/8 to ARIN (see
http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml ) and Amazon brought some on the IPv4 Space 50.0.0.0/8, for unknown reasons i cant access your IPv4 adresses; it seems that you have the IPv4 Network  87.238.36.96/27 or at least some IP's bettween 87.238.36.97  and 87.238.36.126.

Surfing on the net there are some references of system administrators
whose have blocked the 50.0.0.0/8 pool. Probably it was used to botnet
activities and/or IP Spoofing. But now there is no reason to block. IANA
assigned these addresses in 02/2010.

I have tested several IP's of Redpill Linpro AS (it seems like your
Hosting Provider) and mostly of them works, but varnish related (deb
trac planet www) IP's doesn't work. As you know there are a huge of
users in amazon EC2 using Varnish. It will be a problem to all of them.

I'm not saying that the blocking is in your servers. But all my tests
appoint to that. Please help us with that.

PD: Please forgive me if my English is not polite. Spanish is my natural
language.

Regards,

-- 


Ivan Dario Diaz Hoyos

____________________________________

Administrador de Sistemas
PBX: (57)(1) - 327 5888 Ext 2540
Email: idiaz at e-nnovva.com 

Este mensaje se dirige exclusivamente a su destinatario y puede contener
informaci?n privilegiada o confidencial. Si no es usted el destinatario
indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n
y/o copia sin autorizaci?n est? prohibida en virtud de la legislaci?n
vigente. Si ha recibido este mensaje por error, le rogamos que nos lo
comunique inmediatamente por esta misma v?a y proceda a su destrucci?n.
El correo electr?nico v?a Internet no permite asegurar la
confidencialidad de los mensajes que se transmiten ni su integridad o
correcta recepci?n. e-NNOVVA no asume ninguna responsabilidad por estas
circunstancias. 

This message is intended exclusively for its addresses and may contain
information that is CONFIDENTIAL and protected by a professional
privilege or whose disclosure is prohibited by law. If you are not the
intended recipient you are hereby notified that any read, dissemination,
copy or disclosure of this communication is strictly prohibited by law.
If this message has been received in error, please immediately notify us
via e-mail and delete it.  Internet e-mail neither guarantees the
confidentiality nor the integrity or proper receipt of the messages
sent. e-NNOVVA does not assume any liability for those circumstances.

________________________________________________________________________





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101116/d8fccfdc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Firma_Left.gif
Type: image/gif
Size: 3333 bytes
Desc: Firma_Left.gif
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101116/d8fccfdc/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Firma_Left.gif
Type: image/gif
Size: 3333 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101116/d8fccfdc/attachment-0001.gif>

From tfheen at varnish-software.com  Wed Nov 17 14:04:16 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 17 Nov 2010 15:04:16 +0100
Subject: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
In-Reply-To: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com> (Ivan
	Dario Diaz's message of "Tue, 16 Nov 2010 12:57:08 -0500")
References: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>
Message-ID: <87ipzwxd0f.fsf@qurzaw.linpro.no>

]] Ivan Dario Diaz 

| I have tested several IP's of Redpill Linpro AS (it seems like your
| Hosting Provider) and mostly of them works, but varnish related (deb
| trac planet www) IP's doesn't work. As you know there are a huge of
| users in amazon EC2 using Varnish. It will be a problem to all of them.
| 
| I'm not saying that the blocking is in your servers. But all my tests
| appoint to that. Please help us with that.

Indeed, our firewall had the ?block bogon networks? turned on, and it
seems like the list hadn't been updated in a while.  I've fixed this
now.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From tfheen at varnish-software.com  Wed Nov 17 14:07:40 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 17 Nov 2010 15:07:40 +0100
Subject: Offtopic Bikeshedding: deflate compression
In-Reply-To: <alpine.DEB.2.00.1011131218170.5692@green.darkmere.gen.nz> (Simon
	Lyall's message of "Sat, 13 Nov 2010 12:49:34 +1300 (NZDT)")
References: <alpine.DEB.2.00.1011131218170.5692@green.darkmere.gen.nz>
Message-ID: <87eiakxcur.fsf@qurzaw.linpro.no>

]] Simon Lyall 

Hi,

| This bugs me when I see the sample code for Compression here
| http://www.varnish-cache.org/trac/wiki/FAQ/Compression
| 
| The bit that bugs me is that "deflate" compression doesn't work with
| IE since Microsoft implimented it differently from everybody else.

That's silly of them.  I've updated the page now, I believe it's not
good and correct.  Feedback is most welcome.

Regards,
-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From tfheen at varnish-software.com  Wed Nov 17 14:09:49 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 17 Nov 2010 15:09:49 +0100
Subject: Dealing with large unwanted objects
In-Reply-To: <AANLkTimmDJeD1k6raZ=zyqsSKHMNMe-e4N74PHG25-QM@mail.gmail.com>
	(Mark Moseley's message of "Fri, 5 Nov 2010 11:54:37 -0700")
References: <AANLkTi=SSBCF+_EGa+TYZmEarvR7hmTW4jp4L8p0iPEv@mail.gmail.com>
	<874obyzwxt.fsf@qurzaw.linpro.no>
	<AANLkTinFKoYCon0mWqdCjMk=DSh9VuMGGeHyPcrFG-Uu@mail.gmail.com>
	<877hgsur6n.fsf@qurzaw.linpro.no>
	<AANLkTimmDJeD1k6raZ=zyqsSKHMNMe-e4N74PHG25-QM@mail.gmail.com>
Message-ID: <87aal8xcr6.fsf@qurzaw.linpro.no>

]] Mark Moseley 

| On Fri, Nov 5, 2010 at 1:10 AM, Tollef Fog Heen
| <tfheen at varnish-software.com> wrote:
| > ]] Mark Moseley
| >
| > | Wow, that's cool. I figured that the body had also been completely
| > | fetched by the time vcl_fetch is invoked, so that changes things
| > | completely. I'll definitely give this a try.
| >
| > It is in 2.0, but not in 2.1.
| >
| > | BTW, is there any "cast" trick to be able to use arithmetic operators
| > | like ">" with beresp.http.Content-Length (or other numeric headers)?
| >
| > There'll be a way to do that in 3.0, yes.
| 
| So if it's not fetched the complete body yet, once 'restart' is
| called, does varnish just dump the body?

No, we don't.

| When I tried out your recipe
| (which works like a charm, btw), I noticed that varnish still reads
| the entirety of the first (i.e. the restarted one) request's response
| body, before proceeding to fetch the restarted request.

We need to do this, since the backend has already filled the connection
with the body of the request, we just haven't read it.  Alternatively,
we could close the connection and open a new one, but that might well
have a higher overhead.

| All this happens over localhost, so it's quite fast, but in the
| interest of efficiency, is there something I can set or call so that
| it closes that first connection almost immediately? Having to refetch
| a 800meg file off of NFS might hurt -- even if a good chunk of it is
| still in the OS block cache.

You'd need to do this using inline C, but yes, anything is possible.
(Sorry, I don't have an example for it here)

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From crcook at gmail.com  Wed Nov 17 16:56:45 2010
From: crcook at gmail.com (Chris Cook)
Date: Wed, 17 Nov 2010 11:56:45 -0500
Subject: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
In-Reply-To: <87ipzwxd0f.fsf@qurzaw.linpro.no>
References: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>
	<87ipzwxd0f.fsf@qurzaw.linpro.no>
Message-ID: <EB253729-BCEC-41A2-924A-A28942660023@gmail.com>

Just as a friendly FYI - your bogon filter should probably match this: http://www.team-cymru.org/Services/Bogons/bogon-bn-nonagg.txt (and realistically you might as well not block most of that since it will all be allocated within the next 12 months).

On Nov 17, 2010, at 9:04 AM, Tollef Fog Heen wrote:

> ]] Ivan Dario Diaz 
> 
> | I have tested several IP's of Redpill Linpro AS (it seems like your
> | Hosting Provider) and mostly of them works, but varnish related (deb
> | trac planet www) IP's doesn't work. As you know there are a huge of
> | users in amazon EC2 using Varnish. It will be a problem to all of them.
> | 
> | I'm not saying that the blocking is in your servers. But all my tests
> | appoint to that. Please help us with that.
> 
> Indeed, our firewall had the ?block bogon networks? turned on, and it
> seems like the list hadn't been updated in a while.  I've fixed this
> now.
> 
> -- 
> Tollef Fog Heen
> Varnish Software
> t: +47 21 98 62 64
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From richard.chiswell at mangahigh.com  Wed Nov 17 17:02:11 2010
From: richard.chiswell at mangahigh.com (Richard Chiswell)
Date: Wed, 17 Nov 2010 17:02:11 +0000
Subject: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
In-Reply-To: <EB253729-BCEC-41A2-924A-A28942660023@gmail.com>
References: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>	<87ipzwxd0f.fsf@qurzaw.linpro.no>
	<EB253729-BCEC-41A2-924A-A28942660023@gmail.com>
Message-ID: <4CE40A93.5020905@mangahigh.com>

Well, it's worth blocking
0.0.0.0/8, 10.0.0.0/8, 127.0.0.0/8, 169.254.0.0/16, 172.16.0.0/12, 
192.168.0.0/16
as those ranges shouldn't appear on the public intertubes. [ 
http://www.rfc-editor.org/rfc/rfc3330.txt ]

Rich

On 17/11/2010 16:56, Chris Cook wrote:
> Just as a friendly FYI - your bogon filter should probably match this: http://www.team-cymru.org/Services/Bogons/bogon-bn-nonagg.txt (and realistically you might as well not block most of that since it will all be allocated within the next 12 months).
>
> On Nov 17, 2010, at 9:04 AM, Tollef Fog Heen wrote:
>
>> ]] Ivan Dario Diaz
>>
>> | I have tested several IP's of Redpill Linpro AS (it seems like your
>> | Hosting Provider) and mostly of them works, but varnish related (deb
>> | trac planet www) IP's doesn't work. As you know there are a huge of
>> | users in amazon EC2 using Varnish. It will be a problem to all of them.
>> |
>> | I'm not saying that the blocking is in your servers. But all my tests
>> | appoint to that. Please help us with that.
>>
>> Indeed, our firewall had the ?block bogon networks? turned on, and it
>> seems like the list hadn't been updated in a while.  I've fixed this
>> now.
>>
>> -- 
>> Tollef Fog Heen
>> Varnish Software
>> t: +47 21 98 62 64
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From crcook at gmail.com  Wed Nov 17 17:06:21 2010
From: crcook at gmail.com (Chris Cook)
Date: Wed, 17 Nov 2010 12:06:21 -0500
Subject: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
In-Reply-To: <4CE40A93.5020905@mangahigh.com>
References: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>	<87ipzwxd0f.fsf@qurzaw.linpro.no>
	<EB253729-BCEC-41A2-924A-A28942660023@gmail.com>
	<4CE40A93.5020905@mangahigh.com>
Message-ID: <02C50682-497E-4682-AA0C-C11806730EA8@gmail.com>

Correct - I was mostly referring to the unallocated yet public space (the 11 remaining /8s that are currently "bogon") that will most likely be allocated and then not bogon within the next 12 months (probably much sooner).  If you're manually updating your bogon filters, most likely you'll run into problems like this (I have when we were still blocking 2.0.0.0/8).

If you want to get really fancy you can just use this list instead - http://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt (though I would only use such a list if I was automatically updating my bogon filters)

On Nov 17, 2010, at 12:02 PM, Richard Chiswell wrote:

> Well, it's worth blocking
> 0.0.0.0/8, 10.0.0.0/8, 127.0.0.0/8, 169.254.0.0/16, 172.16.0.0/12, 192.168.0.0/16
> as those ranges shouldn't appear on the public intertubes. [ http://www.rfc-editor.org/rfc/rfc3330.txt ]
> 
> Rich
> 
> On 17/11/2010 16:56, Chris Cook wrote:
>> Just as a friendly FYI - your bogon filter should probably match this: http://www.team-cymru.org/Services/Bogons/bogon-bn-nonagg.txt (and realistically you might as well not block most of that since it will all be allocated within the next 12 months).
>> 
>> On Nov 17, 2010, at 9:04 AM, Tollef Fog Heen wrote:
>> 
>>> ]] Ivan Dario Diaz
>>> 
>>> | I have tested several IP's of Redpill Linpro AS (it seems like your
>>> | Hosting Provider) and mostly of them works, but varnish related (deb
>>> | trac planet www) IP's doesn't work. As you know there are a huge of
>>> | users in amazon EC2 using Varnish. It will be a problem to all of them.
>>> |
>>> | I'm not saying that the blocking is in your servers. But all my tests
>>> | appoint to that. Please help us with that.
>>> 
>>> Indeed, our firewall had the ?block bogon networks? turned on, and it
>>> seems like the list hadn't been updated in a while.  I've fixed this
>>> now.
>>> 
>>> -- 
>>> Tollef Fog Heen
>>> Varnish Software
>>> t: +47 21 98 62 64
>>> 
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>> 
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> 
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101117/fe978a0e/attachment.html>

From ask at develooper.com  Wed Nov 17 20:15:35 2010
From: ask at develooper.com (=?utf-8?Q?Ask_Bj=C3=B8rn_Hansen?=)
Date: Wed, 17 Nov 2010 12:15:35 -0800
Subject: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
In-Reply-To: <02C50682-497E-4682-AA0C-C11806730EA8@gmail.com>
References: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>
	<87ipzwxd0f.fsf@qurzaw.linpro.no>
	<EB253729-BCEC-41A2-924A-A28942660023@gmail.com>
	<4CE40A93.5020905@mangahigh.com>
	<02C50682-497E-4682-AA0C-C11806730EA8@gmail.com>
Message-ID: <7D599E1D-34C2-4632-9720-6F03F2F945CA@develooper.com>

On Nov 17, 2010, at 9:06, Chris Cook <crcook at gmail.com> wrote:

> If you want to get really fancy you can just use this list instead - http://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt (though I would only use such a list if I was automatically updating my bogon filters)

They also offer a BGP feed with this information which might be easier to maintain.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101117/96e3de2e/attachment.html>

From tfheen at varnish-software.com  Thu Nov 18 07:04:34 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Thu, 18 Nov 2010 08:04:34 +0100
Subject: To varnish sysadmin: Blocked IPv4 Space 50.0.0.0/8
In-Reply-To: <EB253729-BCEC-41A2-924A-A28942660023@gmail.com> (Chris Cook's
	message of "Wed, 17 Nov 2010 11:56:45 -0500")
References: <1289930228.2634.133.camel@sysadmin-linux.e-nnovva.com>
	<87ipzwxd0f.fsf@qurzaw.linpro.no>
	<EB253729-BCEC-41A2-924A-A28942660023@gmail.com>
Message-ID: <8762vvxgcd.fsf@qurzaw.linpro.no>

]] Chris Cook 

| Just as a friendly FYI - your bogon filter should probably match this:
| http://www.team-cymru.org/Services/Bogons/bogon-bn-nonagg.txt (and
| realistically you might as well not block most of that since it will
| all be allocated within the next 12 months).

Yeah, I just turned off bogon filtering (but kept the RFC1918 filter in
place).

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 62 64



From russ at vshift.com  Thu Nov 18 20:17:12 2010
From: russ at vshift.com (Ruslan Sivak)
Date: Thu, 18 Nov 2010 15:17:12 -0500
Subject: ESI within an img tag
Message-ID: <4CE589C8.7020402@vshift.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

<img src='http://www.somesite.com/test.gif?random=<esi:include
src="/esi/random.pl"/>' alt="" height="1" width="1" />


This doesn't seem to get processed properly by varnish, but something like

<esi:include src="/esi/random.pl"/>


does get processed properly.  Is it because it's within a single quote?
 Is there a way of escaping things or something to get it to work the
way I want?

Russ
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFM5YnIGUZBYlCheYcRArlxAKDR9iWWj8USPrdpsOdjCsu0dL91YACgwc3J
DsgijQdUYKaUpywXPq81+3A=
=59gQ
-----END PGP SIGNATURE-----



From lists at rtty.us  Thu Nov 18 21:41:57 2010
From: lists at rtty.us (Bob Camp)
Date: Thu, 18 Nov 2010 16:41:57 -0500
Subject: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5
	x86
In-Reply-To: <674B23B1CEE7B140B4868996CE484506109C8A0793@hoexchange1.fire.tas.gov.au>
References: <Act74xB84v0tjXmARRurjFpm8yFobA==>
	<674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>
	<AANLkTi=1vJ6kC9mJYpwmub0hsRoxCenxP0WzaawUR3+O@mail.gmail.com>
	<674B23B1CEE7B140B4868996CE484506109C8A0761@hoexchange1.fire.tas.gov.au>
	<1FE6E036-49E5-452D-8E94-997A83973C3B@rtty.us>
	<674B23B1CEE7B140B4868996CE484506109C8A0793@hoexchange1.fire.tas.gov.au>
Message-ID: <F7DD9932E3CF4232829040894E39C0B2@vectron.com>

Hi

 

The issue with CentOS / yum / rpm / keys seems to have been fixed. Varnish
now installs fine via Yum.

 

Bob

 

  _____  

From: Glen Kelly [mailto:Glen.Kelly at fire.tas.gov.au] 
Sent: Thursday, November 04, 2010 8:29 PM
To: Bob Camp
Cc: varnish-misc at varnish-cache.org
Subject: RE: Varnish 2.1.4 rpm's - bad key error when installing on Centos
5.5 x86

 

Hi Bob

 

Ah, I understand now.

 

I assumed the * in Ruslan's example was just for me to change and insert the
relevant filename, I see now he actually meant it as a literal '*'. Oops.

 

That is interesting to know the rpm can resolve interlinking dependencies
like that, an upgrade will be considerably faster if I don't need to
uninstall, reinstall and then reconfigure.

 

Thank you for the clarification.

 

Regards Glen

 

From: Bob Camp [mailto:lists at rtty.us] 
Sent: Friday, 5 November 2010 11:18 AM
To: Glen Kelly
Cc: varnish-misc at varnish-cache.org
Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos
5.5 x86

 

Hi

 

If you have both of the files present you can do a rpm --nosignature -Uvh
varnish*

 

That will get rid of the failed dependency issue.

 

Bob

 

On Nov 4, 2010, at 7:21 PM, Glen Kelly wrote:

 

Hi Ruslan

 

That sort of worked, I had to use the -nosignature flag instead. Also
because of dependencies it would not let me upgrade, I had to uninstall and
reinstall.

 

But big negative is that it breaks the rpm -q option. Until these rpm's are
fixed I cannot use them in production. I assume the developers read this
mailing list but in case they don't how do I go about raising a bug report?

 

(root at bolrpdev1:/tmp)# rpm --nosignature -Uvh
varnish-libs-2.1.4-1.el5.x86_64.rpm

error: Failed dependencies:

        varnish-libs = 2.1.3-1.el5 is needed by (installed)
varnish-2.1.3-1.el5.x86_64

 

(root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-2.1.4-1.el5.x86_64.rpm

error: Failed dependencies:

        varnish-libs = 2.1.4-1.el5 is needed by varnish-2.1.4-1.el5.x86_64

 

 (root at bolrpdev1:/tmp)# rpm -qa | grep varn

varnish-libs-2.1.3-1.el5

varnish-2.1.3-1.el5

 

(root at bolrpdev1:/tmp)# rpm -e varnish-2.1.3-1.el5

warning: /etc/varnish/default.vcl saved as /etc/varnish/default.vcl.rpmsave

warning: /etc/sysconfig/varnish saved as /etc/sysconfig/varnish.rpmsave

warning: /etc/logrotate.d/varnish saved as /etc/logrotate.d/varnish.rpmsave

 

(root at bolrpdev1:/tmp)# rpm -e varnish-libs-2.1.3-1.el5

 

(root at bolrpdev1:/tmp)# rpm --nosignature -i
varnish-libs-2.1.4-1.el5.x86_64.rpm

 

(root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-2.1.4-1.el5.x86_64.rpm

 

(root at bolrpdev1:/tmp)# rpm -qa | grep varn

error: rpmdbNextIterator: skipping h#     395 Header V4 RSA/SHA256
signature: BAD, key ID c4deffeb

error: rpmdbNextIterator: skipping h#     394 Header V4 RSA/SHA256
signature: BAD, key ID c4deffeb

 

(root at bolrpdev1:/tmp)# service varnish start

Starting varnish HTTP accelerator:                         [  OK  ]

 

(root at bolrpdev1:/tmp)# service varnishlog start

Starting varnish logging daemon:                           [  OK  ]

 

(root at bolrpdev1:/tmp)# varnishd -V

varnishd (varnish-2.1.4 SVN 5447M)

Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS

 

Regards Glen

 


 <http://www.fire.tas.gov.au/mysite/Show?pageId=colDayLightSavings>
<tasfire_wakeup_small4cad.jpg>


P Do you need to print this? Consider the environment, prevent paper waste.


CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by
legal professional privilege, and is intended only for the person or persons
to whom it is addressed. If you are not such a person, you are warned that
any disclosure, copying or dissemination of the information is unauthorised.
If you have received the transmission in error, please immediately contact
this office by telephone, fax or email, to inform us of the error and to
enable arrangements to be made for the destruction of the transmission, or
its return at our cost. No liability is accepted for any unauthorised use of
the information contained in this transmission.

 

From: Ruslan Sivak [mailto:russ at vshift.com] 
Sent: Friday, 5 November 2010 12:47 AM
To: Glen Kelly
Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos
5.5 x86

 

The workaround is to download the 2 rpms manually and run

rpm --signature -Uvh varnish*

I hope whoever is generating the rpms fixes this soon as well as generates
some x32 rpms.

Is this possibly the wrong list to ask for this?  Should it be on the  dev
list, or should I file an issue in the bug tracker?

Russ

On Nov 4, 2010 1:42 AM, "Glen Kelly" <Glen.Kelly at fire.tas.gov.au> wrote:
> Hi
> 
> I am receiving the same errors as described here when installing on Centos
5.5 x86.
> 
> error: rpmts_HdrFromFdno: Header V4 RSA/SHA256 signature: BAD, key ID
c4deffeb
> 
> Is there a known solution?
> 
> Also will the 2.1.4 rpm's be eventually available from
http://sourceforge.net/projects/varnish/files/?
> 
> Regards Glen
>

<tasfire_wakeup_small4cad.jpg>______________________________________________
_
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101118/a345eb86/attachment.html>

From Glen.Kelly at fire.tas.gov.au  Thu Nov 18 21:45:48 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Fri, 19 Nov 2010 08:45:48 +1100
Subject: Varnish 2.1.4 rpm's - bad key error when installing on Centos
	5.5 x86
In-Reply-To: <F7DD9932E3CF4232829040894E39C0B2@vectron.com>
References: <Act74xB84v0tjXmARRurjFpm8yFobA==>
	<674B23B1CEE7B140B4868996CE484506109C8A06B7@hoexchange1.fire.tas.gov.au>
	<AANLkTi=1vJ6kC9mJYpwmub0hsRoxCenxP0WzaawUR3+O@mail.gmail.com>
	<674B23B1CEE7B140B4868996CE484506109C8A0761@hoexchange1.fire.tas.gov.au>
	<1FE6E036-49E5-452D-8E94-997A83973C3B@rtty.us>
	<674B23B1CEE7B140B4868996CE484506109C8A0793@hoexchange1.fire.tas.gov.au>
	<F7DD9932E3CF4232829040894E39C0B2@vectron.com>
Message-ID: <674B23B1CEE7B140B4868996CE484506109DEC5BBA@hoexchange1.fire.tas.gov.au>

Hi Bob

Yes I noticed yesterday that issue 810 (http://www.varnish-cache.org/trac/ticket/810) had been closed and was going to trying installing 2.1.4 on our test server today.

Thank you for providing confirmation that the rpm's are now okay.

Regards Glen

From: Bob Camp [mailto:lists at rtty.us]
Sent: Friday, 19 November 2010 8:42 AM
To: Glen Kelly
Cc: varnish-misc at varnish-cache.org
Subject: RE: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86

Hi

The issue with CentOS / yum / rpm / keys seems to have been fixed. Varnish now installs fine via Yum.

Bob

________________________________
From: Glen Kelly [mailto:Glen.Kelly at fire.tas.gov.au]
Sent: Thursday, November 04, 2010 8:29 PM
To: Bob Camp
Cc: varnish-misc at varnish-cache.org
Subject: RE: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86

Hi Bob

Ah, I understand now.

I assumed the * in Ruslan's example was just for me to change and insert the relevant filename, I see now he actually meant it as a literal '*'. Oops.

That is interesting to know the rpm can resolve interlinking dependencies like that, an upgrade will be considerably faster if I don't need to uninstall, reinstall and then reconfigure.

Thank you for the clarification.

Regards Glen

From: Bob Camp [mailto:lists at rtty.us]
Sent: Friday, 5 November 2010 11:18 AM
To: Glen Kelly
Cc: varnish-misc at varnish-cache.org
Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86

Hi

If you have both of the files present you can do a rpm --nosignature -Uvh varnish*

That will get rid of the failed dependency issue.

Bob

On Nov 4, 2010, at 7:21 PM, Glen Kelly wrote:

Hi Ruslan

That sort of worked, I had to use the -nosignature flag instead. Also because of dependencies it would not let me upgrade, I had to uninstall and reinstall.

But big negative is that it breaks the rpm -q option. Until these rpm's are fixed I cannot use them in production. I assume the developers read this mailing list but in case they don't how do I go about raising a bug report?

(root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-libs-2.1.4-1.el5.x86_64.rpm
error: Failed dependencies:
        varnish-libs = 2.1.3-1.el5 is needed by (installed) varnish-2.1.3-1.el5.x86_64

(root at bolrpdev1:/tmp)# rpm --nosignature -Uvh varnish-2.1.4-1.el5.x86_64.rpm
error: Failed dependencies:
        varnish-libs = 2.1.4-1.el5 is needed by varnish-2.1.4-1.el5.x86_64

 (root at bolrpdev1:/tmp)# rpm -qa | grep varn
varnish-libs-2.1.3-1.el5
varnish-2.1.3-1.el5

(root at bolrpdev1:/tmp)# rpm -e varnish-2.1.3-1.el5
warning: /etc/varnish/default.vcl saved as /etc/varnish/default.vcl.rpmsave
warning: /etc/sysconfig/varnish saved as /etc/sysconfig/varnish.rpmsave
warning: /etc/logrotate.d/varnish saved as /etc/logrotate.d/varnish.rpmsave

(root at bolrpdev1:/tmp)# rpm -e varnish-libs-2.1.3-1.el5

(root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-libs-2.1.4-1.el5.x86_64.rpm

(root at bolrpdev1:/tmp)# rpm --nosignature -i varnish-2.1.4-1.el5.x86_64.rpm

(root at bolrpdev1:/tmp)# rpm -qa | grep varn
error: rpmdbNextIterator: skipping h#     395 Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
error: rpmdbNextIterator: skipping h#     394 Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb

(root at bolrpdev1:/tmp)# service varnish start
Starting varnish HTTP accelerator:                         [  OK  ]

(root at bolrpdev1:/tmp)# service varnishlog start
Starting varnish logging daemon:                           [  OK  ]

(root at bolrpdev1:/tmp)# varnishd -V
varnishd (varnish-2.1.4 SVN 5447M)
Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS

Regards Glen

<tasfire_wakeup_small4cad.jpg><http://www.fire.tas.gov.au/mysite/Show?pageId=colDayLightSavings>

P Do you need to print this? Consider the environment, prevent paper waste.

CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.


From: Ruslan Sivak [mailto:russ at vshift.com]
Sent: Friday, 5 November 2010 12:47 AM
To: Glen Kelly
Subject: Re: Varnish 2.1.4 rpm's - bad key error when installing on Centos 5.5 x86


The workaround is to download the 2 rpms manually and run

rpm --signature -Uvh varnish*

I hope whoever is generating the rpms fixes this soon as well as generates some x32 rpms.

Is this possibly the wrong list to ask for this?  Should it be on the  dev list, or should I file an issue in the bug tracker?

Russ
On Nov 4, 2010 1:42 AM, "Glen Kelly" <Glen.Kelly at fire.tas.gov.au<mailto:Glen.Kelly at fire.tas.gov.au>> wrote:
> Hi
>
> I am receiving the same errors as described here when installing on Centos 5.5 x86.
>
> error: rpmts_HdrFromFdno: Header V4 RSA/SHA256 signature: BAD, key ID c4deffeb
>
> Is there a known solution?
>
> Also will the 2.1.4 rpm's be eventually available from http://sourceforge.net/projects/varnish/files/?
>
> Regards Glen
>
<tasfire_wakeup_small4cad.jpg>_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101119/1f85dfd1/attachment.html>

From phk at phk.freebsd.dk  Thu Nov 18 21:54:22 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 18 Nov 2010 21:54:22 +0000
Subject: ESI within an img tag 
In-Reply-To: Your message of "Thu, 18 Nov 2010 15:17:12 EST."
	<4CE589C8.7020402@vshift.com> 
Message-ID: <1798.1290117262@critter.freebsd.dk>

In message <4CE589C8.7020402 at vshift.com>, Ruslan Sivak writes:

><img src='http://www.somesite.com/test.gif?random=<esi:include
>src="/esi/random.pl"/>' alt="" height="1" width="1" />
>
>This doesn't seem to get processed properly by varnish, but something like

By default we only process well-formed XML.

Try setting the param esi_syntax to 0x2


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From russ at vshift.com  Thu Nov 18 23:45:00 2010
From: russ at vshift.com (Ruslan Sivak)
Date: Thu, 18 Nov 2010 18:45:00 -0500
Subject: ESI within an img tag
In-Reply-To: <1798.1290117262@critter.freebsd.dk>
References: <1798.1290117262@critter.freebsd.dk>
Message-ID: <4CE5BA7C.9010904@vshift.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/18/2010 4:54 PM, Poul-Henning Kamp wrote:
> In message <4CE589C8.7020402 at vshift.com>, Ruslan Sivak writes:
> 
>> <img src='http://www.somesite.com/test.gif?random=<esi:include
>> src="/esi/random.pl"/>' alt="" height="1" width="1" />
>>
>> This doesn't seem to get processed properly by varnish, but something like
> 
> By default we only process well-formed XML.
> 
> Try setting the param esi_syntax to 0x2
> 
> 

Thank you, that worked.  I couldn't find any documentation on setting
the parameters, but ended up doing it in cat /etc/sysconfig/varnish

It would be helpful if all this stuff was documented somewhere.

Russ
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFM5bp8GUZBYlCheYcRApX4AJ4xTwsj0mCohaiBJT1JstKZOBHDzACgn2v0
8bMvFqJpMRjKPbGvltsd+20=
=laE5
-----END PGP SIGNATURE-----



From frederik.schumacher at googlemail.com  Thu Nov 18 16:38:24 2010
From: frederik.schumacher at googlemail.com (Frederik Schumacher)
Date: Thu, 18 Nov 2010 17:38:24 +0100
Subject: First request is always a miss (per client)
Message-ID: <AANLkTinLcW55eMPBnu=mReV3jfW9ii8WswVdqnsw_mKz@mail.gmail.com>

Hello,

I'm running a Pressflow site on a LAMP with Apache on port 8080 and
Varnish on port 80. My problem is, that the very first request from
any client is always a miss, this even happens on the same machine -
using for example Firefox for the first request (miss), then curl -I
(miss), then Firefox (hit), then curl -I (hit). Every request after
that is a cache hit. Let's say I request "localhost/foo" using Firefox
and it's a miss (and then gets cached), shouldn't "curl -I
http://localhost/foo" right after that be a hit? Is this expected
behavior, or a mis-configuration?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: default.vcl
Type: application/octet-stream
Size: 3344 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101118/0f355e1c/attachment.obj>

From kristian at varnish-software.com  Fri Nov 19 06:50:19 2010
From: kristian at varnish-software.com (Kristian Lyngstol)
Date: Fri, 19 Nov 2010 07:50:19 +0100
Subject: First request is always a miss (per client)
In-Reply-To: <AANLkTinLcW55eMPBnu=mReV3jfW9ii8WswVdqnsw_mKz@mail.gmail.com>
References: <AANLkTinLcW55eMPBnu=mReV3jfW9ii8WswVdqnsw_mKz@mail.gmail.com>
Message-ID: <20101119065017.GC618@sunrider>

On Thu, Nov 18, 2010 at 05:38:24PM +0100, Frederik Schumacher wrote:
> I'm running a Pressflow site on a LAMP with Apache on port 8080 and
> Varnish on port 80. My problem is, that the very first request from
> any client is always a miss, this even happens on the same machine -
> using for example Firefox for the first request (miss), then curl -I
> (miss), then Firefox (hit), then curl -I (hit). Every request after
> that is a cache hit. Let's say I request "localhost/foo" using Firefox
> and it's a miss (and then gets cached), shouldn't "curl -I
> http://localhost/foo" right after that be a hit? Is this expected
> behavior, or a mis-configuration?

The objects are most likely being cached independently. The most likely
suspect is the headers you Vary on. For pressflow, I believe that will be
Cookie and Accept-Encoding by default. If those are not normalized in
Varnish you will cache one object for each variation (hence Vary:) of the
client-provided Accept-Encoding and Cookie headers.

Since pressflow doesn't send cookies for anonymous users, that only really
leaves accept-encoding - which is the most common header you have to
normalize anyway.

Take a look at the example on the wiki [1] for more discussion, and look
for a Vary header in the response from your web server.

[1] http://www.varnish-cache.org/trac/wiki/VCLExampleNormalizeAcceptEncoding

-- 
Kristian Lyngst?l
Product Specialist, Varnish Software
Training and Commercial services:
http://www.varnish-software.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: Digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101119/085982d8/attachment.bin>

From phk at phk.freebsd.dk  Fri Nov 19 09:40:49 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 19 Nov 2010 09:40:49 +0000
Subject: ESI within an img tag 
In-Reply-To: Your message of "Thu, 18 Nov 2010 18:45:00 EST."
	<4CE5BA7C.9010904@vshift.com> 
Message-ID: <3639.1290159649@critter.freebsd.dk>

In message <4CE5BA7C.9010904 at vshift.com>, Ruslan Sivak writes:

>It would be helpful if all this stuff was documented somewhere.

Try this CLI command:

	param.show esi_syntax


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From flylm.ml at gmail.com  Sat Nov 20 17:08:12 2010
From: flylm.ml at gmail.com (FabD [ML])
Date: Sat, 20 Nov 2010 18:08:12 +0100
Subject: Varnish 2.1.3 / 2.1.4 on Debian
Message-ID: <AANLkTinV-Q7rsOOodroX8QG_snm6LarK1Jaib+GinYMO@mail.gmail.com>

Hi,

I've two questions about Varnish and its use on my Debian Lenny with
Apache2 (mpm-worker) as backend.

1. I'm using Varnish 2.1.3 on my blog (WP), I've seen (on Gtmetrix.com
monitoring) that sometimes Varnish send "no-compressed" content when
it should send "gzip" content. For the moment, I don't succeded to
reproduce manually that.

Maybe someone would have an idea how to explain that ?

2. I've tried to replace Varnish 2.1.3 (Debian Lenny-backports) by the
2.1.4 (from Varnish repository). Varnish seems to work fine as long as
you aren't logged. When you're logged, Varnish sends the output very
late (at the end of Apache KeepAliveTimeOut => 15s, 2 x 15 s for 4
ressources, 30 s to see a simple little page...).

Turn off Apache KeepAlive solves the trouble, but is-it a good way, we
don't have to do that with the 2.1.3 ?

Thanks for your feedback.
Fabien. (be indulgent with my english)



From perbu at varnish-software.com  Sat Nov 20 20:14:20 2010
From: perbu at varnish-software.com (Per Buer)
Date: Sat, 20 Nov 2010 21:14:20 +0100
Subject: Varnish 2.1.3 / 2.1.4 on Debian
In-Reply-To: <AANLkTinV-Q7rsOOodroX8QG_snm6LarK1Jaib+GinYMO@mail.gmail.com>
References: <AANLkTinV-Q7rsOOodroX8QG_snm6LarK1Jaib+GinYMO@mail.gmail.com>
Message-ID: <AANLkTim5E4rTur9uKK5MtLnbEiHZbtte_jk25xU6eNsr@mail.gmail.com>

Hi,

On Sat, Nov 20, 2010 at 6:08 PM, FabD [ML] <flylm.ml at gmail.com> wrote:
> Hi,
>
> I've two questions about Varnish and its use on my Debian Lenny with
> Apache2 (mpm-worker) as backend.
>
> 1. I'm using Varnish 2.1.3 on my blog (WP), I've seen (on Gtmetrix.com
> monitoring) that sometimes Varnish send "no-compressed" content when
> it should send "gzip" content. For the moment, I don't succeded to
> reproduce manually that.
>
> Maybe someone would have an idea how to explain that ?

Sure. If Apache sends compressed content it should also send a
"Vary: accept-encoding" along so Varnish will _vary_ the content
depending on the content of that header. If the backend doesn't
you might send compressed content to someone you hasn't indicated
that they support it.

> 2. I've tried to replace Varnish 2.1.3 (Debian Lenny-backports) by the
> 2.1.4 (from Varnish repository). Varnish seems to work fine as long as
> you aren't logged. When you're logged, Varnish sends the output very
> late (at the end of Apache KeepAliveTimeOut => 15s, 2 x 15 s for 4
> ressources, 30 s to see a simple little page...).

This was discussed here a couple of weeks back, I think. Try searching
the archives.

-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From perbu at varnish-software.com  Sun Nov 21 09:46:56 2010
From: perbu at varnish-software.com (Per Buer)
Date: Sun, 21 Nov 2010 10:46:56 +0100
Subject: Varnish 2.1.3 / 2.1.4 on Debian
In-Reply-To: <AANLkTi=Uzqt8ospXK7L7WL-9KxPy0QfDFBnc4byHirtH@mail.gmail.com>
References: <AANLkTinV-Q7rsOOodroX8QG_snm6LarK1Jaib+GinYMO@mail.gmail.com>
	<AANLkTim5E4rTur9uKK5MtLnbEiHZbtte_jk25xU6eNsr@mail.gmail.com>
	<AANLkTi=Uzqt8ospXK7L7WL-9KxPy0QfDFBnc4byHirtH@mail.gmail.com>
Message-ID: <AANLkTi=Swqm40whHeuXdxsh1ZX8vd7+7bSXyV6rcEcxY@mail.gmail.com>

Hi,

Please keep the correspondence on the mailing list.

On Sat, Nov 20, 2010 at 10:19 PM, FabD [ML] <flylm.ml at gmail.com> wrote:
>> Sure. If Apache sends compressed content it should also send a
>> "Vary: accept-encoding" along so Varnish will _vary_ the content
>> depending on the content of that header. If the backend doesn't
>> you might send compressed content to someone you hasn't indicated
>> that they support it.
>>
>
> In my case, the client supports gzip encoding and (sometimes)
> receives a "no-compressed" content until I restart varnish (or
> until the cache expires I guess).
> With this configuration of Apache2, it should not be possible ?
>
> => Header append Vary User-Agent env=!dont-vary
>
> I don't know if it's a bad configuration issue of Varnish or Apache2 ?

I don't know much about Apache2 and how it is configured. I don't know
what the above line does but I recommend that you read the  art about
the Vary header in the docs:
http://www.varnish-cache.org/docs/trunk/tutorial/increasing_your_hitrate.html#vary

Varnish does Vary just as instructed by the Vary header. There isn't
much you can do to alter the behavior except to manipulate the headers
that you vary on.


-- 
Per Buer,? Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer



From martin.boer at bizztravel.nl  Mon Nov 22 09:08:16 2010
From: martin.boer at bizztravel.nl (Martin Boer)
Date: Mon, 22 Nov 2010 10:08:16 +0100
Subject: problem with varnishadm (varnishd 2.1.2, debian package)
Message-ID: <4CEA3300.7000509@bizztravel.nl>

Hello,

We have the following setup;

2 varnish instances on 2 servers running in parallel and forwarding 
requests to each other to reduce load on the backends and to make sure 
both instances have as much data as possible.
When we do a release of our websites we have to purge a lot of data and 
to make sure that we don't end up in a messy state we do the following 
in a script;

1)/usr/bin/varnishadm -T $load_2:$port stop
2) usr/bin/varnishadm -T $load_1:$port "purge whatever"
3) /usr/bin/varnishadm -T $load_2:$port start


When we do this we get the following error after issueing (2);
         Command failed with error code 300

I -think- this is the response from (1) but at this moment I can't be 
sure as the errorcode seems to just pop up as from STDERR instead of
STDOUT as it doesn't stay in the formatted output I scripted around it.

After doing (3) we see;
         Child in state stopping

Which really seems to be a response from (3); it fits nicely in the 
formatted output and seems to be a valid response.


When we retry the script we get the following;
         Command failed with error code 300

Even before the script tells me it issued (1).
The scripted output from (1) then tells me
         Child in state stopped

The other 2 commands don't give any response at all.
When trying the script for the third time we get the following just 
before the scripted output from (1)
         Rejected 400
         CLI communication error (hdr)

Then there is also the following STDERR after running (2);
         Command failed with error code 300

After this we try the script one more time and everything seems to work 
all right.
Thoughts;

- The script only runs into problems when we haven't used it for a 
couple of weeks. When we use it more often it works fine.
- We use varnish 2.1.2, the debian package from testing.

So, I think there is a bug somewhere in the communication between 
varnishadm and varnishd.
Regards,
Martin





From l at lrowe.co.uk  Mon Nov 22 17:54:29 2010
From: l at lrowe.co.uk (Laurence Rowe)
Date: Mon, 22 Nov 2010 17:54:29 +0000
Subject: First request is always a miss (per client)
In-Reply-To: <20101119065017.GC618@sunrider>
References: <AANLkTinLcW55eMPBnu=mReV3jfW9ii8WswVdqnsw_mKz@mail.gmail.com>
	<20101119065017.GC618@sunrider>
Message-ID: <AANLkTiku9YQ7Q-BRY0umszmm1kZ0pzH32obEvjv4U4Pe@mail.gmail.com>

While Pressflow itself may not set cookies, with google analytics or
similar you'll see the cookies set by its javascript. I've used
something like this in the past to normalize cookies:

    if (req.http.Cookie) {
      # Simplify tokenization
      set req.http.Cookie = "; " req.http.Cookie;
      # Remove osas and google analytics cookies
      set req.http.Cookie = regsuball(req.http.Cookie, ";
(osas_id|__utm.)=[^;]*", "");
      # Finally remove cookie header if it is now empty
      if (!req.http.Cookie ~ "[^ ;]") {
        remove req.http.Cookie;
      } else {
        # Remove the leading delimiter
        set req.http.Cookie = regsub(req.http.Cookie, "^; ", "");
      }
    }

Laurence


On 19 November 2010 06:50, Kristian Lyngstol
<kristian at varnish-software.com> wrote:
> On Thu, Nov 18, 2010 at 05:38:24PM +0100, Frederik Schumacher wrote:
>> I'm running a Pressflow site on a LAMP with Apache on port 8080 and
>> Varnish on port 80. My problem is, that the very first request from
>> any client is always a miss, this even happens on the same machine -
>> using for example Firefox for the first request (miss), then curl -I
>> (miss), then Firefox (hit), then curl -I (hit). Every request after
>> that is a cache hit. Let's say I request "localhost/foo" using Firefox
>> and it's a miss (and then gets cached), shouldn't "curl -I
>> http://localhost/foo" right after that be a hit? Is this expected
>> behavior, or a mis-configuration?
>
> The objects are most likely being cached independently. The most likely
> suspect is the headers you Vary on. For pressflow, I believe that will be
> Cookie and Accept-Encoding by default. If those are not normalized in
> Varnish you will cache one object for each variation (hence Vary:) of the
> client-provided Accept-Encoding and Cookie headers.
>
> Since pressflow doesn't send cookies for anonymous users, that only really
> leaves accept-encoding - which is the most common header you have to
> normalize anyway.
>
> Take a look at the example on the wiki [1] for more discussion, and look
> for a Vary header in the response from your web server.
>
> [1] http://www.varnish-cache.org/trac/wiki/VCLExampleNormalizeAcceptEncoding
>
> --
> Kristian Lyngst?l
> Product Specialist, Varnish Software
> Training and Commercial services:
> http://www.varnish-software.com
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iQEcBAEBAgAGBQJM5h4lAAoJEIC1IG5rXEfqsNAIAJxrQBKqk9Z32575aS8npYra
> x2CzjT3Qzk9zaXfezVtBYgwCuazzV/A3kq5yj+XgydI6woh5rWwCDnbKB/d15e60
> 1ygAjEiW5DKMPj+4e/3svWHVm2DA1no5cNhn45tM3y/vm1LM+ytctNcgQKWDKS2v
> hxX1cMsEAQC30FdZ0DZqD/dCa4Kq2q7ySDxQty/F2VjFkVTONvzrauE8l/Q1mN2X
> z7J2HEjIp0DW/3WQ9IJ46MwjeoQxaq7Z/NgtF0I0TlOfLlv3imk2x+c0kgrPaBvE
> UEZLy9HW75Kp4YcKYbUDLfqj88XkN75oZY4lIwECpodGNy7Mt0ZXKmBSZ7Hg7i8=
> =WXrp
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



From pfreeman at entrepreneur.com  Mon Nov 22 22:49:12 2010
From: pfreeman at entrepreneur.com (pfreeman at entrepreneur.com)
Date: Mon, 22 Nov 2010 14:49:12 -0800 (PST)
Subject: configuration references piling up.
Message-ID: <1290466152.88039444@192.168.2.227>

It seems that we have references to the configuration which are piling up ... where they are normally less than 80 but sometimes go high for a short period, since Friday, the reference count (as displayed by "varnishadm vcl.list") has continued to grow into the tens of thousands (if left alone).  The count goes to normal on a child stop/start, but grows rapidly.  We made some vcl changes, which we rolled back to no effect.  varnishlog produces so much output, I was wondering if anyone has a tip on how to begin troubleshooting this problem? (perhaps which tags to filter from varnishlog and what to look for?).

Thanks,

Patrick Freeman




From Glen.Kelly at fire.tas.gov.au  Mon Nov 22 22:57:11 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Tue, 23 Nov 2010 09:57:11 +1100
Subject: Finding out number of healthy backends
Message-ID: <674B23B1CEE7B140B4868996CE484506109DEC605B@hoexchange1.fire.tas.gov.au>

Is there any way of using varnishstat to determine the number of healthy backends?

I currently use 'varnishstat -1 -f n_backend' but this only returns the number of backends, not the number of healthy backends.

I am using Nagios to monitor our environment and want to be able to monitor when we lose a backend. I already am monitoring the backends directly so I have that covered but I want to also monitor when varnish thinks it has lost a backend. i.e. monitoring at multiple levels

If this is not current achievable how do I go about requesting this feature to be added to varnish?

Regards Glen

[cid:tasfire_wakeup_small5f32.jpg]<http://www.fire.tas.gov.au/Show?pageId=colDayLightSavings>
P Do you need to print this? Consider the environment, prevent paper waste.
CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101123/bf7dc24f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tasfire_wakeup_small5f32.jpg
Type: image/jpeg
Size: 27459 bytes
Desc: tasfire_wakeup_small5f32.jpg
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101123/bf7dc24f/attachment.jpg>

From chaokovsky.lee at gmail.com  Tue Nov 23 01:48:15 2010
From: chaokovsky.lee at gmail.com (Chaos Lee)
Date: Tue, 23 Nov 2010 09:48:15 +0800
Subject: How to config varnish to NOT cache content within cookies
Message-ID: <AANLkTin3NhVr9J-k4oKjnQYNNC21cfeBZw5FWX3gQpMm@mail.gmail.com>

In varnish document, it said varnish dosen't cache content within cookies.
But I find varnish cached the content in my environment, so the login user
chaos at all. This is my config vcl:

backend www {
    .host = "www.sample.com";
    .port = "80";
}

backend v {
    .host = "v.sample.com";
    .port = "80";
}

backend u {
    .host = "u.sample.com";
    .port = "80";
}


backend so {
    .host = "so.sample.com";
    .port = "80";
}

acl purge {
    "localhost";
    "10.0.0.0"/8;
}

sub vcl_recv {
    set req.grace = 30m;

    if (req.http.x-forwarded-for) {
        set req.http.X-Forwarded-For = req.http.X-Forwarded-For ", "
client.ip;
    } else {
        set req.http.X-Forwarded-For = client.ip;
    }

    if (req.http.range && !req.url ~ "^/player/get\w*M3U8/.*") {
        error 403 "Forbidden";
    }

    if (req.http.host ~ "^(www.)?sample.com$") {
        set req.backend = www;
    } elsif (req.http.host ~ "^v.sample.com$") {
        set req.backend = v;
    } elsif (req.http.host ~ "^u.sample.com$") {
        set req.backend = u;
    } elsif (req.http.host ~ "^so.sample.com$") {
        set req.backend = so;
    } else {
        error 404 "Unknown virtual host";
    }

    # Allow ip from PURGE acl
    if (req.request == "PURGE") {
        if (!client.ip ~ purge) {
            error 405 "Not allowed.";
        }
            return (lookup);
    }

    if (req.request != "GET" && req.request != "HEAD") {
        return (pass);
    }

    if (req.http.authenticate || req.http.authorization) {
        return (pass);
    }

    if (req.http.cookie && req.http.cookie ~ "authtoken=") {
        return (pass);
    }

    return (lookup);
}

sub vcl_hit {
    if (req.request == "PURGE") {
        set obj.ttl = 0s;
        error 200 "Purged";
    }

    if (!obj.cacheable) {
        return (pass);
    }

    return (deliver);
}

sub vcl_miss {
    if (req.request == "PURGE") {
        error 405 "Not in cache";
    }

    return (fetch);
}

sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT on a29.squid";
    } else {
        set resp.http.X-Cache = "MISS from a29.squid";
    }

    # Remove unused varnish header
    remove resp.http.X-Varnish;
    return (deliver);
}

sub vcl_fetch {
    set beresp.grace = 30m;

    return (deliver);
}

sub vcl_pass {
    remove bereq.http.X-Varnish;

    return (pass);
}

sub vcl_error {
    set obj.http.Cache-Control = "no-store, no-cache, must-revalidate";
    set obj.http.Content-Type = "text/html; charset=utf-8";
    synthetic {""};
    return (deliver);
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101123/5e264fd1/attachment.html>

From lists at rtty.us  Tue Nov 23 02:09:37 2010
From: lists at rtty.us (Bob Camp)
Date: Mon, 22 Nov 2010 21:09:37 -0500
Subject: How to config varnish to NOT cache content within cookies
In-Reply-To: <AANLkTin3NhVr9J-k4oKjnQYNNC21cfeBZw5FWX3gQpMm@mail.gmail.com>
References: <AANLkTin3NhVr9J-k4oKjnQYNNC21cfeBZw5FWX3gQpMm@mail.gmail.com>
Message-ID: <D9E103B0-3660-49BE-8B3B-90F0D4B70AD1@rtty.us>

Hi

I believe that by terminating your VCL sections with "fetch",  "deliver" and "lookup" you have cut out the default VCL logic. By so doing you "hop around" much of the built in stuff.

Bob


On Nov 22, 2010, at 8:48 PM, Chaos Lee wrote:

> In varnish document, it said varnish dosen't cache content within cookies.  But I find varnish cached the content in my environment, so the login user chaos at all. This is my config vcl:
> 
> backend www {
>     .host = "www.sample.com";
>     .port = "80";
> }
> 
> backend v {
>     .host = "v.sample.com";
>     .port = "80";
> }
> 
> backend u {
>     .host = "u.sample.com";
>     .port = "80";
> }
> 
> 
> backend so {
>     .host = "so.sample.com";
>     .port = "80";
> }
> 
> acl purge {
>     "localhost";
>     "10.0.0.0"/8;
> }
> 
> sub vcl_recv {
>     set req.grace = 30m;
> 
>     if (req.http.x-forwarded-for) {
>         set req.http.X-Forwarded-For = req.http.X-Forwarded-For ", " client.ip;
>     } else {
>         set req.http.X-Forwarded-For = client.ip;
>     }
> 
>     if (req.http.range && !req.url ~ "^/player/get\w*M3U8/.*") {
>         error 403 "Forbidden";
>     }
> 
>     if (req.http.host ~ "^(www.)?sample.com$") {
>         set req.backend = www;
>     } elsif (req.http.host ~ "^v.sample.com$") {
>         set req.backend = v;
>     } elsif (req.http.host ~ "^u.sample.com$") {
>         set req.backend = u;
>     } elsif (req.http.host ~ "^so.sample.com$") {
>         set req.backend = so;
>     } else {
>         error 404 "Unknown virtual host";
>     }
> 
>     # Allow ip from PURGE acl
>     if (req.request == "PURGE") {
>         if (!client.ip ~ purge) {
>             error 405 "Not allowed.";
>         }
>             return (lookup);
>     }
> 
>     if (req.request != "GET" && req.request != "HEAD") {
>         return (pass);
>     }
>     
>     if (req.http.authenticate || req.http.authorization) {
>         return (pass);
>     }
> 
>     if (req.http.cookie && req.http.cookie ~ "authtoken=") {
>         return (pass);
>     }
> 
>     return (lookup);
> }
> 
> sub vcl_hit {
>     if (req.request == "PURGE") {
>         set obj.ttl = 0s;
>         error 200 "Purged";
>     }
> 
>     if (!obj.cacheable) {
>         return (pass);
>     }
> 
>     return (deliver);
> }
> 
> sub vcl_miss {
>     if (req.request == "PURGE") {
>         error 405 "Not in cache";
>     }
>     
>     return (fetch);
> }
> 
> sub vcl_deliver {
>     if (obj.hits > 0) {
>         set resp.http.X-Cache = "HIT on a29.squid";
>     } else {
>         set resp.http.X-Cache = "MISS from a29.squid";
>     }
> 
>     # Remove unused varnish header
>     remove resp.http.X-Varnish;
>     return (deliver);
> }
> 
> sub vcl_fetch {
>     set beresp.grace = 30m;
> 
>     return (deliver);
> }
> 
> sub vcl_pass {
>     remove bereq.http.X-Varnish;
>     
>     return (pass);
> }
> 
> sub vcl_error {
>     set obj.http.Cache-Control = "no-store, no-cache, must-revalidate";
>     set obj.http.Content-Type = "text/html; charset=utf-8";
>     synthetic {""};
>     return (deliver);
> }
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101122/9ecc3ae9/attachment.html>

From guido.bakker at gmail.com  Tue Nov 23 06:33:27 2010
From: guido.bakker at gmail.com (Guido Bakker)
Date: Tue, 23 Nov 2010 07:33:27 +0100
Subject: Finding out number of healthy backends
In-Reply-To: <674B23B1CEE7B140B4868996CE484506109DEC605B@hoexchange1.fire.tas.gov.au>
References: <674B23B1CEE7B140B4868996CE484506109DEC605B@hoexchange1.fire.tas.gov.au>
Message-ID: <AANLkTiktWu5e6C-VDWkRsQP6WkTxk7Dr4M+oxHn_xHKP@mail.gmail.com>

We build a nagios check using:

varnishadm -T localhost:6082 debug.health | grep \ is

This works fine.

2010/11/22 Glen Kelly <Glen.Kelly at fire.tas.gov.au>

> Is there any way of using varnishstat to determine the number of healthy
> backends?
>
>
>
> I currently use ?varnishstat -1 -f n_backend? but this only returns the
> number of backends, not the number of healthy backends.
>
>
>
> I am using Nagios to monitor our environment and want to be able to monitor
> when we lose a backend. I already am monitoring the backends directly so I
> have that covered but I want to also monitor when varnish thinks it has lost
> a backend. i.e. monitoring at multiple levels
>
>
>
> If this is not current achievable how do I go about requesting this feature
> to be added to varnish?
>
>
>
> Regards Glen
>
>
>   <http://www.fire.tas.gov.au/Show?pageId=colDayLightSavings> P Do you
> need to print this? Consider the environment, prevent paper waste.  CONFIDENTIALITY
> NOTICE AND DISCLAIMER
> The information in this transmission may be confidential and/or protected
> by legal professional privilege, and is intended only for the person or
> persons to whom it is addressed. If you are not such a person, you are
> warned that any disclosure, copying or dissemination of the information is
> unauthorised. If you have received the transmission in error, please
> immediately contact this office by telephone, fax or email, to inform us of
> the error and to enable arrangements to be made for the destruction of the
> transmission, or its return at our cost. No liability is accepted for any
> unauthorised use of the information contained in this transmission.
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101123/32364c58/attachment.html>

From amoiz.shine at gmail.com  Tue Nov 23 09:05:32 2010
From: amoiz.shine at gmail.com (Sharl.Jimh.Tsin)
Date: Tue, 23 Nov 2010 17:05:32 +0800
Subject: question about the client ip sent to back-end server
Message-ID: <AANLkTik6cggX_zOJDgiCpjSrSwzqcLKxYqKpqep_M+0v@mail.gmail.com>

hi,everyone
  now i have a problem about the IPs from cache server to web
server.that is,my varnish runs on port 80 in the front,and there is a
hiawatha server running on port 81.for cache,I deny all requests from
81 port for someone access the back-end server directly,configuration
in the hiawatha is:
_______
allow 127.0.0.1,localhost etc.
deny all.
_______
it seems that it is OK now.BUT the poll plugin on my website breaken
now :( Once one person polled,then other visitors can not poll again
due to the poll plugin's keep-state.
the poll plugin distinguish the visitor has polled or not by its
IP,and in fact all visitors request the hiawatha via varnish on the
same ip,it is 127.0.0.1.
it is really a conflict!!

is there any solution? my friends? thanks!

Best regards,
Sharl.Jimh.Tsin (From China **Obviously Taiwan INCLUDED**)



From chaokovsky.lee at gmail.com  Tue Nov 23 09:45:22 2010
From: chaokovsky.lee at gmail.com (Chaos Lee)
Date: Tue, 23 Nov 2010 17:45:22 +0800
Subject: How to config varnish to NOT cache content within cookies
In-Reply-To: <D9E103B0-3660-49BE-8B3B-90F0D4B70AD1@rtty.us>
References: <AANLkTin3NhVr9J-k4oKjnQYNNC21cfeBZw5FWX3gQpMm@mail.gmail.com>
	<D9E103B0-3660-49BE-8B3B-90F0D4B70AD1@rtty.us>
Message-ID: <AANLkTi=DaDF+Jt0CO_ifSdDa0HONSJxQg05thtvAeXnQ@mail.gmail.com>

Thanks for your help. Remove the Set-Cookie when hit can resolve the
problem, and this is what the squid do.

2010/11/23 Bob Camp <lists at rtty.us>

> Hi
>
> I believe that by terminating your VCL sections with "fetch",  "deliver"
> and "lookup" you have cut out the default VCL logic. By so doing you "hop
> around" much of the built in stuff.
>
> Bob
>
>
> On Nov 22, 2010, at 8:48 PM, Chaos Lee wrote:
>
> In varnish document, it said varnish dosen't cache content within cookies.
> But I find varnish cached the content in my environment, so the login user
> chaos at all. This is my config vcl:
>
> backend www {
>     .host = "www.sample.com";
>     .port = "80";
> }
>
> backend v {
>     .host = "v.sample.com";
>     .port = "80";
> }
>
> backend u {
>     .host = "u.sample.com";
>     .port = "80";
> }
>
>
> backend so {
>     .host = "so.sample.com";
>     .port = "80";
> }
>
> acl purge {
>     "localhost";
>     "10.0.0.0"/8;
> }
>
> sub vcl_recv {
>     set req.grace = 30m;
>
>     if (req.http.x-forwarded-for) {
>         set req.http.X-Forwarded-For = req.http.X-Forwarded-For ", "
> client.ip;
>     } else {
>         set req.http.X-Forwarded-For = client.ip;
>     }
>
>     if (req.http.range && !req.url ~ "^/player/get\w*M3U8/.*") {
>         error 403 "Forbidden";
>     }
>
>     if (req.http.host ~ "^(www.)?sample.com$") {
>         set req.backend = www;
>     } elsif (req.http.host ~ "^v.sample.com$") {
>         set req.backend = v;
>     } elsif (req.http.host ~ "^u.sample.com$") {
>         set req.backend = u;
>     } elsif (req.http.host ~ "^so.sample.com$") {
>         set req.backend = so;
>     } else {
>         error 404 "Unknown virtual host";
>     }
>
>     # Allow ip from PURGE acl
>     if (req.request == "PURGE") {
>         if (!client.ip ~ purge) {
>             error 405 "Not allowed.";
>         }
>             return (lookup);
>     }
>
>     if (req.request != "GET" && req.request != "HEAD") {
>         return (pass);
>     }
>
>     if (req.http.authenticate || req.http.authorization) {
>         return (pass);
>     }
>
>     if (req.http.cookie && req.http.cookie ~ "authtoken=") {
>         return (pass);
>     }
>
>     return (lookup);
> }
>
> sub vcl_hit {
>     if (req.request == "PURGE") {
>         set obj.ttl = 0s;
>         error 200 "Purged";
>     }
>
>     if (!obj.cacheable) {
>         return (pass);
>     }
>
>     return (deliver);
> }
>
> sub vcl_miss {
>     if (req.request == "PURGE") {
>         error 405 "Not in cache";
>     }
>
>     return (fetch);
> }
>
> sub vcl_deliver {
>     if (obj.hits > 0) {
>         set resp.http.X-Cache = "HIT on a29.squid";
>     } else {
>         set resp.http.X-Cache = "MISS from a29.squid";
>     }
>
>     # Remove unused varnish header
>     remove resp.http.X-Varnish;
>     return (deliver);
> }
>
> sub vcl_fetch {
>     set beresp.grace = 30m;
>
>     return (deliver);
> }
>
> sub vcl_pass {
>     remove bereq.http.X-Varnish;
>
>     return (pass);
> }
>
> sub vcl_error {
>     set obj.http.Cache-Control = "no-store, no-cache, must-revalidate";
>     set obj.http.Content-Type = "text/html; charset=utf-8";
>     synthetic {""};
>     return (deliver);
> }
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101123/515b93fd/attachment.html>

From scaunter at topscms.com  Tue Nov 23 14:05:48 2010
From: scaunter at topscms.com (Caunter, Stefan)
Date: Tue, 23 Nov 2010 09:05:48 -0500
Subject: Finding out number of healthy backends
In-Reply-To: <AANLkTiktWu5e6C-VDWkRsQP6WkTxk7Dr4M+oxHn_xHKP@mail.gmail.com>
References: <674B23B1CEE7B140B4868996CE484506109DEC605B@hoexchange1.fire.tas.gov.au>
	<AANLkTiktWu5e6C-VDWkRsQP6WkTxk7Dr4M+oxHn_xHKP@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902C8F208B@TMG-EVS02.torstar.net>

Use the health check page, not varnish itself. 

The health checks hit a page and checks if the webapp can build a page
within a reasonable time frame. 

Put something in it to notify you that it failed.

 

Stefan Caunter :: Senior Systems Administrator :: TOPS

e: scaunter at topscms.com  ::  m: (416) 561-4871

www.thestar.com www.topscms.com

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Guido
Bakker
Sent: November-23-10 1:33 AM
To: Glen Kelly
Cc: varnish-misc at varnish-cache.org
Subject: Re: Finding out number of healthy backends

 

We build a nagios check using:

varnishadm -T localhost:6082 debug.health | grep \ is

This works fine.

2010/11/22 Glen Kelly <Glen.Kelly at fire.tas.gov.au>

Is there any way of using varnishstat to determine the number of healthy
backends?

 

I currently use 'varnishstat -1 -f n_backend' but this only returns the
number of backends, not the number of healthy backends.

 

I am using Nagios to monitor our environment and want to be able to
monitor when we lose a backend. I already am monitoring the backends
directly so I have that covered but I want to also monitor when varnish
thinks it has lost a backend. i.e. monitoring at multiple levels

 

If this is not current achievable how do I go about requesting this
feature to be added to varnish?

 

Regards Glen

 

Error! Filename not specified.
<http://www.fire.tas.gov.au/Show?pageId=colDayLightSavings> 

P Do you need to print this? Consider the environment, prevent paper
waste. 

CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or
protected by legal professional privilege, and is intended only for the
person or persons to whom it is addressed. If you are not such a person,
you are warned that any disclosure, copying or dissemination of the
information is unauthorised. If you have received the transmission in
error, please immediately contact this office by telephone, fax or
email, to inform us of the error and to enable arrangements to be made
for the destruction of the transmission, or its return at our cost. No
liability is accepted for any unauthorised use of the information
contained in this transmission. 

 


_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101123/74126dba/attachment.html>

From pfreeman at entrepreneur.com  Wed Nov 24 00:46:55 2010
From: pfreeman at entrepreneur.com (pfreeman at entrepreneur.com)
Date: Tue, 23 Nov 2010 16:46:55 -0800 (PST)
Subject: configuration references piling up.
In-Reply-To: <1290466152.88039444@192.168.2.227>
References: <1290466152.88039444@192.168.2.227>
Message-ID: <1290559615.67781148@192.168.2.229>

On Monday, November 22, 2010 2:49pm, pfreeman at entrepreneur.com said:

> It seems that we have references to the configuration which are piling up ...
> where they are normally less than 80 but sometimes go high for a short period,
> since Friday, the reference count (as displayed by "varnishadm vcl.list") has
> continued to grow into the tens of thousands (if left alone).  The count goes to
> normal on a child stop/start, but grows rapidly.  We made some vcl changes, which
> we rolled back to no effect.  varnishlog produces so much output, I was wondering
> if anyone has a tip on how to begin troubleshooting this problem? (perhaps which
> tags to filter from varnishlog and what to look for?).
> 
> Thanks,
> 
> Patrick Freeman
> 


Turns out that there was a link out there which had been broken for some time and last week, the cache-control was bumped down from 3 days to 5 minutes (wasn't supposed to be 3 days).  In either case, it was an rss feed that was timing out, I guess ... munin didn't show apache connections lingering, so I'm not sure why varnish was holding these references open.  Anyway, fixing a rewrite in apache solved the error, which in turn solved the problem in varnish ... false alarm.




From russ at vshift.com  Wed Nov 24 05:00:28 2010
From: russ at vshift.com (Ruslan Sivak)
Date: Wed, 24 Nov 2010 00:00:28 -0500
Subject: Virtual hosts
Message-ID: <AANLkTin1_T-GiN1jXRo5R8VHzRF+M8xQro9DAP_xSUiL@mail.gmail.com>

Is there any sort of notion of virtual hosts in varnish?  How would I go
about separating the config for different sites, or does everything need to
go into one giant vcl file?

Russ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101124/e77933ee/attachment.html>

From varnish at mm.quex.org  Wed Nov 24 05:26:07 2010
From: varnish at mm.quex.org (Michael Alger)
Date: Wed, 24 Nov 2010 13:26:07 +0800
Subject: Virtual hosts
In-Reply-To: <AANLkTin1_T-GiN1jXRo5R8VHzRF+M8xQro9DAP_xSUiL@mail.gmail.com>
References: <AANLkTin1_T-GiN1jXRo5R8VHzRF+M8xQro9DAP_xSUiL@mail.gmail.com>
Message-ID: <20101124052607.GA4303@grum.quex.org>

On Wed, Nov 24, 2010 at 12:00:28AM -0500, Ruslan Sivak wrote:
> Is there any sort of notion of virtual hosts in varnish?  How would I
> go about separating the config for different sites, or does everything
> need to go into one giant vcl file?

Yes and no. As far as Varnish is concerned, it has one config file.
However you can use "include" statements in the main config file to
insert the contents of other files at any point in the configuration, so
you can split it up between as many separate files as you like.

For my configuration, I have /etc/varnish/main.vcl which provides the
overall processing. This includes a few other files. At the very start,
I pull in /etc/varnish/acls.vcl - which just provides a central place to
define any ACLs which are used anywhere else in the config.

I also pull in a file /etc/varnish/inc-backends-prod.vcl which itself
simply includes a bunch of other files, with each file defining one set
of backends and accompanying directors.

So, you can split up the config however it suits you to do so.

If you want to do IP-based virtual hosting, you'll likely want to set up
a file that looks something like:

  if (server.ip == "10.22.152.11")
  {
    include "/etc/varnish/site-something.com.vcl";
  }
  elsif (server.ip == "10.22.152.25")
  {
    include "/etc/varnish/site-somethingelse.com.vcl";
  }
  elsif (server.ip == "10.22.152.21")
  {
    include "/etc/varnish/site-anotherthing.com.vcl";
  }

For name-based virtual hosting, you'll probably want to first sanitise
the host header (remove or add ":80" suffix, remove or add "www."
prefix) and then include a site configuration based on the hostname
requested:

  if (! req.http.Host)
  {
    error 404 "Need a host header";
  }

  set req.http.Host = regsub(req.http.Host, "^www\.", "");
  set req.http.Host = regsub(req.http.Host, ":80$", "");

  if (req.http.Host = "something.com")
  {
    include "/etc/varnish/site-something.com.vcl";
  }
  elsif (req.http.Host = "somethingelse.com")
  {
    include "/etc/varnish/site-somethingelse.com.vcl";
  }
  ...

Alternatively you can use regular expressions to match the hostname, or
any other fields you want to use to select the configuration to use.

I actually use a mix, with each main website on a dedicated IP address,
and then another shared IP for vanity domain name redirects and that
kind of thing.

Too long? Use  include "/some/file.vcl";  to bring in an additional file
at any point in the configuration, and structure those files however you
like.



From Glen.Kelly at fire.tas.gov.au  Wed Nov 24 05:51:58 2010
From: Glen.Kelly at fire.tas.gov.au (Glen Kelly)
Date: Wed, 24 Nov 2010 16:51:58 +1100
Subject: Finding out number of healthy backends
In-Reply-To: <AANLkTiktWu5e6C-VDWkRsQP6WkTxk7Dr4M+oxHn_xHKP@mail.gmail.com>
References: <674B23B1CEE7B140B4868996CE484506109DEC605B@hoexchange1.fire.tas.gov.au>
	<AANLkTiktWu5e6C-VDWkRsQP6WkTxk7Dr4M+oxHn_xHKP@mail.gmail.com>
Message-ID: <674B23B1CEE7B140B4868996CE484506109DEC63C8@hoexchange1.fire.tas.gov.au>

A decision was made not to run the console, one less avenue for an attack. The servers are protected behind firewalls but it was thought there would be one less potential vulnerability.

Is there any other way of finding the number of healthy backends?

Regards Glen

From: Guido Bakker [mailto:guido.bakker at gmail.com]
Sent: Tuesday, 23 November 2010 5:33 PM
To: Glen Kelly
Cc: varnish-misc at varnish-cache.org
Subject: Re: Finding out number of healthy backends

We build a nagios check using:

varnishadm -T localhost:6082 debug.health | grep \ is

This works fine.
2010/11/22 Glen Kelly <Glen.Kelly at fire.tas.gov.au<mailto:Glen.Kelly at fire.tas.gov.au>>
Is there any way of using varnishstat to determine the number of healthy backends?

I currently use 'varnishstat -1 -f n_backend' but this only returns the number of backends, not the number of healthy backends.

I am using Nagios to monitor our environment and want to be able to monitor when we lose a backend. I already am monitoring the backends directly so I have that covered but I want to also monitor when varnish thinks it has lost a backend. i.e. monitoring at multiple levels

If this is not current achievable how do I go about requesting this feature to be added to varnish?

Regards Glen

Error! Filename not specified.<http://www.fire.tas.gov.au/Show?pageId=colDayLightSavings>

P Do you need to print this? Consider the environment, prevent paper waste.

CONFIDENTIALITY NOTICE AND DISCLAIMER
The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission.



_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101124/44adfb84/attachment.html>

From fangel at sevengoslings.net  Tue Nov 23 15:40:27 2010
From: fangel at sevengoslings.net (Morten Fangel)
Date: Tue, 23 Nov 2010 16:40:27 +0100
Subject: Very slow responses from backend-requests
Message-ID: <2332B073-B32A-4CC0-8A14-7CE1F28E9CF9@sevengoslings.net>

tfheen at varnish-software wrote:
> ]] Martin Schuhfuss 
> 
> | It seems that nearly any request that is passed on to the backend takes 
> | (more or less) exactly 15sec to complete. 
> | Anything coming from the cache is delivered in no time. 
> | There is, however, no reason I can see since the backend-processing takes 
> | only 82ms. 
> 
> I believe this is now fixed in the 2.1 branch in SVN. Testing would be 
> much appreciated. 
> 

I had the exact same error on the Debian 2.1.4-SVN package. I installed both trunk and 2.1-branch, and both of those worked fine.

So I agree, this is fixed in the 2.1-branch.

-Morten Fangel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101123/485394cf/attachment.html>

From robrecc at gmail.com  Thu Nov 25 17:33:34 2010
From: robrecc at gmail.com (Robert Recchia)
Date: Thu, 25 Nov 2010 12:33:34 -0500
Subject: varnish cacing
Message-ID: <AANLkTik8-ZwG6G7i0S_wVbCSH1SvjsKXxuOeHetX-8cQ@mail.gmail.com>

Hi everyone

I run varnish on my linux server at work - we use just a ip address
for the server 192.168.1.120.  We run wordpress and munin of this
server.  So i have varnish caching things

I want varnish to be able to cache http://192.168.1.120/wordpress/
but i do not want varnsih to cache http://192.168.1.120/munin/

Is this possible, becuase varnish is causing some problems with the
munin graphs where they are not getting updated becuase they are
cached by varnish.


-- 
Robert Recchia



From rtshilston at gmail.com  Thu Nov 25 17:44:11 2010
From: rtshilston at gmail.com (Robert Shilston)
Date: Thu, 25 Nov 2010 17:44:11 +0000
Subject: varnish cacing
In-Reply-To: <AANLkTik8-ZwG6G7i0S_wVbCSH1SvjsKXxuOeHetX-8cQ@mail.gmail.com>
References: <AANLkTik8-ZwG6G7i0S_wVbCSH1SvjsKXxuOeHetX-8cQ@mail.gmail.com>
Message-ID: <9BE06D68-60E4-4B67-94CE-F25D70995BBD@gmail.com>

Hi,

In vcl_recv, you can use something like:

        if (req.url ~ "munin") {
                pass;
        } 

Rob


On 25 Nov 2010, at 17:33, Robert Recchia wrote:

> Hi everyone
> 
> I run varnish on my linux server at work - we use just a ip address
> for the server 192.168.1.120.  We run wordpress and munin of this
> server.  So i have varnish caching things
> 
> I want varnish to be able to cache http://192.168.1.120/wordpress/
> but i do not want varnsih to cache http://192.168.1.120/munin/
> 
> Is this possible, becuase varnish is causing some problems with the
> munin graphs where they are not getting updated becuase they are
> cached by varnish.
> 
> 
> -- 
> Robert Recchia
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From lists at rtty.us  Thu Nov 25 20:11:00 2010
From: lists at rtty.us (Bob Camp)
Date: Thu, 25 Nov 2010 15:11:00 -0500
Subject: varnish cacing
In-Reply-To: <AANLkTik8-ZwG6G7i0S_wVbCSH1SvjsKXxuOeHetX-8cQ@mail.gmail.com>
References: <AANLkTik8-ZwG6G7i0S_wVbCSH1SvjsKXxuOeHetX-8cQ@mail.gmail.com>
Message-ID: <799B5650-02C1-4DE5-B226-2EFD84611303@rtty.us>

Hi

Within Wordpress you will want to avoid caching stuff under wp-admin. Same sort of URL trick to make that happen.

Bob


Sent from my iPhone

On Nov 25, 2010, at 12:33 PM, Robert Recchia <robrecc at gmail.com> wrote:

> Hi everyone
> 
> I run varnish on my linux server at work - we use just a ip address
> for the server 192.168.1.120.  We run wordpress and munin of this
> server.  So i have varnish caching things
> 
> I want varnish to be able to cache http://192.168.1.120/wordpress/
> but i do not want varnsih to cache http://192.168.1.120/munin/
> 
> Is this possible, becuase varnish is causing some problems with the
> munin graphs where they are not getting updated becuase they are
> cached by varnish.
> 
> 
> -- 
> Robert Recchia
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc



From indranilc at rediff-inc.com  Fri Nov 26 06:14:00 2010
From: indranilc at rediff-inc.com (Indranil Chakravorty)
Date: 26 Nov 2010 06:14:00 -0000
Subject: =?utf-8?B?VW5hYmxlIHRvIGdldCB2YXJuaXNoIHJ1bm5pbmcgaW4gcGVyc2lzdGVudCBjYWNoZSBtb2Rl?=
Message-ID: <20101126061400.5695.qmail@pro237-182.mxout.rediffmailpro.com>

Hi,&nbsp;&nbsp; There are many links which tend to point to the fact that varnish can be used to persist cache across restarts. However, I have not been able to get it working for me yet. I also saw few queries on the net which did not give a clear explanation or answer on how to get it working. &nbsp;I believe I need a trunk version of the source code which probably has the persistence implemented as an experiment? Please let me know if I can get varnish to work with a persistent cache and where to get the version which would have persistent mode implemented in it?&nbsp;
I have been trying to get varnish running on persistent mode using following options.-P /var/run/varnish.pid -a :80 -T *:6082 -f /etc/varnish/rediff.vcl -u varnish -g varnish -s persistent,/var/lib/varnish/varnish_storage.bin,500G
Thanks,Neel
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101126/c1e45cb1/attachment.html>

From perbu at varnish-software.com  Fri Nov 26 15:30:05 2010
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 26 Nov 2010 16:30:05 +0100
Subject: User survey
Message-ID: <AANLkTikErpBWCDMxwLaMa0zEax_74WNd1E5x_01uTdsB@mail.gmail.com>

Hi.

I just closed down the survey we started in the beginning of November. 132
people answered the survey. We got a lot of useful comments and some good
ideas. Thanks to everyone who participated.

I tried to sum up the results here:
http://www.varnish-software.com/blog/results-user-survey - if anyone wants
the whole data set let me know.

Have a good weekend, people.

Per.

-- 
Per Buer,  Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101126/16ce119a/attachment.html>

From A.Hongens at netmatch.nl  Fri Nov 26 15:41:57 2010
From: A.Hongens at netmatch.nl (=?iso-8859-1?Q?Angelo_H=F6ngens?=)
Date: Fri, 26 Nov 2010 15:41:57 +0000
Subject: file vs malloc
Message-ID: <6A7ABA19243F1E4EADD8BB1563CDDCCB11C12D@TIL-EXCH-05.netmatch.local>

Per wrote in his survey result: "62% use -s file, which I find somewhat surprising. -s file doesn't really perform that well under pressure on rotating hard drives due to some changes in recent Linux and FreeBSD kernels."

But does anyone have a link to (or write up) some explanation for dummies on cache file types? I know there's 'file' and 'malloc', but I don't know the difference or the pros and cons.

I tried searching the wiki, but I my google-fu lacks sometimes ;)

-- 

 
With kind regards,
 
 
Angelo H?ngens
 
Systems Administrator
 
------------------------------------------
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:A.Hongens at netmatch.nl
http://www.netmatch.nl
------------------------------------------





From perbu at varnish-software.com  Fri Nov 26 19:39:27 2010
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 26 Nov 2010 20:39:27 +0100
Subject: file vs malloc
In-Reply-To: <6A7ABA19243F1E4EADD8BB1563CDDCCB11C12D@TIL-EXCH-05.netmatch.local>
References: <AcuNf/q2DuDBQ0VyR0qZXXvshj7GCA==>
	<6A7ABA19243F1E4EADD8BB1563CDDCCB11C12D@TIL-EXCH-05.netmatch.local>
Message-ID: <AANLkTimHObYP9AYAQ-gyzRybCSFT0XS2tOqi7TBFDF4X@mail.gmail.com>

On Fri, Nov 26, 2010 at 4:41 PM, Angelo H?ngens <A.Hongens at netmatch.nl>wrote:

> Per wrote in his survey result: "62% use -s file, which I find somewhat
> surprising. -s file doesn't really perform that well under pressure on
> rotating hard drives due to some changes in recent Linux and FreeBSD
> kernels."
>
> But does anyone have a link to (or write up) some explanation for dummies
> on cache file types? I know there's 'file' and 'malloc', but I don't know
> the difference or the pros and cons.
>

In Varnish 1.0 we had -s file and everything was fine. People where using
Varnish on  Linux 2.6.9 (and thereabout) and FreeBSD6. Then came 2.6.18 with
RHEL5 and some other Linux distros and Varnish started behaving a bit
strange. The IO pressure had increased and we saw load skyrocket with
Varnish being stuck IO sleep a lot of the time, waiting for writes to
finish. Then FreeBSD7 came out and showed the same behavior. Obviously the
kernel developers of the world where ganging up on us.

Alterations in vm behavior are difficult things to track down and the
chances of changing the behavior back might be slim - Varnish being
something of an oddball doing crazy stuff like mmaping several gigabytes and
accessing it in a random fasion (and most of them saying things like "Wow,
that actually works?" when hearing what we do).

Anyway, Poul came up with -s malloc, which just allocates slabs of ordinary
memory and stores everything there. Since ordinary memory isn't backed by
disk a write to memory doesn't trigger a write to disk so there are no sync
disk writes to slow us down. On the flip side - at least on Linux, if your
data set doesn't dit in memory - the moment you start to actively use your
swapping space things grind to a halt. FreeBSD seems to cope better with
programs being larger then the amount of memory.

Several people are running high traffic sites with -s file backed by SSD.
That seem to perform very well on Linux. Our main FreeBSD user, Anders N,
uses -s malloc so I don't know much about how -s file on FreeBSD anymore.

So, to sum up:
 - If you have SSD drives you can mostly do what you want. I don't know of
anyone running huge varnish process that swap alot - but it _might_ work. If
you are, let me know.
 - If your varnish process doesn't write that much you're probably in the
clear no matter what you'll do.
 - If you have a small data set that fits neatly in memory, use -s malloc.
 - If you you have a large data set that doesn't fit in memory you have to
get SSD drives.

Oh, and at least on Linux there are a bunch of tunables that might alter or
even rectify some of the misbehavior we've seen.


I should clean this up, get some tests done and write it up somewhere.
-- 
Per Buer,  Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101126/283de4f7/attachment.html>

From wido at widodh.nl  Fri Nov 26 21:47:45 2010
From: wido at widodh.nl (Wido den Hollander)
Date: Fri, 26 Nov 2010 22:47:45 +0100
Subject: file vs malloc
In-Reply-To: <AANLkTimHObYP9AYAQ-gyzRybCSFT0XS2tOqi7TBFDF4X@mail.gmail.com>
References: <AcuNf/q2DuDBQ0VyR0qZXXvshj7GCA==>
	<6A7ABA19243F1E4EADD8BB1563CDDCCB11C12D@TIL-EXCH-05.netmatch.local>
	<AANLkTimHObYP9AYAQ-gyzRybCSFT0XS2tOqi7TBFDF4X@mail.gmail.com>
Message-ID: <1290808065.1457.1.camel@wido-laptop.pcextreme.nl>

Hi,

> - If you have SSD drives you can mostly do what you want. I don't know
> of anyone running huge varnish process that swap alot - but it _might_
> work. If you are, let me know.

Just wanted to add, there is someone running with a lot of storage with
malloc/swap:

* http://varnish-cache.org/trac/ticket/784

Wido

On Fri, 2010-11-26 at 20:39 +0100, Per Buer wrote:
> 
> 
> On Fri, Nov 26, 2010 at 4:41 PM, Angelo H?ngens
> <A.Hongens at netmatch.nl> wrote:
>         Per wrote in his survey result: "62% use -s file, which I find
>         somewhat surprising. -s file doesn't really perform that well
>         under pressure on rotating hard drives due to some changes in
>         recent Linux and FreeBSD kernels."
>         
>         But does anyone have a link to (or write up) some explanation
>         for dummies on cache file types? I know there's 'file' and
>         'malloc', but I don't know the difference or the pros and
>         cons.
> 
> 
> In Varnish 1.0 we had -s file and everything was fine. People where
> using Varnish on  Linux 2.6.9 (and thereabout) and FreeBSD6. Then came
> 2.6.18 with RHEL5 and some other Linux distros and Varnish started
> behaving a bit strange. The IO pressure had increased and we saw load
> skyrocket with Varnish being stuck IO sleep a lot of the time, waiting
> for writes to finish. Then FreeBSD7 came out and showed the
> same behavior. Obviously the kernel developers of the world where
> ganging up on us.
> 
> 
> Alterations in vm behavior are difficult things to track down and the
> chances of changing the behavior back might be slim - Varnish being
> something of an oddball doing crazy stuff like mmaping several
> gigabytes and accessing it in a random fasion (and most of them saying
> things like "Wow, that actually works?" when hearing what we do). 
> 
> 
> Anyway, Poul came up with -s malloc, which just allocates slabs of
> ordinary memory and stores everything there. Since ordinary memory
> isn't backed by disk a write to memory doesn't trigger a write to disk
> so there are no sync disk writes to slow us down. On the flip side -
> at least on Linux, if your data set doesn't dit in memory - the moment
> you start to actively use your swapping space things grind to a halt.
> FreeBSD seems to cope better with programs being larger then the
> amount of memory.
> 
> 
> Several people are running high traffic sites with -s file backed by
> SSD. That seem to perform very well on Linux. Our main FreeBSD user,
> Anders N, uses -s malloc so I don't know much about how -s file on
> FreeBSD anymore.
> 
> 
> So, to sum up:
>  - If you have SSD drives you can mostly do what you want. I don't
> know of anyone running huge varnish process that swap alot - but it
> _might_ work. If you are, let me know.
>  - If your varnish process doesn't write that much you're probably in
> the clear no matter what you'll do.
>  - If you have a small data set that fits neatly in memory, use -s
> malloc.
>  - If you you have a large data set that doesn't fit in memory you
> have to get SSD drives.
> 
> 
> Oh, and at least on Linux there are a bunch of tunables that might
> alter or even rectify some of the misbehavior we've seen. 
> 
> 
> 
> 
> I should clean this up, get some tests done and write it up
> somewhere. 
> -- 
> Per Buer,  Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From checker at d6.com  Sat Nov 27 08:33:42 2010
From: checker at d6.com (Chris Hecker)
Date: Sat, 27 Nov 2010 00:33:42 -0800
Subject: file vs malloc
In-Reply-To: <6A7ABA19243F1E4EADD8BB1563CDDCCB11C12D@TIL-EXCH-05.netmatch.local>
References: <6A7ABA19243F1E4EADD8BB1563CDDCCB11C12D@TIL-EXCH-05.netmatch.local>
Message-ID: <4CF0C266.5050000@d6.com>


My guess on "-s file" is that a lot of users might be like me, which is 
to say I'm using varnish to take the load off httpd for a few vhost 
websites on a single wimpy machine, but I don't have insane traffic or 
anything, it's just that varnish is sooo much more efficient than httpd 
that it still makes sense.

Not sure if the survey could separate out the user types and correlate 
the cache type with those types, but it might be interesting.

Chris



On 2010/11/26 07:41, Angelo H?ngens wrote:
> Per wrote in his survey result: "62% use -s file, which I find somewhat surprising. -s file doesn't really perform that well under pressure on rotating hard drives due to some changes in recent Linux and FreeBSD kernels."
>
> But does anyone have a link to (or write up) some explanation for dummies on cache file types? I know there's 'file' and 'malloc', but I don't know the difference or the pros and cons.
>
> I tried searching the wiki, but I my google-fu lacks sometimes ;)
>



From lists at rtty.us  Sat Nov 27 13:37:53 2010
From: lists at rtty.us (Bob Camp)
Date: Sat, 27 Nov 2010 08:37:53 -0500
Subject: file vs malloc
In-Reply-To: <4CF0C266.5050000@d6.com>
References: <6A7ABA19243F1E4EADD8BB1563CDDCCB11C12D@TIL-EXCH-05.netmatch.local>
	<4CF0C266.5050000@d6.com>
Message-ID: <34CDEEB8-6787-4518-99E2-F7CA3BD5A02C@rtty.us>

Hi

Given that most VPS setups give you a lot of disk and relatively little RAM, "-s file" makes a lot of sense. Varnish is more efficient than HTTPD. VCL gives you a layer of control in front of the server to do stuff that otherwise can be tough to layer. All of that is a good thing on a cramped little server setup. 

Bob

On Nov 27, 2010, at 3:33 AM, Chris Hecker wrote:

> 
> My guess on "-s file" is that a lot of users might be like me, which is to say I'm using varnish to take the load off httpd for a few vhost websites on a single wimpy machine, but I don't have insane traffic or anything, it's just that varnish is sooo much more efficient than httpd that it still makes sense.
> 
> Not sure if the survey could separate out the user types and correlate the cache type with those types, but it might be interesting.
> 
> Chris
> 
> 
> 
> On 2010/11/26 07:41, Angelo H?ngens wrote:
>> Per wrote in his survey result: "62% use -s file, which I find somewhat surprising. -s file doesn't really perform that well under pressure on rotating hard drives due to some changes in recent Linux and FreeBSD kernels."
>> 
>> But does anyone have a link to (or write up) some explanation for dummies on cache file types? I know there's 'file' and 'malloc', but I don't know the difference or the pros and cons.
>> 
>> I tried searching the wiki, but I my google-fu lacks sometimes ;)
>> 
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From baldur.norddahl at gmail.com  Sun Nov 28 09:56:18 2010
From: baldur.norddahl at gmail.com (Baldur Norddahl)
Date: Sun, 28 Nov 2010 10:56:18 +0100
Subject: timeout problem with Play Framework
Message-ID: <AANLkTi=7zcTP98rUcnkE_bz7veBWVfD4AKFUYu7qZ11V@mail.gmail.com>

Hello

I am using Varnish-cache in front of a website based on the Play Framework (
http://www.playframework.org/ - a java web framework). I started out with a
really simple varnish config file:

backend default {
    .host = "127.0.0.1";
    .port = "9010";
}

backend joytools {
        .host = "127.0.0.1";
        .port = "9020";
}

sub vcl_recv {
    if (req.http.host ~ "^(www.)?(joytools.dk|xn--bkkenbund-g3a.dk)$") {
        set req.backend = joytools;
    } else {
        set req.backend = default;
    }
}

The purpose is to sort out requests to a HTTP virtual host to one backend
server and all other requests to another. This works.

However, on the first request or when doing a "refresh" in a browser the
site becomes extremely slow and unresponsive. I did some work with tcpdump
to find out what is going on. I captured this:

--REQUEST--
GET /public/javascripts/jquery-1.4.2.min.js HTTP/1.1
Host: xn--bkkenbund-g3a.dk
Referer: http://xn--bkkenbund-g3a.dk/
If-Modified-Since: Thu, 11 Mar 2010 09:57:53 GMT
Accept: */*
If-None-Match: "1268301473000--44924593"
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.7
(KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7
Accept-Encoding: gzip,deflate,sdch
Accept-Language: da,en-US;q=0.8,en;q=0.6
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: PLAY_FLASH=; PLAY_ERRORS=;
PLAY_SESSION=9df837ef975871bcd0e1fc61189dd457420accaf-%00___ID%3A327ad683-323e-4d25-a824-347ad85bc5df%00
X-Forwarded-For: 79.98.193.196
X-Varnish: 287151663

--REQUEST END--
--RESPONSE--
HTTP/1.1 304 Not Modified
Server: Play! Framework;1.1;dev
Cache-Control: no-cache
ETag: "1268301473000--44924593"

--RESPONSE END--

After this exchange between Play Framework and Varnish the connection is
idle for exactly 60 seconds before closing. It looks to me as if Play thinks
it is done replying but Varnish is expecting something more.

I hope an expert here can give me a quick pointer to how I proceed resolving
this issue.

Thanks,

Baldur
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101128/5b84e73d/attachment.html>

From baldur.norddahl at gmail.com  Sun Nov 28 11:55:51 2010
From: baldur.norddahl at gmail.com (Baldur Norddahl)
Date: Sun, 28 Nov 2010 12:55:51 +0100
Subject: timeout problem with Play Framework
In-Reply-To: <AANLkTikOZ40nscfNj=+YDPfnEVwk7MyFCtzm1yGYApqT@mail.gmail.com>
References: <AANLkTi=7zcTP98rUcnkE_bz7veBWVfD4AKFUYu7qZ11V@mail.gmail.com>
	<AANLkTikOZ40nscfNj=+YDPfnEVwk7MyFCtzm1yGYApqT@mail.gmail.com>
Message-ID: <AANLkTikT5WT1brTboHWPkSZ1MbEWnZoHz3YbXKZoy2pN@mail.gmail.com>

Hi,

Downgrading from Varnish 2.1.4 to 2.1.3 fixes the problem. Thanks for the
help.

I am not using Apache. It is just varnish in front of the play framework,
which has a build in webserver.

Regards,

Baldur

On Sun, Nov 28, 2010 at 11:56 AM, FabD [ML] <flylm.ml at gmail.com> wrote:

> 2010/11/28 Baldur Norddahl <baldur.norddahl at gmail.com>:
> > Hello
>
> Hi,
>
> Do you use Varnish 2.1.4 with Apache2 ? I got the same issue with the
> Debian package of the Varnish repository.
>
> 3 possibilities  :
>
> - Compile by hand the 2.1 branch of the SVN
> - Use Varnish 2.1.3 and wait the next Varnish version.
> - Test with Apache2 KeepAlive Off and wait the next Varnish version.
>
> Fabien
>
> > I am using Varnish-cache in front of a website based on the Play
> Framework
> > (http://www.playframework.org/ - a java web framework). I started out
> with a
> > really simple varnish config file:
> > backend default {
> >     .host = "127.0.0.1";
> >     .port = "9010";
> > }
> > backend joytools {
> >         .host = "127.0.0.1";
> >         .port = "9020";
> > }
> > sub vcl_recv {
> >     if (req.http.host ~ "^(www.)?(joytools.dk|xn--bkkenbund-g3a.dk)$") {
> >         set req.backend = joytools;
> >     } else {
> >         set req.backend = default;
> >     }
> > }
> > The purpose is to sort out requests to a HTTP virtual host to one backend
> > server and all other requests to another. This works.
> > However, on the first request or when doing a "refresh" in a browser the
> > site becomes extremely slow and unresponsive. I did some work with
> tcpdump
> > to find out what is going on. I captured this:
> > --REQUEST--
> > GET /public/javascripts/jquery-1.4.2.min.js HTTP/1.1
> > Host: xn--bkkenbund-g3a.dk
> > Referer: http://xn--bkkenbund-g3a.dk/
> > If-Modified-Since: Thu, 11 Mar 2010 09:57:53 GMT
> > Accept: */*
> > If-None-Match: "1268301473000--44924593"
> > User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.7
> > (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7
> > Accept-Encoding: gzip,deflate,sdch
> > Accept-Language: da,en-US;q=0.8,en;q=0.6
> > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
> > Cookie: PLAY_FLASH=; PLAY_ERRORS=;
> >
> PLAY_SESSION=9df837ef975871bcd0e1fc61189dd457420accaf-%00___ID%3A327ad683-323e-4d25-a824-347ad85bc5df%00
> > X-Forwarded-For: 79.98.193.196
> > X-Varnish: 287151663
> > --REQUEST END--
> > --RESPONSE--
> > HTTP/1.1 304 Not Modified
> > Server: Play! Framework;1.1;dev
> > Cache-Control: no-cache
> > ETag: "1268301473000--44924593"
> > --RESPONSE END--
> > After this exchange between Play Framework and Varnish the connection is
> > idle for exactly 60 seconds before closing. It looks to me as if Play
> thinks
> > it is done replying but Varnish is expecting something more.
> > I hope an expert here can give me a quick pointer to how I proceed
> resolving
> > this issue.
> > Thanks,
> > Baldur
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101128/ee69fc5b/attachment.html>

From perbu at varnish-software.com  Sun Nov 28 12:56:57 2010
From: perbu at varnish-software.com (Per Buer)
Date: Sun, 28 Nov 2010 13:56:57 +0100
Subject: timeout problem with Play Framework
In-Reply-To: <AANLkTikT5WT1brTboHWPkSZ1MbEWnZoHz3YbXKZoy2pN@mail.gmail.com>
References: <AANLkTi=7zcTP98rUcnkE_bz7veBWVfD4AKFUYu7qZ11V@mail.gmail.com>
	<AANLkTikOZ40nscfNj=+YDPfnEVwk7MyFCtzm1yGYApqT@mail.gmail.com>
	<AANLkTikT5WT1brTboHWPkSZ1MbEWnZoHz3YbXKZoy2pN@mail.gmail.com>
Message-ID: <AANLkTik5QWtR9MKETEBju0mrFB8GNAZz6USA6t=gmk6r@mail.gmail.com>

Hi.

You can also add the following to your VCL to work around the bug in 2.1.4:

sub vcl_pass {
    unset bereq.http.if-modified-since;
    unset bereq.http.if-none-match;
    return (pass);
}

The problem is that Varnish expects a body when backend issues a 304 and
keeps waiting until timeout.

On Sun, Nov 28, 2010 at 12:55 PM, Baldur Norddahl <baldur.norddahl at gmail.com
> wrote:

> Hi,
>
> Downgrading from Varnish 2.1.4 to 2.1.3 fixes the problem. Thanks for the
> help.
>
> I am not using Apache. It is just varnish in front of the play framework,
> which has a build in webserver.
>
> Regards,
>
> Baldur
>
> On Sun, Nov 28, 2010 at 11:56 AM, FabD [ML] <flylm.ml at gmail.com> wrote:
>
>> 2010/11/28 Baldur Norddahl <baldur.norddahl at gmail.com>:
>> > Hello
>>
>> Hi,
>>
>> Do you use Varnish 2.1.4 with Apache2 ? I got the same issue with the
>> Debian package of the Varnish repository.
>>
>> 3 possibilities  :
>>
>> - Compile by hand the 2.1 branch of the SVN
>> - Use Varnish 2.1.3 and wait the next Varnish version.
>> - Test with Apache2 KeepAlive Off and wait the next Varnish version.
>>
>> Fabien
>>
>>
>> > I am using Varnish-cache in front of a website based on the Play
>> Framework
>> > (http://www.playframework.org/ - a java web framework). I started out
>> with a
>> > really simple varnish config file:
>> > backend default {
>> >     .host = "127.0.0.1";
>> >     .port = "9010";
>> > }
>> > backend joytools {
>> >         .host = "127.0.0.1";
>> >         .port = "9020";
>> > }
>> > sub vcl_recv {
>> >     if (req.http.host ~ "^(www.)?(joytools.dk|xn--bkkenbund-g3a.dk)$")
>> {
>> >         set req.backend = joytools;
>> >     } else {
>> >         set req.backend = default;
>> >     }
>> > }
>> > The purpose is to sort out requests to a HTTP virtual host to one
>> backend
>> > server and all other requests to another. This works.
>> > However, on the first request or when doing a "refresh" in a browser the
>> > site becomes extremely slow and unresponsive. I did some work with
>> tcpdump
>> > to find out what is going on. I captured this:
>> > --REQUEST--
>> > GET /public/javascripts/jquery-1.4.2.min.js HTTP/1.1
>> > Host: xn--bkkenbund-g3a.dk
>> > Referer: http://xn--bkkenbund-g3a.dk/
>> > If-Modified-Since: Thu, 11 Mar 2010 09:57:53 GMT
>> > Accept: */*
>> > If-None-Match: "1268301473000--44924593"
>> > User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.7
>> > (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7
>> > Accept-Encoding: gzip,deflate,sdch
>> > Accept-Language: da,en-US;q=0.8,en;q=0.6
>> > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
>> > Cookie: PLAY_FLASH=; PLAY_ERRORS=;
>> >
>> PLAY_SESSION=9df837ef975871bcd0e1fc61189dd457420accaf-%00___ID%3A327ad683-323e-4d25-a824-347ad85bc5df%00
>> > X-Forwarded-For: 79.98.193.196
>> > X-Varnish: 287151663
>> > --REQUEST END--
>> > --RESPONSE--
>> > HTTP/1.1 304 Not Modified
>> > Server: Play! Framework;1.1;dev
>> > Cache-Control: no-cache
>> > ETag: "1268301473000--44924593"
>> > --RESPONSE END--
>> > After this exchange between Play Framework and Varnish the connection is
>> > idle for exactly 60 seconds before closing. It looks to me as if Play
>> thinks
>> > it is done replying but Varnish is expecting something more.
>> > I hope an expert here can give me a quick pointer to how I proceed
>> resolving
>> > this issue.
>> > Thanks,
>> > Baldur
>> > _______________________________________________
>> > varnish-misc mailing list
>> > varnish-misc at varnish-cache.org
>> > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>> >
>>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



-- 
Per Buer,  Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101128/e6209ef4/attachment.html>

From indranilc at rediff-inc.com  Mon Nov 29 11:07:06 2010
From: indranilc at rediff-inc.com (Indranil Chakravorty)
Date: 29 Nov 2010 11:07:06 -0000
Subject: =?utf-8?B?UmU6IFVuYWJsZSB0byBnZXQgdmFybmlzaCBydW5uaW5nIGluIHBlcnNpc3RlbnQgY2FjaGUgbW9kZQ==?=
Message-ID: <1290752041.S.3591.5725.pro-237-182.rediffmailpro.com.old.1291028826.51286@webmail.rediffmail.com>

Hi All,&nbsp;&nbsp;&nbsp; Could anyone please give me some pointers about the same? Sorry for spamming.

Thanks,
Neel


On Fri, 26 Nov 2010 11:44:01 +0530 "Indranil Chakravorty" wrote

&gt;Hi, There are many links which tend to point to the fact that varnish can be used to persist cache across restarts. However, I have not been able to get it working for me yet. I also saw few queries on the net which did not give a clear explanation or answer on how to get it working. I believe I need a trunk version of the source code which probably has the persistence implemented as an experiment? Please let me know if I can get varnish to work with a persistent cache and where to get the version which would have persistent mode implemented in it?

&gt;I have been trying to get varnish running on persistent mode using following options.-P /var/run/varnish.pid -a :80 -T *:6082 -f /etc/varnish/rediff.vcl -u varnish -g varnish -s persistent,/var/lib/varnish/varnish_storage.bin,500G

&gt;Thanks,Neel

&gt; 

&gt; 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101129/f0f6bdf4/attachment.html>

From Klapproth.M at sportresult.com  Mon Nov 29 14:01:58 2010
From: Klapproth.M at sportresult.com (Klapproth, Martin)
Date: Mon, 29 Nov 2010 15:01:58 +0100
Subject: Send old content if backend is down
Message-ID: <4BC50911656FBB49858C6ADB1B8D223F038D96C3@mail.wigemic.lpz>

Hello,

 

I have a general question relating to varnish's vcl configuration. I
want the varnish server to send old cached objects if the backend is
down (marked unhealthy). I have tested various grace configurations but
nothing worked, because grace takes only effect if the backend is slow,
but not if it's down. So is there a way (maybe in vcl_error)?

 

Thanks,

 

Martin

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101129/dba10d00/attachment.html>

From perbu at varnish-software.com  Mon Nov 29 14:09:42 2010
From: perbu at varnish-software.com (Per Buer)
Date: Mon, 29 Nov 2010 15:09:42 +0100
Subject: Send old content if backend is down
In-Reply-To: <4BC50911656FBB49858C6ADB1B8D223F038D96C3@mail.wigemic.lpz>
References: <4BC50911656FBB49858C6ADB1B8D223F038D96C3@mail.wigemic.lpz>
Message-ID: <AANLkTimLP=LRuG8_sCSDc=AgDgTTk-kN+rGe_DQtO_vN@mail.gmail.com>

On Mon, Nov 29, 2010 at 3:01 PM, Klapproth, Martin <
Klapproth.M at sportresult.com> wrote:

>  Hello,
>
>
>
> I have a general question relating to varnish?s vcl configuration. I want
> the varnish server to send old cached objects if the backend is down (marked
> unhealthy). I have tested various grace configurations but nothing worked,
> because grace takes only effect if the backend is slow, but not if it?s
> down. So is there a way (maybe in vcl_error)?
>

Yes. There is a way.

http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html#saint-mode

-- 
Per Buer,  Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101129/48dd62ae/attachment.html>

From stewsnooze at gmail.com  Mon Nov 29 14:09:55 2010
From: stewsnooze at gmail.com (Stewart Robinson)
Date: Mon, 29 Nov 2010 14:09:55 +0000
Subject: Send old content if backend is down
In-Reply-To: <4BC50911656FBB49858C6ADB1B8D223F038D96C3@mail.wigemic.lpz>
References: <4BC50911656FBB49858C6ADB1B8D223F038D96C3@mail.wigemic.lpz>
Message-ID: <AANLkTi=sf2n52cd=fwUunXd9zpHcsz0ZVFHOFjpDegFz@mail.gmail.com>

HI,

It depends on how down your site is. You might benefit from saint mode
http://www.varnish-cache.org/trac/wiki/VCLExampleSaintMode

Stew

On 29 November 2010 14:01, Klapproth, Martin <Klapproth.M at sportresult.com>wrote:

>  Hello,
>
>
>
> I have a general question relating to varnish?s vcl configuration. I want
> the varnish server to send old cached objects if the backend is down (marked
> unhealthy). I have tested various grace configurations but nothing worked,
> because grace takes only effect if the backend is slow, but not if it?s
> down. So is there a way (maybe in vcl_error)?
>
>
>
> Thanks,
>
>
>
> Martin
>
>
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101129/e7016543/attachment.html>

From vburshteyn at broadway.com  Mon Nov 29 18:26:46 2010
From: vburshteyn at broadway.com (Vitaly Burshteyn)
Date: Mon, 29 Nov 2010 13:26:46 -0500
Subject: custom error pages
Message-ID: <0F69574F9901D4459C6B75C9FF64FBC40522A9B2@mxfl01.hollywoodmedia.corp>

Hi,

 

I am trying to set up my VCL to deliver a custom static page in case of
a 503 or another error.

 

Can anybody point me in the right direction?

 

Vitaly Burshteyn

Senior Network Engineer

Broadway.com, Theatre Direct International

729  7th Avenue

New York, New York 10019

Phone: 212.817.9117

Cell# 917-701-5732

 


____________________________________
The information contained in this transmission may contain privileged 
and confidential information.  It is intended only for the use of the 
person(s) named above. If you are not the intended recipient,  you are 
hereby notified that any review, dissemination, distribution or 
duplication of this communication is strictly prohibited. If you are 
not the intended recipient, please contact the sender by reply email 
and destroy all copies of the original message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101129/cf9e338d/attachment.html>

From bedis9 at gmail.com  Mon Nov 29 18:30:15 2010
From: bedis9 at gmail.com (Bedis 9)
Date: Mon, 29 Nov 2010 19:30:15 +0100
Subject: custom error pages
In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC40522A9B2@mxfl01.hollywoodmedia.corp>
References: <0F69574F9901D4459C6B75C9FF64FBC40522A9B2@mxfl01.hollywoodmedia.corp>
Message-ID: <AANLkTinDvWg_OTGoovYxNVGo43AcOOM-d2fEwFQ_PyZv@mail.gmail.com>

Hi,

Have a look to vcl_error.

cheers

On Mon, Nov 29, 2010 at 7:26 PM, Vitaly Burshteyn
<vburshteyn at broadway.com> wrote:
> Hi,
>
>
>
> I am trying to set up my VCL to deliver a custom static page in case of a
> 503 or another error.
>
>
>
> Can anybody point me in the right direction?
>
>
>
> Vitaly Burshteyn
>
> Senior Network Engineer
>
> Broadway.com, Theatre Direct International
>
> 729? 7th Avenue
>
> New York, New York 10019
>
> Phone: 212.817.9117
>
> Cell# 917-701-5732
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>



From scaunter at topscms.com  Mon Nov 29 18:30:15 2010
From: scaunter at topscms.com (Caunter, Stefan)
Date: Mon, 29 Nov 2010 13:30:15 -0500
Subject: custom error pages
In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC40522A9B2@mxfl01.hollywoodmedia.corp>
References: <0F69574F9901D4459C6B75C9FF64FBC40522A9B2@mxfl01.hollywoodmedia.corp>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902C9B5E6A@TMG-EVS02.torstar.net>

You mean a fail page?

 

http://caunter.ca/?p=124

 

(You probably want to redirect to another domain for failure)

 

Stefan Caunter :: Senior Systems Administrator :: TOPS

e: scaunter at topscms.com  ::  m: (416) 561-4871

www.thestar.com www.topscms.com

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Vitaly
Burshteyn
Sent: November-29-10 1:27 PM
To: varnish-misc at varnish-cache.org
Subject: custom error pages

 

Hi,

 

I am trying to set up my VCL to deliver a custom static page in case of
a 503 or another error.

 

Can anybody point me in the right direction?

 

Vitaly Burshteyn

Senior Network Engineer

Broadway.com, Theatre Direct International

729  7th Avenue

New York, New York 10019

Phone: 212.817.9117

Cell# 917-701-5732

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101129/67322c4f/attachment.html>

From Klapproth.M at sportresult.com  Tue Nov 30 07:16:12 2010
From: Klapproth.M at sportresult.com (Klapproth, Martin)
Date: Tue, 30 Nov 2010 08:16:12 +0100
Subject: Send old content if backend is down
In-Reply-To: <AANLkTimLP=LRuG8_sCSDc=AgDgTTk-kN+rGe_DQtO_vN@mail.gmail.com>
Message-ID: <4BC50911656FBB49858C6ADB1B8D223F038D96C4@mail.wigemic.lpz>

Thank you. This option helped a lot. But varnish delivers still 503
errors when the backend is completely down (no route to host, connection
refused). Is there a way to handle such situations too?

 

________________________________

From: Per Buer [mailto:perbu at varnish-software.com] 
Sent: Montag, 29. November 2010 15:10
To: Klapproth, Martin
Cc: varnish-misc at varnish-cache.org
Subject: Re: Send old content if backend is down

 

On Mon, Nov 29, 2010 at 3:01 PM, Klapproth, Martin
<Klapproth.M at sportresult.com> wrote:

	Hello,

	 

	I have a general question relating to varnish's vcl
configuration. I want the varnish server to send old cached objects if
the backend is down (marked unhealthy). I have tested various grace
configurations but nothing worked, because grace takes only effect if
the backend is slow, but not if it's down. So is there a way (maybe in
vcl_error)?

 

Yes. There is a way.

 

http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_se
rvers.html#saint-mode


-- 
Per Buer,  Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101130/98d62c41/attachment.html>

From wido at widodh.nl  Tue Nov 30 07:44:29 2010
From: wido at widodh.nl (Wido den Hollander)
Date: Tue, 30 Nov 2010 08:44:29 +0100
Subject: Send old content if backend is down
In-Reply-To: <4BC50911656FBB49858C6ADB1B8D223F038D96C4@mail.wigemic.lpz>
References: <4BC50911656FBB49858C6ADB1B8D223F038D96C4@mail.wigemic.lpz>
Message-ID: <1291103069.12922.13.camel@wido-laptop.pcextreme.nl>

Hi,

Just to be sure, are you doing health checks on your backend?

Wido

On Tue, 2010-11-30 at 08:16 +0100, Klapproth, Martin wrote:
> Thank you. This option helped a lot. But varnish delivers still 503
> errors when the backend is completely down (no route to host,
> connection refused). Is there a way to handle such situations too?
> 
>  
> 
>                                    
> ______________________________________________________________________
> From:Per Buer [mailto:perbu at varnish-software.com] 
> Sent: Montag, 29. November 2010 15:10
> To: Klapproth, Martin
> Cc: varnish-misc at varnish-cache.org
> Subject: Re: Send old content if backend is down
> 
> 
>  
> 
> On Mon, Nov 29, 2010 at 3:01 PM, Klapproth, Martin
> <Klapproth.M at sportresult.com> wrote:
> 
>         Hello,
>         
>          
>         
>         I have a general question relating to varnish?s vcl
>         configuration. I want the varnish server to send old cached
>         objects if the backend is down (marked unhealthy). I have
>         tested various grace configurations but nothing worked,
>         because grace takes only effect if the backend is slow, but
>         not if it?s down. So is there a way (maybe in vcl_error)?
>         
>         
>  
> 
> 
> Yes. There is a way.
> 
> 
>  
> 
> 
> http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html#saint-mode
> 
> 
> 
> -- 
> Per Buer,  Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
> 
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From andrewgbailey at gmail.com  Mon Nov 29 21:48:46 2010
From: andrewgbailey at gmail.com (Andy Bailey)
Date: Mon, 29 Nov 2010 21:48:46 +0000
Subject: pass user IP to file
Message-ID: <AANLkTi=5fcf0pCMiukiJPwjG85+0ex-OS=kUC7=W-fDg@mail.gmail.com>

I have a file on my server that checks a user hasn't accessed it more than 5
times in the past 5 minutes by saving the IP address of every request. Since
having varnish installed, the file is denying access to everyone because for
some reason the IP address that is being stored is always my server IP. How
would I make sure that requests to that file pass the users IP address?

hope you can help?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101129/8ac4ad36/attachment.html>

From Klapproth.M at sportresult.com  Tue Nov 30 07:59:49 2010
From: Klapproth.M at sportresult.com (Klapproth, Martin)
Date: Tue, 30 Nov 2010 08:59:49 +0100
Subject: Send old content if backend is down
Message-ID: <4BC50911656FBB49858C6ADB1B8D223F038D96C6@mail.wigemic.lpz>

Yes, I'm polling every 10 seconds. When the backend is down, the hosts
are marked as unhealthy in varnishlog.

-----Original Message-----
From: Wido den Hollander [mailto:wido at widodh.nl] 
Sent: Dienstag, 30. November 2010 08:44
To: Klapproth, Martin
Cc: varnish-misc at varnish-cache.org
Subject: RE: Send old content if backend is down

Hi,

Just to be sure, are you doing health checks on your backend?

Wido

On Tue, 2010-11-30 at 08:16 +0100, Klapproth, Martin wrote:
> Thank you. This option helped a lot. But varnish delivers still 503
> errors when the backend is completely down (no route to host,
> connection refused). Is there a way to handle such situations too?
> 
>  
> 
>                                    
> ______________________________________________________________________
> From:Per Buer [mailto:perbu at varnish-software.com] 
> Sent: Montag, 29. November 2010 15:10
> To: Klapproth, Martin
> Cc: varnish-misc at varnish-cache.org
> Subject: Re: Send old content if backend is down
> 
> 
>  
> 
> On Mon, Nov 29, 2010 at 3:01 PM, Klapproth, Martin
> <Klapproth.M at sportresult.com> wrote:
> 
>         Hello,
>         
>          
>         
>         I have a general question relating to varnish's vcl
>         configuration. I want the varnish server to send old cached
>         objects if the backend is down (marked unhealthy). I have
>         tested various grace configurations but nothing worked,
>         because grace takes only effect if the backend is slow, but
>         not if it's down. So is there a way (maybe in vcl_error)?
>         
>         
>  
> 
> 
> Yes. There is a way.
> 
> 
>  
> 
> 
>
http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_se
rvers.html#saint-mode
> 
> 
> 
> -- 
> Per Buer,  Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
> 
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From TFigueiro at au.westfield.com  Tue Nov 30 08:05:20 2010
From: TFigueiro at au.westfield.com (Thiago Figueiro)
Date: Tue, 30 Nov 2010 19:05:20 +1100
Subject: Panic message: Assert error in VRT_IP_string()
Message-ID: <64E73E81AAC26A49AC9EA28CBE6536510797C9D6@AUSYDEVS01.au.ad.westfield.com>

(long message - heaps of debug goodies - sorry about the length)

Hi there!  I've run into some varnish crashes and was hoping someone
could help.

As our site gets busier I have seen Varnish crashing.  One thing in
common between the requests was the presence of long Cookies.

I've googled for the errors we get and the best I could come up with was
to increase sess_workspace to 32k and upgrade to 2.1.4 (we were running
2.1.2 when I started seeing the errors about 3 weeks ago).

We're starting varnish with:

/usr/sbin/varnishd -P /var/run/varnish.pid -f /etc/varnish/default.vcl
-T 127.0.0.1:6082 -t 120 -p thread_pools 6 -w 150,3000,120 -u varnish -g
varnish -h classic,91193 -s
file,/var/lib/varnish/varnish_storage.bin,25G -p listen_depth 4096 -p
session_linger 50/100/150 -p sess_workspace 32768 -p lru_interval 60

This is the latest crash message in /var/log/messages:

2010-11-30T12:31:47.093463+11:00 rp1 varnishd[3174]: 
Child (3178) Panic message: Assert error in VRT_IP_string(), cache_vrt.c
line 888:
Condition((p = WS_Alloc(sp->http->ws, len)) != 0) not true.
thread = (cache-worker)
ident = Linux,2.6.18-194.26.1.el5,x86_64,-sfile,-hclassic,epoll
Backtrace:
  0x423aa6: pan_ic+b6
  0x42b8fc: VRT_IP_string+11c
  0x2ab0eb904cb8: _end+2ab0eb2916b0
  0x428ab3: VCL_recv_method+43
  0x41450a: CNT_Session+5ca
  0x425e68: wrk_do_cnt_sess+b8
  0x42516e: wrk_thread_real+32e
  0x2b2c2611573d: _end+2b2c25aa2135
  0x2b2c26899f6d: _end+2b2c26226965
sp = 0x2ab0ec07c008 {
  fd = 30, id = 30, xid = 508103012,
  client = 208.80.194.26 57092,
  step = STP_RECV,
  handling = deliver,
  err_code = 400, err_reason = (null),
  restarts = 4, esis = 0
  ws = 0x2ab0ec07c080 { overflow
    id = "sess",
    {s,f,r,e} = {0x2ab0ec07ccd8,+32760,(nil),+32768},
  },
  http[req] = {
    ws = 0x2ab0ec07c080[sess]
      "GET",
      "/derby/images/derby/downloads/Car_Park_Map_Jan2010.pdf",
      "HTTP/1.0",
      "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; America Online
Browser 1.1; rev1.2; Windows NT 5.1; SV1; Fu
nWebProducts; hbtools 4.7.0)",
      "Cookie:
%BC%C3%C4%FE%C8%FD%BA%C5%C3%BA%BF%F3%B9%A4%BB%E1=ViewUrl=%2Findex%2Easp;
.ASPXANONYMOUS=S9gNy_-uzAEkAAAAYTQ5ODM3NzgtNDgxYy00ZWQ0LTgyNzAtNjBhZDUyM
2ZjZWI40; 20gp=R3954494786; 240plan=R3762637933; 300gp=R3396056608;
60gp=R525209263;
7c4cd48b6985de8e58134182f87d9d92=b4e9c9c9dcff9b856004afc652d5a0ac;
82690c47ee3cca07192225ab40e9b146=a84911870bc1f0a803596001388321ef;
91489bd6ed9a2e2903386db3df6e1458=3a22e5281e6bbd2adcf9c91aba899fc1;
AA002=001291080700-8970232; ANON_CONFIRM=TRUE; ARPT=LWYXXLSSm2CYLU;
ASP.NET_SessionId=suavbzndghxhvg55fgyfyl55;
ASPSESSIONIDAAABSRDB=AAFEEIMBEICDABKPHIKMKBPH;
ASPSESSIONIDAAADBRDS=CFCEJBCCAAKMDHPLMMBLJMCG;
ASPSESSIONIDAABCQRBD=HHGOAKNBJPADCIICAEEOBBCP;
ASPSESSIONIDAABCSBCR=JFGFLMPBKDAJAMOMJBJJFNHJ;
ASPSESSIONIDAADTCQBD=GKMAFAMBIMBAIFECLGFCHJBL; ASPSESSIO

(entry seems truncated - perhaps maximum syslog message size?)

I managed to find the request in varnishlog:

   30 SessionOpen  c 208.80.194.26 57092 :80
   30 Debug        c "herding"
   30 ReqStart     c 208.80.194.26 57092 508103012
   30 RxRequest    c GET
   30 RxURL        c
/derby/images/derby/downloads/Car_Park_Map_Jan2010.pdf
   30 RxProtocol   c HTTP/1.0
   30 RxHeader     c Host: uk.westfield.com
   30 RxHeader     c User-Agent: Mozilla/4.0 (compatible; MSIE 6.0;
America Online Browser 1.1; rev1.2; Windows NT 5.1; SV1; FunWebProducts;
hbtools 4.7.0)
   30 RxHeader     c Cookie:
%BC%C3%C4%FE%C8%FD%BA%C5%C3%BA%BF%F3%B9%A4%BB%E1=ViewUrl=%2Findex%2Easp;
.ASPXANONYMOUS=S9gNy_-uzAEkAAAAYTQ5ODM3NzgtNDgxYy00ZWQ0LTgyNzAtNjBhZDUyM
2ZjZWI40; 20gp=R3954494786; 240plan=R3762637933; 300gp=R3396056608;
60gp=R525209263; 7c4cd48b6985de8e5
   30 VCL_call     c recv
   30 VCL_return   c lookup
   30 VCL_call     c hash
   30 VCL_return   c hash
   30 VCL_call     c miss
   30 VCL_return   c fetch
   30 Backend      c 49 centre_rails centre_rail_uschi_app01
   49 TxRequest    b GET
   49 TxURL        b
/derby/images/derby/downloads/Car_Park_Map_Jan2010.pdf
   49 TxProtocol   b HTTP/1.1
   49 TxHeader     b User-Agent: Mozilla/4.0 (compatible; MSIE 6.0;
America Online Browser 1.1; rev1.2; Windows NT 5.1; SV1; FunWebProducts;
hbtools 4.7.0)
   49 TxHeader     b X-Forwarded-For: 208.80.194.26
   49 TxHeader     b X-Forwarded-Host: uk.westfield.com
   49 TxHeader     b Host: premium.prod.dbg.westfield.com
   49 TxHeader     b Cookie:
%BC%C3%C4%FE%C8%FD%BA%C5%C3%BA%BF%F3%B9%A4%BB%E1=ViewUrl=%2Findex%2Easp;
.ASPXANONYMOUS=S9gNy_-uzAEkAAAAYTQ5ODM3NzgtNDgxYy00ZWQ0LTgyNzAtNjBhZDUyM
2ZjZWI40; 20gp=R3954494786; 240plan=R3762637933; 300gp=R3396056608;
60gp=R525209263; 7c4cd48b6985de8e5
   49 TxHeader     b X-Varnish: 508103012
   49 RxProtocol   b HTTP/1.1
   49 RxStatus     b 400
   49 RxResponse   b Bad Request
   49 RxHeader     b Date: Tue, 30 Nov 2010 01:31:47 GMT
   49 RxHeader     b Server: Apache
   49 RxHeader     b Content-Length: 7573
   49 RxHeader     b Connection: close
   49 RxHeader     b Content-Type: text/html; charset=iso-8859-1
   30 TTL          c 508103012 RFC 120 1291080707 0 0 0 0
   30 VCL_call     c fetch
   30 VCL_return   c restart
   30 ObjProtocol  c HTTP/1.1
   30 ObjStatus    c 400
   30 ObjResponse  c Bad Request
   30 ObjHeader    c Date: Tue, 30 Nov 2010 01:31:47 GMT
   30 ObjHeader    c Content-Type: text/html; charset=iso-8859-1
   30 ObjHeader    c Server: Apache
   49 Length       b 7573
   49 BackendClose b centre_rail_uschi_app01
   30 VCL_call     c recv
   30 LostHeader   c Cookie:
   30 VCL_return   c lookup
   30 VCL_call     c hash
   30 VCL_return   c hash
   30 VCL_call     c miss
   30 VCL_return   c fetch
   30 Backend      c 41 centre_rails centre_rail_uschi_app02

(repeats x4 until varnish crashes and restarts)

   30 TTL          c 508103012 RFC 120 1291080707 0 0 0 0
   30 VCL_call     c fetch
   30 VCL_return   c restart
   30 ObjProtocol  c HTTP/1.1
   30 ObjStatus    c 400
   30 ObjResponse  c Bad Request
   30 ObjHeader    c Date: Tue, 30 Nov 2010 01:31:47 GMT
   30 ObjHeader    c Content-Type: text/html; charset=iso-8859-1
   30 ObjHeader    c Server: Apache
   18 Length       b 7679
   18 BackendClose b centre_rail_uschi_app03
    0 WorkThread   - 0x420e2e50 start
    0 CLI          - Rd vcl.load "boot" ./vcl.jCxU17wG.so

I can see all backends Varnish tried to use returned 400.  I could
probably fix the request by stripping the whole Cookie (it's in my
backlog) or passing the 400 to the client but that doesn't change the
fact that Varnish crashed.

We collect statistics every 5 minutes and the ones below collected about
2 minutes before Varnish crashed (it had been running for about 4 hours
since the last restart).

2010-11-30T12:30:05.379422+11:00 297136-rp1 varnishd[3174]: (...)
136842  Client connections accepted
159246  Client requests received
115849  Cache hits
39311  Cache misses
16038  Backend conn. success
26600  Backend conn. reuses
14285  Backend conn. was closed
40921  Backend conn. recycles
2      Fetch head
31677  Fetch with Length
10592  Fetch chunked
170    Fetch wanted close
78     N struct sess_mem
26     N struct sess
31448  N struct object
31491  N struct objectcore
30457  N struct objecthead
62889  N struct smf
1      N large free smf
37     N struct vbe_conn
900    N worker threads
900    N worker threads created
48     N overflowed work requests
31     N backends
5030   N expired objects
20571  N LRU moved objects
154544 Objects sent with write
136840 Total Sessions
159246 Total Requests
4517   Total pass
41102  Total fetch
49574908  Total header bytes
2419039389  Total body bytes
99858  Session Closed
131864 Session Linger
58812  Session herd
7607244  SHM records
766387  SHM writes
77  SHM MTX contention
2  SHM cycles through buffer
78841  allocator requests
62888  outstanding allocations
1381466112 bytes allocated
25462079488 bytes free
1721  SMS allocator requests
1667264  SMS bytes allocated
1667264  SMS bytes freed
42635  Backend requests made
1  N vcl total
1  N vcl available
5383  N total active purges
5394  N new purges added
11  N old purges deleted
44017  N objects tested
10261150  N regexps tested against
2716  N duplicate purges rem



Did you read until here? Wow.  Thanks!  What do you think then?



Cheers,

Thiago Figueiro
Technical Operations Manager

Westfield Digital Business
T +61 (0)2 9358 7123
F +61 (0)2 9357 7866
W westfield.com.au
E tfigueiro at au.westfield.com





______________________________________________________
    CONFIDENTIALITY NOTICE    
This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is (are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction all copies of the original electronic mail message (along with any attachments).
______________________________________________________



From wido at widodh.nl  Tue Nov 30 08:17:26 2010
From: wido at widodh.nl (Wido den Hollander)
Date: Tue, 30 Nov 2010 09:17:26 +0100
Subject: pass user IP to file
In-Reply-To: <AANLkTi=5fcf0pCMiukiJPwjG85+0ex-OS=kUC7=W-fDg@mail.gmail.com>
References: <AANLkTi=5fcf0pCMiukiJPwjG85+0ex-OS=kUC7=W-fDg@mail.gmail.com>
Message-ID: <1291105048.12922.14.camel@wido-laptop.pcextreme.nl>

Hi Andy,

Take a look at mod_extract_forwarded for Apache, this will help you.

Or mod_rpath does the same.

Just make sure you pass the correct X-Forwarded-For header in your VCL.

Wido

On Mon, 2010-11-29 at 21:48 +0000, Andy Bailey wrote:
> I have a file on my server that checks a user hasn't accessed it more
> than 5 times in the past 5 minutes by saving the IP address of every
> request. Since having varnish installed, the file is denying access to
> everyone because for some reason the IP address that is being stored
> is always my server IP. How would I make sure that requests to that
> file pass the users IP address?
> 
> 
> hope you can help?
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From tfheen at varnish-software.com  Tue Nov 30 08:45:56 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Tue, 30 Nov 2010 09:45:56 +0100
Subject: Panic message: Assert error in VRT_IP_string()
In-Reply-To: <64E73E81AAC26A49AC9EA28CBE6536510797C9D6@AUSYDEVS01.au.ad.westfield.com>
	(Thiago Figueiro's message of "Tue, 30 Nov 2010 19:05:20 +1100")
References: <64E73E81AAC26A49AC9EA28CBE6536510797C9D6@AUSYDEVS01.au.ad.westfield.com>
Message-ID: <87eia319ob.fsf@qurzaw.varnish-software.com>

]] "Thiago Figueiro" 

Hi,

| 2010-11-30T12:31:47.093463+11:00 rp1 varnishd[3174]: 
| Child (3178) Panic message: Assert error in VRT_IP_string(), cache_vrt.c
| line 888:
| Condition((p = WS_Alloc(sp->http->ws, len)) != 0) not true.

This typically means you're out of workspace.  32k is not particularly
much.  The default is 64k, and if that causes problems, try increasing
it to 128k or 256k.  Having it set too high will just cause the virtual
size of the varnish process to grow, it won't actually consume more
memory.

regards,
-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 92 64



From pom at dmsp.de  Tue Nov 30 10:46:09 2010
From: pom at dmsp.de (Stefan Pommerening)
Date: Tue, 30 Nov 2010 11:46:09 +0100
Subject: pass user IP to file
In-Reply-To: <1291105048.12922.14.camel@wido-laptop.pcextreme.nl>
References: <AANLkTi=5fcf0pCMiukiJPwjG85+0ex-OS=kUC7=W-fDg@mail.gmail.com>
	<1291105048.12922.14.camel@wido-laptop.pcextreme.nl>
Message-ID: <4CF4D5F1.4010707@dmsp.de>

Hi,

please note that for passing the correct X-Forwarded-For header to
your backend webserver you will need to unset the X-Forwarded-For
header in your VCL because any client-side squid will set the same
header resulting in a list of IP addresses contained in one single
X-Forwarded-For header.

sub vcl_recv {
    # Remove incoming X-Forwarded-For header to let
    # Varnish insert a new header useful for Apache logging
    remove req.http.X-Forwarded-For;
    [...]
}

Now you can perform any actions (on backend) based on the
setting of the X-Forwarded-For header.

Nevertheless you should ensure that you've configured this
specific file not to be cached in varnish by 'pass' it based on
a regular expression (url based) in your VCL, too.

Stefan


Am 30.11.2010 09:17, schrieb Wido den Hollander:
> Hi Andy,
>
> Take a look at mod_extract_forwarded for Apache, this will help you.
>
> Or mod_rpath does the same.
>
> Just make sure you pass the correct X-Forwarded-For header in your VCL.
>
> Wido
>
> On Mon, 2010-11-29 at 21:48 +0000, Andy Bailey wrote:
>> I have a file on my server that checks a user hasn't accessed it more
>> than 5 times in the past 5 minutes by saving the IP address of every
>> request. Since having varnish installed, the file is denying access to
>> everyone because for some reason the IP address that is being stored
>> is always my server IP. How would I make sure that requests to that
>> file pass the users IP address?
>>
>>
>> hope you can help?




From lists at rtty.us  Tue Nov 30 12:52:59 2010
From: lists at rtty.us (Bob Camp)
Date: Tue, 30 Nov 2010 07:52:59 -0500
Subject: pass user IP to file
In-Reply-To: <AANLkTi=5fcf0pCMiukiJPwjG85+0ex-OS=kUC7=W-fDg@mail.gmail.com>
References: <AANLkTi=5fcf0pCMiukiJPwjG85+0ex-OS=kUC7=W-fDg@mail.gmail.com>
Message-ID: <6FBA6339-9C9F-460F-9986-7F394D9A2C28@rtty.us>

Hi

Is it a single file, or does each user get their own file (just curious) ? 

One way or the other you will need to be sure that the file request is passed to the backend and that the results are always honored. These setups can be a big pain with a cache, since the cache does not know much about who is and who is not logged in. It will server stuff when it's not supposed to. It can block (as you have seen) the keep alive. If this is for occasional use, you might consider a separate path to the backend for this stuff that goes around the cache. I do it by looking at the requested URL's and deciding that it's a "don't bother with the cache" sort of request. 

Bob


On Nov 29, 2010, at 4:48 PM, Andy Bailey wrote:

> I have a file on my server that checks a user hasn't accessed it more than 5 times in the past 5 minutes by saving the IP address of every request. Since having varnish installed, the file is denying access to everyone because for some reason the IP address that is being stored is always my server IP. How would I make sure that requests to that file pass the users IP address?
> 
> hope you can help?
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc




From vburshteyn at broadway.com  Tue Nov 30 18:50:10 2010
From: vburshteyn at broadway.com (Vitaly Burshteyn)
Date: Tue, 30 Nov 2010 13:50:10 -0500
Subject: normalize the header
Message-ID: <0F69574F9901D4459C6B75C9FF64FBC405283E69@mxfl01.hollywoodmedia.corp>

Hi folks,

 So it seems as if the cache is working and i am getting cache hits.
When i shut down the back end i can still get to the cached pages.

On the other hand when i try a new browser i keep getting 503 pages.

 

I included part of my VCL code.  I try to normalize my headers but I am
missing something.

 

 

sub vcl_hash {

    set req.hash += req.http.cookie;

    set req.hash += req.http.set-cookie;

    }

 

 

########################################################################
##############################

sub vcl_recv {

    set req.backend = bw;  #backend information

 

 

    unset req.http.cookie;

 

    #redirect /admin to 404

    if (req.url ~ "^/admin/"){ set req.url = regsub(req.url, "^/admin/",
"/oops/");    }

 

    # Never cache this path

    if (req.url ~ "/affiliates/visa-signature/") {   return(pass);
}

    if (req.url ~ "/r/") { return(pass);    }

 

    if (req.http.cookie) {

        set req.http.cookie = regsuball(req.http.Cookie,
"(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");

        set req.http.cookie = regsub(req.http.Cookie, "^;\s*", "");    }

 

    if (req.http.Accept-Encoding) {

 

    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {

        # No point in compressing these

        remove req.http.Accept-Encoding;

    } elsif (req.http.Accept-Encoding ~ "gzip") {

        set req.http.Accept-Encoding = "gzip";

    } elsif (req.http.Accept-Encoding ~ "deflate") {

        set req.http.Accept-Encoding = "deflate";

    } else {

        # unkown algorithm

        remove req.http.Accept-Encoding;  }

}

   return(lookup);}

 

Vitaly Burshteyn

Senior Network Engineer

Broadway.com, Theatre Direct International

729  7th Avenue

New York, New York 10019

Phone: 212.817.9117

Cell# 917-701-5732

 


____________________________________
The information contained in this transmission may contain privileged 
and confidential information.  It is intended only for the use of the 
person(s) named above. If you are not the intended recipient,  you are 
hereby notified that any review, dissemination, distribution or 
duplication of this communication is strictly prohibited. If you are 
not the intended recipient, please contact the sender by reply email 
and destroy all copies of the original message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101130/9a611337/attachment.html>

From andrewgbailey at gmail.com  Tue Nov 30 13:01:33 2010
From: andrewgbailey at gmail.com (Andy Bailey)
Date: Tue, 30 Nov 2010 13:01:33 +0000
Subject: pass user IP to file
In-Reply-To: <6FBA6339-9C9F-460F-9986-7F394D9A2C28@rtty.us>
References: <AANLkTi=5fcf0pCMiukiJPwjG85+0ex-OS=kUC7=W-fDg@mail.gmail.com>
	<6FBA6339-9C9F-460F-9986-7F394D9A2C28@rtty.us>
Message-ID: <AANLkTikEdo71ZE-CNmT-7gnCU-Bof+kMTpLaWLZ-Zbk_@mail.gmail.com>

thanks for your help. This is just for one file that is accessed by all and
it doesn't matter if the user is logged in or not.
if have tried

if (req.url ~ "commentluvapi.php") {

return(pass);

}

but requests made to that file show headers that tell me it was cached, I
made sure I did a /etc/init.d/varnish restart after changing default.vcl .Is
there anything else I should do to make sure varnish reloads the config?

I have just had mod_extract_forwarded added to apache so I will be adding set
req.http.X-Forwarded-For = client.ip; also, will this help?

thanks so much for your replies

Andy

---
Download the CommentLuv plugin for Wordpress and reward your readers.
http://comluv.com


On Tue, Nov 30, 2010 at 12:52 PM, Bob Camp <lists at rtty.us> wrote:

> Hi
>
> Is it a single file, or does each user get their own file (just curious) ?
>
> One way or the other you will need to be sure that the file request is
> passed to the backend and that the results are always honored. These setups
> can be a big pain with a cache, since the cache does not know much about who
> is and who is not logged in. It will server stuff when it's not supposed to.
> It can block (as you have seen) the keep alive. If this is for occasional
> use, you might consider a separate path to the backend for this stuff that
> goes around the cache. I do it by looking at the requested URL's and
> deciding that it's a "don't bother with the cache" sort of request.
>
> Bob
>
>
> On Nov 29, 2010, at 4:48 PM, Andy Bailey wrote:
>
> > I have a file on my server that checks a user hasn't accessed it more
> than 5 times in the past 5 minutes by saving the IP address of every
> request. Since having varnish installed, the file is denying access to
> everyone because for some reason the IP address that is being stored is
> always my server IP. How would I make sure that requests to that file pass
> the users IP address?
> >
> > hope you can help?
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20101130/8127bf47/attachment.html>