From hugo.cisneiros at gmail.com  Fri Mar  1 03:26:51 2013
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Fri, 1 Mar 2013 00:26:51 -0300
Subject: Rewriting served URLs dependent on user agent
In-Reply-To: <CABiY0=jE7Ouj_+Hk9U5sbVjz9Zfoqa48gFUXPr7VLKCS0kLa1g@mail.gmail.com>
References: <CABiY0=jE7Ouj_+Hk9U5sbVjz9Zfoqa48gFUXPr7VLKCS0kLa1g@mail.gmail.com>
Message-ID: <CA+KACLmw6u2V0XYcszfDJPi9+8W3dum6Ob5eLcq-py98skuXog@mail.gmail.com>

On Thu, Feb 28, 2013 at 8:33 PM, Ian Evans <dheianevans at gmail.com> wrote:

> I've been looking at this site's discussion of how they're handling
> the traffic loss caused by Google's redesign of their image search.
>
>
> http://pixabay.com/en/blog/posts/hotlinking-protection-and-watermarking-for-google-32/


[...]

Is there a way that Varnish could cache two versions of the page?
>
> One, human visitors would get the cached page with the?i
> Two, robot user agents would get a cached version where Varnish would
> strip all the ?i from urls.
>
> Is that possible? Thanks for any pointers.
>

Yes. On vcl_recv you can detect if it's a bot and do anything on the URL,
like:

sub vcl_recv {
  if (req.http.User-Agent ~ "googlebot") {
    regsub(req.http.url, "\?i", "");
  }

  ...

  return (lookup);
}

This will tell Varnish to strip the "?i" only when the HTTP header
User-Agent contains "googlebot".

Since vcl_recv is executed before any cache lookup, it'll store two
different caches (when missed): one for the url "image.jpg?i" and other for
"image.jpg".

[]'s
Hugo
www.devin.com.br
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130301/0cca22a4/attachment.html>

From jnerin+varnish at gmail.com  Sun Mar  3 21:30:18 2013
From: jnerin+varnish at gmail.com (=?UTF-8?B?Sm9yZ2UgTmVyw61u?=)
Date: Sun, 3 Mar 2013 22:30:18 +0100
Subject: [PATCH] varnish_ munin plugin
Message-ID: <CAF-dNMa5KgedhZ_WF0MFKrrUVGdFuhF8aBGmnnd-pbc2kPLyDQ@mail.gmail.com>

Hello, in the varnish_ munin plugin included with official munin package
and hosted at
http://munin-monitoring.org/browser/munin/plugins/node.d/varnish_.in it
says the official way to send patches is this list.

We have been using a modified varnish_ plugin for some time and I finally I
have taken the code in sync with current plugin in order to send upstream
our modifications.

Attached is a patch that does add a few values to some graphs:
* Add client_drop to request_rate graph.
* Add s_pass s_pipe to hit_rate graph (now they are closer to sum 100% as
we have ~3% of s_pass which weren't shown previously).
* Add backend_toolate & backend_retry to backend_traffic graph.
* Add n_wrk_lqueue & n_wrk_queued to threads graph.
* Add Transient memory to memory_usage graph.

In our old plugin we also have a graph that I'm not sure how to do now and
I think it's useful. It shows the average object size measured as:
                                'rpn' => [ 'SMF_s0_g_bytes',
'SMA_s0_g_bytes', '+', 'n_object', '/' ]

We sum the values of the two storages because at one time we switched from
the file storage to the malloc storage and we lost the graphs. Usually one
will be zero and the other will be the current memory in use. I know that
it's not the exact average object size as there are more things stored, but
at least for our cache contents seems to be a valuable metric, we are in
the range 15-20k.

But in the current version I'm not sure how to replicate this graph.

Regards.

-- 
Jorge Ner?n
<jnerin at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130303/d083e8e5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: varnish_.in.patch
Type: application/octet-stream
Size: 2114 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130303/d083e8e5/attachment.obj>

From jdmls at yahoo.com  Mon Mar  4 10:39:58 2013
From: jdmls at yahoo.com (John Doe)
Date: Mon, 4 Mar 2013 02:39:58 -0800 (PST)
Subject: Timing the backend response...
Message-ID: <1362393598.69895.YahooMailNeo@web121605.mail.ne1.yahoo.com>

Hi,

Is there an option to time the backend response...?
I cannot find any timing info in varnishstat...

Is the only way to get it to continually parse the logs for the
time_firstbyte (which is only a part of the total time)?

Thx,
JD




From jefe78 at gmail.com  Tue Mar  5 03:17:40 2013
From: jefe78 at gmail.com (Jeffrey Taylor)
Date: Mon, 4 Mar 2013 22:17:40 -0500
Subject: Pass 200 on 404
In-Reply-To: <CABtDKm70z46aohB8m2F13O+X2eg-kFa_aTpOHxir60XcuiCXSQ@mail.gmail.com>
References: <CAHjcbpXwJYKTUQsOg+heomSNtiHnb8qJLMj6gSj89UnQ1mAvpw@mail.gmail.com>
 <CA+KACLn0g1DAYE1=Bd4YZPnKi4xAEdB-_1+ot73FhV4S4N7hFw@mail.gmail.com>
 <CAHjcbpWKr5vtkBxXBX1oJYWfePMo2dZVcQZKLKef-LPD_EJH7A@mail.gmail.com>
 <CA+KACLkNt4U1JF_iAPiO3vTH9VG2PshCWXVht-HZMYKEdOcc-Q@mail.gmail.com>
 <CAHjcbpU04o7wQJOQMdFDnOKE61j4CiB6ZintCPhP_bOzSWXMEA@mail.gmail.com>
 <CABtDKm70z46aohB8m2F13O+X2eg-kFa_aTpOHxir60XcuiCXSQ@mail.gmail.com>
Message-ID: <CAHjcbpXJqQadV13XJvt1XsWaByzTrA=oPPHVFBhsac2y7EdSVA@mail.gmail.com>

In regards to this issue. I had a further question.

Is it possible to only pass this rule specifically when certain(3) file
extensions are involved. I.e., Only when extentsions are .abc .abd .abe ?

Thanks,
Jeff


On Mon, Feb 25, 2013 at 5:10 PM, Dridi Boukelmoune <
dridi.boukelmoune at zenika.com> wrote:

> Hi,
>
> Have you tried the grace mechanism ? It allows an additional TTL when a
> backend is sick. If you want to mark only specific URLs as sick instead of
> the whole backend you can play with the saint mode.
>
> It will definitely be tricky the way you described your issue. I'd also
> search CF's documentation to make sure whether caching can be tuned.
>
> Best Regards,
> Dridi
>
> Envoy? de mon smartphone
> Le 25 f?vr. 2013 21:31, "Jeffrey Taylor" <jefe78 at gmail.com> a ?crit :
>
> Ya, passing 200's on everything would be fun :)
>>
>> Thanks again Hugo, I'll set that up.
>>
>> Jeff
>>
>>
>> On Mon, Feb 25, 2013 at 3:28 PM, Hugo Cisneiros (Eitch) <
>> hugo.cisneiros at gmail.com> wrote:
>>
>>> On Mon, Feb 25, 2013 at 5:22 PM, Jeffrey Taylor <jefe78 at gmail.com>wrote:
>>>
>>>> We only actually want to do this with our video segments. The reason
>>>> is, if our streaming box or our ingress box crash, we start generating
>>>> 404's to CF. The second CF gets a 404, we're screwed for ~5 minutes. That's
>>>> a lifetime for live video.
>>>>
>>>> CF allows TTLs on all types of files/rules, but a 404 is a solid 5
>>>> minutes cached by them.
>>>>
>>>> I'm open to other ideas!
>>>>
>>>
>>> Ah, if it's only these items, I think it's OK. I thought that replying
>>> 200 to everything should confuse crawlers, robots, users, and so on :-)
>>>
>>> Give the beresp.status and vcl_error a try. I used this a lot to
>>> generate temporary redirects (301) instead of 200 or 404.
>>>
>>> --
>>> []'s
>>> Hugo
>>> www.devin.com.br
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130304/b4a1f4ab/attachment.html>

From hugo.cisneiros at gmail.com  Tue Mar  5 04:32:02 2013
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Tue, 5 Mar 2013 01:32:02 -0300
Subject: Pass 200 on 404
In-Reply-To: <CAHjcbpXJqQadV13XJvt1XsWaByzTrA=oPPHVFBhsac2y7EdSVA@mail.gmail.com>
References: <CAHjcbpXwJYKTUQsOg+heomSNtiHnb8qJLMj6gSj89UnQ1mAvpw@mail.gmail.com>
 <CA+KACLn0g1DAYE1=Bd4YZPnKi4xAEdB-_1+ot73FhV4S4N7hFw@mail.gmail.com>
 <CAHjcbpWKr5vtkBxXBX1oJYWfePMo2dZVcQZKLKef-LPD_EJH7A@mail.gmail.com>
 <CA+KACLkNt4U1JF_iAPiO3vTH9VG2PshCWXVht-HZMYKEdOcc-Q@mail.gmail.com>
 <CAHjcbpU04o7wQJOQMdFDnOKE61j4CiB6ZintCPhP_bOzSWXMEA@mail.gmail.com>
 <CABtDKm70z46aohB8m2F13O+X2eg-kFa_aTpOHxir60XcuiCXSQ@mail.gmail.com>
 <CAHjcbpXJqQadV13XJvt1XsWaByzTrA=oPPHVFBhsac2y7EdSVA@mail.gmail.com>
Message-ID: <CA+KACLkNgpX-rCpJSbfHz=M-RG5fTO1Lnpj=LwrWR06d-4M-Vg@mail.gmail.com>

On Tue, Mar 5, 2013 at 12:17 AM, Jeffrey Taylor <jefe78 at gmail.com> wrote:

> In regards to this issue. I had a further question.
>
> Is it possible to only pass this rule specifically when certain(3) file
> extensions are involved. I.e., Only when extentsions are .abc .abd .abe ?
>

Sure!

sub vcl_fetch {
  if (beresp.status == 404 && req.url ~ "\.ab(c|d|e)) {
    set beresp.ttl = 1m;
    error 200 "Not Found";
  }

  [...]
}

adding a regex to match the request url ending with .abc, .abd, .abe.

-- 
[]'s
Hugo
www.devin.com.br
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130305/a030a024/attachment.html>

From jefe78 at gmail.com  Tue Mar  5 04:33:20 2013
From: jefe78 at gmail.com (Jeffrey Taylor)
Date: Mon, 4 Mar 2013 23:33:20 -0500
Subject: Pass 200 on 404
In-Reply-To: <CA+KACLkNgpX-rCpJSbfHz=M-RG5fTO1Lnpj=LwrWR06d-4M-Vg@mail.gmail.com>
References: <CAHjcbpXwJYKTUQsOg+heomSNtiHnb8qJLMj6gSj89UnQ1mAvpw@mail.gmail.com>
 <CA+KACLn0g1DAYE1=Bd4YZPnKi4xAEdB-_1+ot73FhV4S4N7hFw@mail.gmail.com>
 <CAHjcbpWKr5vtkBxXBX1oJYWfePMo2dZVcQZKLKef-LPD_EJH7A@mail.gmail.com>
 <CA+KACLkNt4U1JF_iAPiO3vTH9VG2PshCWXVht-HZMYKEdOcc-Q@mail.gmail.com>
 <CAHjcbpU04o7wQJOQMdFDnOKE61j4CiB6ZintCPhP_bOzSWXMEA@mail.gmail.com>
 <CABtDKm70z46aohB8m2F13O+X2eg-kFa_aTpOHxir60XcuiCXSQ@mail.gmail.com>
 <CAHjcbpXJqQadV13XJvt1XsWaByzTrA=oPPHVFBhsac2y7EdSVA@mail.gmail.com>
 <CA+KACLkNgpX-rCpJSbfHz=M-RG5fTO1Lnpj=LwrWR06d-4M-Vg@mail.gmail.com>
Message-ID: <CAHjcbpWxtA6EGj5kZ4OA0Dv9LECr3V4NyVEf4XozWJXinQ=YqQ@mail.gmail.com>

Very cool, thanks Hugo!

Jeff


On Mon, Mar 4, 2013 at 11:32 PM, Hugo Cisneiros (Eitch) <
hugo.cisneiros at gmail.com> wrote:

> On Tue, Mar 5, 2013 at 12:17 AM, Jeffrey Taylor <jefe78 at gmail.com> wrote:
>
>> In regards to this issue. I had a further question.
>>
>> Is it possible to only pass this rule specifically when certain(3) file
>> extensions are involved. I.e., Only when extentsions are .abc .abd .abe ?
>>
>
> Sure!
>
> sub vcl_fetch {
>   if (beresp.status == 404 && req.url ~ "\.ab(c|d|e)) {
>     set beresp.ttl = 1m;
>     error 200 "Not Found";
>   }
>
>   [...]
> }
>
> adding a regex to match the request url ending with .abc, .abd, .abe.
>
> --
> []'s
> Hugo
> www.devin.com.br
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130304/ee6dd286/attachment.html>

From miguel_3_gonzalez at yahoo.es  Tue Mar  5 09:57:35 2013
From: miguel_3_gonzalez at yahoo.es (=?ISO-8859-1?Q?Miguel_Gonz=E1lez_Casta=F1os?=)
Date: Tue, 05 Mar 2013 10:57:35 +0100
Subject: Varnish and Symfony2
Message-ID: <5135C18F.5090803@yahoo.es>

Dear all,

   I'm new to Varnish so bear my naive questions. We have a LAMP server 
running a Symfony2 application. I have just configured a very basic 
configuration just pointing to the Apache backend at 8000 port. I've 
realized that some content of the front page is not displayed (javascript).

   How can I check why is that content not displayed?

   Just to let you know, we have cache enabled on Symfony too.

   Regards,

   Miguel



From miguel_3_gonzalez at yahoo.es  Tue Mar  5 11:17:13 2013
From: miguel_3_gonzalez at yahoo.es (=?ISO-8859-1?Q?Miguel_Gonz=E1lez_Casta=F1os?=)
Date: Tue, 05 Mar 2013 12:17:13 +0100
Subject: Varnish and Symfony2
In-Reply-To: <5135C18F.5090803@yahoo.es>
References: <5135C18F.5090803@yahoo.es>
Message-ID: <5135D439.5000907@yahoo.es>

On 05/03/2013 10:57, Miguel Gonz?lez Casta?os wrote:
> Dear all,
>
>   I'm new to Varnish so bear my naive questions. We have a LAMP server 
> running a Symfony2 application. I have just configured a very basic 
> configuration just pointing to the Apache backend at 8000 port. I've 
> realized that some content of the front page is not displayed 
> (javascript).
>
>   How can I check why is that content not displayed?
>
>   Just to let you know, we have cache enabled on Symfony too.
It looks like an issue with javascript Date().getTime() function

Regards,

Miguel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130305/167dbed1/attachment.html>

From ruben at varnish-software.com  Wed Mar  6 21:03:45 2013
From: ruben at varnish-software.com (=?UTF-8?Q?Rub=C3=A9n_Romero?=)
Date: Wed, 6 Mar 2013 22:03:45 +0100
Subject: Do you have Varnish 3 party pictures and want to share? :-)
In-Reply-To: <BANLkTikAzkKCco=FC88rGeCE2-yQm7xEcQ@mail.gmail.com>
References: <BANLkTikAzkKCco=FC88rGeCE2-yQm7xEcQ@mail.gmail.com>
Message-ID: <CAEfnkDaARxjCy2CYbXzLF-LGK0-88VKfgaZUeLv57-1QA11Pcw@mail.gmail.com>

On Fri, Jun 17, 2011 at 10:49 PM, Rub?n Romero
<ruben at varnish-software.com>wrote:

> Hi everyone!
>
> 18 parties all in all! Wow. Hope you have had your fun this far!
>

Actually 19, since Miami got proof:
http://www.flickr.com/photos/63991227 at N08/5841599828/in/pool-1731128 at N22/


> As most people has already celebrated (some parties are hold tomorrow) I
> would like to ask you to not forget to tag your pictures with #v3rp. We
> only have Flickr integration on the site, but I have noticed that almost no
> one uses Flickr anymore :-)
>
> I was thinking about adding all of the pictures (from twitpic and what
> not) to one album in Flickr where we can see how the fun went everywhere.
> So feel free to send me links to your pictures or even the pictures
> themselves and I will upload them to Flickr and let you all know once that
> is done, or at least have all the links in one page so people can easily
> have an overview.
>

Today I was planning some community activities, thinking about VUG7 and the
next Varnish main release. So as I was compiling all media from the V3
party, I thought I should share what I found for those interested:

   - Party Site: http://twitpic.com/5d7gjt
   - London http://www.flickr.com/photos/90276624 at N00/5839415403/
   - Tokyo: http://www.flickr.com/photos/hmsk/sets/72157626949027592/
   - Tokyo Video: http://www.ustream.tv/recorded/15453305
   - Buenos Aires: http://twitpic.com/tag/v3rp
   - Oslo:
   https://plus.google.com/photos/118002861639621675838/albums/5619329249899280257?banner=pwa
   - Oslo2: http://twitpic.com/photos/sthalvor
   - Milan: http://twitter.yfrog.com/user/zephirworks/photos
   - Milan2:
   http://www.flickr.com/photos/zephirworks/with/5839728634/#photo_5839728634
   - Boston: http://twitpic.com/5cilj7
   - Boston/Portalnd (Dries' blog): http://buytaert.net/party-with-varnish

Awesomeness!

So, please do. I will add my own, I promise ;-)
>

My phone was stolen in D?sseldorf that weekend so no pictures from me.
Sorry.

And with this email I am officially starting the idea to start creating
more meetups <http://varnish.meetup.com/> around the world and gear up
towards the next Varnish release, hopefully later this year (4.0, right? ;-)


Al the best,
-- 
*Rub?n Romero*
Self-appointed Varnish Community Cheerleader | Varnish Software AS
Cell: +47 95964088 / Office: +47 21989260
Skype & Twitter: ruben_varnish
We Make Websites Fly!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130306/8fb3ed70/attachment.html>

From ruben at varnish-software.com  Wed Mar  6 21:17:29 2013
From: ruben at varnish-software.com (=?UTF-8?Q?Rub=C3=A9n_Romero?=)
Date: Wed, 6 Mar 2013 22:17:29 +0100
Subject: VUG6 report and Development news
Message-ID: <CAEfnkDbhi+N2Ymoov52HTQ1_i1Tz88SjNpVwbmuwMGqq_LmivQ@mail.gmail.com>

Hello,

Finally I managed to get the VUG6 report out and, in case you missed it,
the agenda, slides, notes and videos from both the user and developer days
are available online:

 https://www.varnish-cache.org/vug6
 https://www.varnish-cache.org/vug6-report

https://docs.google.com/a/varnish-software.com/spreadsheet/ccc?key=0Apb8MyYPKVqZdDlNTHl2dExMRllrSV9HUnBLdkxwb3c#gid=0
(VUG6
Dev Day discussion)

I know it has been ages since the meeting, but the report and notes gives
some nice insights into the user community and what people think about
Varnish.

As for development news since then, Lasse has been blogging about what is
going on:

http://lassekarstensen.wordpress.com/2013/01/24/varnish-cache-development-news/

http://lassekarstensen.wordpress.com/2013/02/27/varnish-development-news-feb-2013/

Enjoy!


Cheers,
-- 
*Rub?n Romero*
Global Sales Executive & Community Cheerleader | Varnish Software AS
Cell: +47 95964088 / Office: +47 21989260
Skype & Twitter: ruben_varnish
We Make Websites Fly!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130306/8886bc01/attachment.html>

From ruben at varnish-software.com  Wed Mar  6 21:26:38 2013
From: ruben at varnish-software.com (=?UTF-8?Q?Rub=C3=A9n_Romero?=)
Date: Wed, 6 Mar 2013 22:26:38 +0100
Subject: VUG7 location NYC or SF?
Message-ID: <CAEfnkDaRpHshxN1PVnHmin2FcQdMYQxMJdxm5gzOGrQSw=Ggcw@mail.gmail.com>

Hi everyone!

It seems that we are going for VUG7 in either the New York City area the
last week of May or in the San Francisco Bay area right after Velocity at
the end of June.

Right now I would like to know what the popular vote wants. So please give
me a quick reply to this email with which option you prefer, or send me a
tweet letting me know.

Thank you very much, Every answer is much appreciated!

(And sorry for spamming -misc)


All the best,
-- 
*Rub?n Romero*
Global Sales Executive & Community Cheerleader HI  | Varnish Software AS
Cell: +47 95964088 / Office: +47 21989260
Skype & Twitter: ruben_varnish
We Make Websites Fly!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130306/0a55fdca/attachment.html>

From lnsano at bol.com.br  Thu Mar  7 10:54:45 2013
From: lnsano at bol.com.br (lnsano at bol.com.br)
Date: Thu, 7 Mar 2013 07:54:45 -0300
Subject: Varnish 3 - thread_pools
In-Reply-To: <5137b3ca67d00_71b77e144784b6@a4-winter14.tmail>
References: <5137b3ca67d00_71b77e144784b6@a4-winter14.tmail>
Message-ID: <513871f586ef2_68da3ab6c6c1a3@a4-winter18.tmail>

An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130307/06168771/attachment.html>

From lnsano at bol.com.br  Thu Mar  7 11:44:59 2013
From: lnsano at bol.com.br (lnsano at bol.com.br)
Date: Thu, 7 Mar 2013 08:44:59 -0300
Subject: Varnish 3 - thread_pools
Message-ID: <51387dbb70b7f_2e2d2cbd46c1b8@a4-winter17.tmail>

Hi,

By the book, the thread_pools should be 2. However the server is multi-processed(8 cpus) and can handle more thread_pools.
This server handles more than 5000 requests/sec. So, I found the thread_pool_max should be around 1000 and up to 5000.

Is it a good ideia increasing the thread_pools to 8 and decreasing the thread_pool_max?
Or, what's the best config for a server with 8 cpus and 32GB ram.


regards,

Leandro Sano



From lkarsten at varnish-software.com  Thu Mar  7 14:39:32 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 7 Mar 2013 15:39:32 +0100
Subject: Varnish 3 - thread_pools
In-Reply-To: <51387dbb70b7f_2e2d2cbd46c1b8@a4-winter17.tmail>
References: <51387dbb70b7f_2e2d2cbd46c1b8@a4-winter17.tmail>
Message-ID: <20130307143931.GA2470@immer.varnish-software.com>

On Thu, Mar 07, 2013 at 08:44:59AM -0300, lnsano at bol.com.br wrote:
> By the book, the thread_pools should be 2. However the server is multi-processed(8 cpus) and can handle more thread_pools.
> This server handles more than 5000 requests/sec. So, I found the thread_pool_max should be around 1000 and up to 5000.
> Is it a good ideia increasing the thread_pools to 8 and decreasing the thread_pool_max?

No.

Keep thread_pools at 2.

> Or, what's the best config for a server with 8 cpus and 32GB ram.

Start out with 1 thread per concurrent connection.

-- 
With regards,
Lasse Karstensen
Varnish Software AS



From juanandrade1 at gmail.com  Thu Mar  7 22:07:47 2013
From: juanandrade1 at gmail.com (Juan Andrade)
Date: Thu, 7 Mar 2013 14:07:47 -0800
Subject: Varnishncsa not working
Message-ID: <CAEcTaTXhJo9SJ+Lm9qSkhmoCZCZ=asDpQoSRiB+ZQ-A1gFxZXw@mail.gmail.com>

I am running 'varnishncsa' with below process running.
nobody    1736 99.0  0.0  95564   788 pts/0    R    13:56   0:31
/usr/bin/varnishncsa -P /var/tmp/varnishncsa.pid -F %h %l %u %t "%r" %s %b
%{Varnish:time_firstbyte}x/%{Varnish:handling}x "%{Referer}i"
"%{User-agent}i"
nobody    1737  0.0  0.0   5476   724 pts/0    S    13:56   0:00
/usr/bin/logger -t varnish -p local7.info

Which should pipe output to 'logger' with 'varnish' tag.  Output is empty.
 I use this same setup with 3.0.3 and works great.  Varnishlog works.
Using latest source: commit 5582b35be3d6a66a1c2764a11781014066a879c6
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130307/001a6826/attachment.html>

From jnerin at gmail.com  Mon Mar 11 07:42:15 2013
From: jnerin at gmail.com (Jorge)
Date: Mon, 11 Mar 2013 08:42:15 +0100
Subject: Fwd: [PATCH] varnish_ munin plugin
In-Reply-To: <CAF-dNMa5KgedhZ_WF0MFKrrUVGdFuhF8aBGmnnd-pbc2kPLyDQ@mail.gmail.com>
References: <CAF-dNMa5KgedhZ_WF0MFKrrUVGdFuhF8aBGmnnd-pbc2kPLyDQ@mail.gmail.com>
Message-ID: <CAF-dNMZDFM5Npgf15Ux=oJvyyYrwVBtiC1U6T8F+ZVjpVKcm=g@mail.gmail.com>

Hello, in the varnish_ munin plugin included with official munin package
and hosted at
http://munin-monitoring.org/browser/munin/plugins/node.d/varnish_.in it
says the official way to send patches is this list.

We have been using a modified varnish_ plugin for some time and I finally I
have taken the code in sync with current plugin in order to send upstream
our modifications.

Attached is a patch that does add a few values to some graphs:
* Add client_drop to request_rate graph.
* Add s_pass s_pipe to hit_rate graph (now they are closer to sum 100% as
we have ~3% of s_pass which weren't shown previously).
* Add backend_toolate & backend_retry to backend_traffic graph.
* Add n_wrk_lqueue & n_wrk_queued to threads graph.
* Add Transient memory to memory_usage graph.

In our old plugin we also have a graph that I'm not sure how to do now and
I think it's useful. It shows the average object size measured as:
                                'rpn' => [ 'SMF_s0_g_bytes',
'SMA_s0_g_bytes', '+', 'n_object', '/' ]

We sum the values of the two storages because at one time we switched from
the file storage to the malloc storage and we lost the graphs. Usually one
will be zero and the other will be the current memory in use. I know that
it's not the exact average object size as there are more things stored, but
at least for our cache contents seems to be a valuable metric, we are in
the range 15-20k.

But in the current version of the munin plugin I'm not sure how to
replicate this graph.

Regards.

-- 
Jorge Ner?n
<jnerin at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130311/d2986b86/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: varnish_.in.patch
Type: application/octet-stream
Size: 2114 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130311/d2986b86/attachment.obj>

From kongfranon at gmail.com  Mon Mar 11 15:02:41 2013
From: kongfranon at gmail.com (Mike Franon)
Date: Mon, 11 Mar 2013 11:02:41 -0400
Subject: varnish giving error 503
Message-ID: <CADoBF4T=21176CMkNYi11ucTsrhSsm2fOBj=EYD5sjwxjg2r4Q@mail.gmail.com>

Hi,

I am trying to get varnish up and running super fast for a production
website, we are hammered.

We only want to cache one URL, that is being hammered, and pass
everything else through, I know this is a just a simple config, but is
there anything I am missing to get a 503?  Is it because our servers
are being hammered that it is not even able to get pages to cache?


sub vcl_recv {

  # set default backend
  set req.backend = default_director;





  # lookup stylesheets in the cache
  if (req.url ~ "^/Tree") {
    return(lookup);
  }

  return(pass);
}


sub vcl_fetch {
  if (req.url ~ "^/Tree") {
    # removing cookie
    unset beresp.http.Set-Cookie;

    # Cache for 1 day
    set beresp.ttl = 1d;
    return(deliver);
  }



From perbu at varnish-software.com  Mon Mar 11 21:41:32 2013
From: perbu at varnish-software.com (Per Buer)
Date: Mon, 11 Mar 2013 22:41:32 +0100
Subject: varnish giving error 503
In-Reply-To: <CADoBF4T=21176CMkNYi11ucTsrhSsm2fOBj=EYD5sjwxjg2r4Q@mail.gmail.com>
References: <CADoBF4T=21176CMkNYi11ucTsrhSsm2fOBj=EYD5sjwxjg2r4Q@mail.gmail.com>
Message-ID: <CAOXZevDe8RK7au28D+ZMa9L7+5tf68GdjF5Fpb8Cp0fs-xM7nw@mail.gmail.com>

Hi,

On Mon, Mar 11, 2013 at 4:02 PM, Mike Franon <kongfranon at gmail.com> wrote:

> Hi,
>
> I am trying to get varnish up and running super fast for a production
> website, we are hammered.
>
> We only want to cache one URL, that is being hammered, and pass
> everything else through, I know this is a just a simple config, but is
> there anything I am missing to get a 503?  Is it because our servers
> are being hammered that it is not even able to get pages to cache?
>

I'm guessing your clients are sending Cookie headers to Varnish and Varnish
isn't able to cache. Instead of forcing a lookup, which is a bad idea, you
should try to unset the Cookie header and let the default logic handle it.
You're already doing this with the response headers so you are half way.


-- 
 <http://www.varnish-software.com/> *Per Buer*
CEO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130311/62ea2f86/attachment.html>

From pablo at libo.com.ar  Tue Mar 12 00:51:56 2013
From: pablo at libo.com.ar (Pablo J. Villarruel)
Date: Mon, 11 Mar 2013 21:51:56 -0300
Subject: varnish giving error 503
In-Reply-To: <CAOXZevDe8RK7au28D+ZMa9L7+5tf68GdjF5Fpb8Cp0fs-xM7nw@mail.gmail.com>
References: <CADoBF4T=21176CMkNYi11ucTsrhSsm2fOBj=EYD5sjwxjg2r4Q@mail.gmail.com>
 <CAOXZevDe8RK7au28D+ZMa9L7+5tf68GdjF5Fpb8Cp0fs-xM7nw@mail.gmail.com>
Message-ID: <CAJPa6L1C8QE6zBuZpQJThd7ROT3gvWBB_e0bLaq9NZU4h3YnPA@mail.gmail.com>

Sorry but, the last week I have the same issue, I can fix it add feature
probe over the zone backend Config.
El mar 11, 2013 6:42 p.m., "Per Buer" <perbu at varnish-software.com> escribi?:

> Hi,
>
> On Mon, Mar 11, 2013 at 4:02 PM, Mike Franon <kongfranon at gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to get varnish up and running super fast for a production
>> website, we are hammered.
>>
>> We only want to cache one URL, that is being hammered, and pass
>> everything else through, I know this is a just a simple config, but is
>> there anything I am missing to get a 503?  Is it because our servers
>> are being hammered that it is not even able to get pages to cache?
>>
>
> I'm guessing your clients are sending Cookie headers to Varnish and
> Varnish isn't able to cache. Instead of forcing a lookup, which is a bad
> idea, you should try to unset the Cookie header and let the default logic
> handle it. You're already doing this with the response headers so you are
> half way.
>
>
> --
>  <http://www.varnish-software.com/> *Per Buer*
> CEO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130311/66d19d30/attachment.html>

From perbu at varnish-software.com  Tue Mar 12 09:52:27 2013
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 12 Mar 2013 10:52:27 +0100
Subject: varnish giving error 503
In-Reply-To: <CAOXZevDe8RK7au28D+ZMa9L7+5tf68GdjF5Fpb8Cp0fs-xM7nw@mail.gmail.com>
References: <CADoBF4T=21176CMkNYi11ucTsrhSsm2fOBj=EYD5sjwxjg2r4Q@mail.gmail.com>
 <CAOXZevDe8RK7au28D+ZMa9L7+5tf68GdjF5Fpb8Cp0fs-xM7nw@mail.gmail.com>
Message-ID: <CAOXZevB67Rzj-aO8198=bZ-jt7=DJBF70Mxz35Pk+N=L7OeNPg@mail.gmail.com>

On Mon, Mar 11, 2013 at 10:41 PM, Per Buer <perbu at varnish-software.com>wrote:

> Hi,
>
> On Mon, Mar 11, 2013 at 4:02 PM, Mike Franon <kongfranon at gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to get varnish up and running super fast for a production
>> website, we are hammered.
>>
>> We only want to cache one URL, that is being hammered, and pass
>> everything else through, I know this is a just a simple config, but is
>> there anything I am missing to get a 503?  Is it because our servers
>> are being hammered that it is not even able to get pages to cache?
>>
>
> I'm guessing your clients are sending Cookie headers to Varnish and
> Varnish isn't able to cache. Instead of forcing a lookup, which is a bad
> idea, you should try to unset the Cookie header and let the default logic
> handle it. You're already doing this with the response headers so you are
> half way.
>

Sorry, I was probably I bit tired when I wrote this. You are already
forcing a lookup so your VCL will store the reponse generated with the
request with a cookie (which is generelly a bad idea). What you need to do
is use varnishlog to look at failing request. If you can't figure it out
you can post it here and we'll probably spot the error right away.



-- 
 <http://www.varnish-software.com/> *Per Buer*
Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130312/95a6d7d4/attachment.html>

From miguel_3_gonzalez at yahoo.es  Tue Mar 12 11:26:12 2013
From: miguel_3_gonzalez at yahoo.es (Miguel Gonzalez)
Date: Tue, 12 Mar 2013 11:26:12 +0000 (GMT)
Subject: benchmarking varnish
Message-ID: <1363087572.33460.YahooMailNeo@web171605.mail.ir2.yahoo.com>

Dear all,

? I have a Varnish 3 installation pointing to an Apache Server. Any manual of how to tune Varnish, OS, Apache and any manual of how to test Varnish considering it's a cache tool?

? I have search around and so far I haven't found something like this:

? http://www.joedog.org/articles-tuning/

? Unfortunately the tips mentioned in the tutorial didn't help to improve the performance of Varnish.

? Regards,

? Miguel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130312/6ed3fdb1/attachment.html>

From contact at jpluscplusm.com  Tue Mar 12 11:35:21 2013
From: contact at jpluscplusm.com (Jonathan Matthews)
Date: Tue, 12 Mar 2013 11:35:21 +0000
Subject: benchmarking varnish
In-Reply-To: <1363087572.33460.YahooMailNeo@web171605.mail.ir2.yahoo.com>
References: <1363087572.33460.YahooMailNeo@web171605.mail.ir2.yahoo.com>
Message-ID: <CAKsTx7BE+_TRF4_OtdnifhKGsAceT+ur7wb3XXxmHCStSBEVkw@mail.gmail.com>

On 12 March 2013 11:26, Miguel Gonzalez <miguel_3_gonzalez at yahoo.es> wrote:
> Dear all,
>
>   I have a Varnish 3 installation pointing to an Apache Server. Any manual
> of how to tune Varnish, OS, Apache and any manual of how to test Varnish
> considering it's a cache tool?

Have you worked through
https://www.varnish-cache.org/docs/3.0/tutorial/increasing_your_hitrate.html
yet?

Jonathan
-- 
Jonathan Matthews // Oxford, London, UK
http://www.jpluscplusm.com/contact.html



From kelvin1111111 at gmail.com  Tue Mar 12 11:38:44 2013
From: kelvin1111111 at gmail.com (Kelvin Loke)
Date: Tue, 12 Mar 2013 19:38:44 +0800
Subject: Varnish to cache accordingly by inspecting Backend response header
Message-ID: <CAAct57wxkVnJEDNehc+ZwGCVQHgB8g4nThNTN8hx0NayxZ7fxw@mail.gmail.com>

I was wondering if it's possible to let Varnish to cache based on Backend
server response header?

For example, if Backend response header has "cache_ttl = 60", then Varnish
will cache it for 60 seconds.

If the cache_ttl = 0 or is not exist in Backend, Varnish will not cache for
that object at all.

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130312/4248d9b7/attachment.html>

From lkarsten at varnish-software.com  Tue Mar 12 11:47:09 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 12 Mar 2013 12:47:09 +0100
Subject: Varnish to cache accordingly by inspecting Backend response header
In-Reply-To: <CAAct57wxkVnJEDNehc+ZwGCVQHgB8g4nThNTN8hx0NayxZ7fxw@mail.gmail.com>
References: <CAAct57wxkVnJEDNehc+ZwGCVQHgB8g4nThNTN8hx0NayxZ7fxw@mail.gmail.com>
Message-ID: <20130312114708.GI29447@immer.varnish-software.com>

On Tue, Mar 12, 2013 at 07:38:44PM +0800, Kelvin Loke wrote:
> I was wondering if it's possible to let Varnish to cache based on Backend
> server response header?
> For example, if Backend response header has "cache_ttl = 60", then Varnish
> will cache it for 60 seconds.

This is the default behaviour, except that the header is named Cache-Control:
max-age=NN.

> If the cache_ttl = 0 or is not exist in Backend, Varnish will not cache for
> that object at all.

If an object doesn't exist, the default behaviour is to cache the 404 for
120 seconds. If you want to change that, you can set beresp.ttl in vcl_fetch.


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From r at roze.lv  Tue Mar 12 11:51:14 2013
From: r at roze.lv (Reinis Rozitis)
Date: Tue, 12 Mar 2013 13:51:14 +0200
Subject: Varnish to cache accordingly by inspecting Backend response header
In-Reply-To: <CAAct57wxkVnJEDNehc+ZwGCVQHgB8g4nThNTN8hx0NayxZ7fxw@mail.gmail.com>
References: <CAAct57wxkVnJEDNehc+ZwGCVQHgB8g4nThNTN8hx0NayxZ7fxw@mail.gmail.com>
Message-ID: <8648082D2BA24BBB8C9F273585802B4F@NeiRoze>

> I was wondering if it's possible to let Varnish to cache based on Backend 
> server response header?
> For example, if Backend response header has "cache_ttl = 60", then Varnish 
> will cache it for 60 seconds.
> If the cache_ttl = 0 or is not exist in Backend, Varnish will not cache 
> for that object at all.

This is how varnish works by default  - it respects the  max-age, 
Cache-Control fields and Expires headers.

You can override it in vcl_fetch by changing the beresp.ttl. For example:

if (beresp.ttl < 5m) {
         set beresp.ttl = 5m;
}



rr 




From souvikbhattacharyas at gmail.com  Tue Mar 12 18:58:19 2013
From: souvikbhattacharyas at gmail.com (Souvik Bhattacharya)
Date: Wed, 13 Mar 2013 00:28:19 +0530
Subject: -u tag in configuration and multi instance varnish communication
Message-ID: <CAAOnAtrOD+CtBFMn1XV7CYM9eatoREjwF-t28Z3acYD0gOpXag@mail.gmail.com>

Hi,
    During my Varnish set up I have few questions on the same.
 a) During set up I found a tag '-u' inside DAEMON_OPTS set up. So,
could you tell me what -u actually define in Varnish.

DAEMON_OPTS="-a 192.170.2.1:80,[2001:db8::2]:80 \             -T
localhost:6082 \             -f /etc/varnish/default.vcl \           *
 -u varnish -g varnish **\*             -s
file,/var/lib/varnish/varnish_storage.bin,4G"

b) According to my system need I have two server (Load Balancing)
running two different instance of Varnish.
Now how to configure the same in a way that they will talk with each
other regarding the cache.

For Example :

Server A served a request and stored the data in cache of Varnish of Server
A not Server B has the same request. Now Server B will check is the data
available in it's cache if not then is the data available in Server A 's
cache.

Or another one approach can be taken when Server A stored a data in it's
cache it will also update the cache of Server B.


Really looking for your help on the same.

-- 
Thanks & Regards,

Souvik Bhattacharya
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130313/4a5b2bc5/attachment.html>

From dridi.boukelmoune at zenika.com  Tue Mar 12 21:39:12 2013
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Tue, 12 Mar 2013 22:39:12 +0100
Subject: -u tag in configuration and multi instance varnish communication
In-Reply-To: <CAAOnAtrOD+CtBFMn1XV7CYM9eatoREjwF-t28Z3acYD0gOpXag@mail.gmail.com>
References: <CAAOnAtrOD+CtBFMn1XV7CYM9eatoREjwF-t28Z3acYD0gOpXag@mail.gmail.com>
Message-ID: <CABtDKm62ARoRheV7FGKefYnkpe=nYEgYZMU07LN1j5FXk41G6g@mail.gmail.com>

Hi,

a)
>From the varnishd(1) man:
-u user
Specifies the name of an unprivileged user to which the child process
should switch before it starts accepting connections. This is a
shortcut for specifying the user run- time parameter.
If specifying both a user and a group, the user should be specified first.

The varnish user and group were probably created during the install if
you used native packages.

b)
This doesn't answer what you ask, but it might help:
https://www.varnish-cache.org/trac/wiki/VCLExampleHashIgnoreBusy

Best Regards,
Dridi

On Tue, Mar 12, 2013 at 7:58 PM, Souvik Bhattacharya
<souvikbhattacharyas at gmail.com> wrote:
>
> Hi,
>     During my Varnish set up I have few questions on the same.
>  a) During set up I found a tag '-u' inside DAEMON_OPTS set up. So, could
> you tell me what -u actually define in Varnish.
>
>
> DAEMON_OPTS="-a 192.170.2.1:80,[2001:db8::2]:80 \
>              -T localhost:6082 \
>              -f /etc/varnish/default.vcl \
>              -u varnish -g varnish \
>              -s file,/var/lib/varnish/varnish_storage.bin,4G"
>
> b) According to my system need I have two server (Load Balancing) running
> two different instance of Varnish.
> Now how to configure the same in a way that they will talk with each other
> regarding the cache.
>
>
> For Example :
>
> Server A served a request and stored the data in cache of Varnish of Server
> A not Server B has the same request. Now Server B will check is the data
> available in it's cache if not then is the data available in Server A 's
> cache.
>
> Or another one approach can be taken when Server A stored a data in it's
> cache it will also update the cache of Server B.
>
>
> Really looking for your help on the same.
>
> --
> Thanks & Regards,
>
> Souvik Bhattacharya
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From juanandrade1 at gmail.com  Wed Mar 13 00:04:39 2013
From: juanandrade1 at gmail.com (Juan Andrade)
Date: Tue, 12 Mar 2013 17:04:39 -0700
Subject: Varnish logging problems
Message-ID: <CAEcTaTUwkbJU4b9EEdjMGPjnojcLYSSc+OZz1aw9NGVa7ieU4g@mail.gmail.com>

Varnishlog works fine but varnishncsa does not output anything.  Also tried
std.syslog but messages are empty

Here is my config:

sub vcl_recv {
    if (req.http.host) {
        std.syslog(191, "vcl_recv(" + req.xid + ") Req: " +
                    req.request + " http://" + req.http.host +
                    req.url + " " + req.proto);
        set req.backend = stage;
    }
}

Log messages: No message as set above
2013-03-12T16:55:23.725466-07:00 stage-cache1 varnishd[10191]:
2013-03-12T16:56:43.413369-07:00 stage-cache1 varnishd[10530]:

Running varnish version
varnishd (varnish-trunk revision 5582b35)

Anybody run into this problem?  Thanks for any help.

Juan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130312/a13fe1f8/attachment.html>

From nkinkade at creativecommons.org  Wed Mar 13 19:56:36 2013
From: nkinkade at creativecommons.org (Nathan Kinkade)
Date: Wed, 13 Mar 2013 15:56:36 -0400
Subject: Varnish logging problems
In-Reply-To: <CAEcTaTUwkbJU4b9EEdjMGPjnojcLYSSc+OZz1aw9NGVa7ieU4g@mail.gmail.com>
References: <CAEcTaTUwkbJU4b9EEdjMGPjnojcLYSSc+OZz1aw9NGVa7ieU4g@mail.gmail.com>
Message-ID: <CAF5oByxU53suhhCuRAZ=PRY2zebhDg8iArOCxnRByE-BDxDTTg@mail.gmail.com>

Have you tried using the -r option to make varnishncsa read from a
file instead of shared memory?  It could be one way to help narrow
down the problem.  In our setup varnishlog logs to
/var/log/varnish/varnish.log, so I can always use:

# varnishncsa -r /var/log/varnish/varnish.log

... this effectively gives me the same output as varnishncsa without
any options.

Nathan

On Tue, Mar 12, 2013 at 8:04 PM, Juan Andrade <juanandrade1 at gmail.com> wrote:
> Varnishlog works fine but varnishncsa does not output anything.  Also tried
> std.syslog but messages are empty
>
> Here is my config:
>
> sub vcl_recv {
>     if (req.http.host) {
>         std.syslog(191, "vcl_recv(" + req.xid + ") Req: " +
>                     req.request + " http://" + req.http.host +
>                     req.url + " " + req.proto);
>         set req.backend = stage;
>     }
> }
>
> Log messages: No message as set above
> 2013-03-12T16:55:23.725466-07:00 stage-cache1 varnishd[10191]:
> 2013-03-12T16:56:43.413369-07:00 stage-cache1 varnishd[10530]:
>
> Running varnish version
> varnishd (varnish-trunk revision 5582b35)
>
> Anybody run into this problem?  Thanks for any help.
>
> Juan
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From juanandrade1 at gmail.com  Wed Mar 13 21:39:51 2013
From: juanandrade1 at gmail.com (Juan Andrade)
Date: Wed, 13 Mar 2013 14:39:51 -0700
Subject: Varnish logging problems
In-Reply-To: <CAF5oByxU53suhhCuRAZ=PRY2zebhDg8iArOCxnRByE-BDxDTTg@mail.gmail.com>
References: <CAEcTaTUwkbJU4b9EEdjMGPjnojcLYSSc+OZz1aw9NGVa7ieU4g@mail.gmail.com>
 <CAF5oByxU53suhhCuRAZ=PRY2zebhDg8iArOCxnRByE-BDxDTTg@mail.gmail.com>
Message-ID: <CAEcTaTX28GOni01R=v=HMfPv4_+TxG5zuyVSswwU_T5mGm+uxw@mail.gmail.com>

Hello Nathan,
So I tried that and no output even reading from file

varnishlog -w /tmp/varnishlog.log  // This writes

varnishncsa -r /tmp/varnishlog // No output

On Wed, Mar 13, 2013 at 12:56 PM, Nathan Kinkade <
nkinkade at creativecommons.org> wrote:

> Have you tried using the -r option to make varnishncsa read from a
> file instead of shared memory?  It could be one way to help narrow
> down the problem.  In our setup varnishlog logs to
> /var/log/varnish/varnish.log, so I can always use:
>
> # varnishncsa -r /var/log/varnish/varnish.log
>
> ... this effectively gives me the same output as varnishncsa without
> any options.
>
> Nathan
>
> On Tue, Mar 12, 2013 at 8:04 PM, Juan Andrade <juanandrade1 at gmail.com>
> wrote:
> > Varnishlog works fine but varnishncsa does not output anything.  Also
> tried
> > std.syslog but messages are empty
> >
> > Here is my config:
> >
> > sub vcl_recv {
> >     if (req.http.host) {
> >         std.syslog(191, "vcl_recv(" + req.xid + ") Req: " +
> >                     req.request + " http://" + req.http.host +
> >                     req.url + " " + req.proto);
> >         set req.backend = stage;
> >     }
> > }
> >
> > Log messages: No message as set above
> > 2013-03-12T16:55:23.725466-07:00 stage-cache1 varnishd[10191]:
> > 2013-03-12T16:56:43.413369-07:00 stage-cache1 varnishd[10530]:
> >
> > Running varnish version
> > varnishd (varnish-trunk revision 5582b35)
> >
> > Anybody run into this problem?  Thanks for any help.
> >
> > Juan
> >
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130313/12c65f04/attachment.html>

From hugues at betabrand.com  Wed Mar 13 21:40:03 2013
From: hugues at betabrand.com (Hugues Alary)
Date: Wed, 13 Mar 2013 14:40:03 -0700
Subject: return(restart); and ESI
Message-ID: <CAL_utwY8HDh_xTne1DBjU9_TPghSDY_9rSTXQTNW-sC5r3iSgg@mail.gmail.com>

Hi,

I'm unclear about the behavior that Varnish will have if there's a
return(restart); happening when Varnish is processing an ESI include.

If I do a return(restart); in vcl_fetch(), when req.esi_level > 0 (meaning
that the current request is an <esi:include src="" /> being processed by
Varnish), will Varnish restart the *parent *request, or will it restart
only the <esi:include /> ?

My gut tells me that Varnish will only restart the esi processing, can
someone confirm that?

In the case the previous statement is true, is there any way I could
instruct Varnish to restart the *parent*?
*
*
I feel like restarting the parent request should be possible considering
that when Varnish processes an <esi:include />, the req.esi_level > 0,
meaning that Varnish is aware that this request has a parent. Although it
certainly is way more complicated than that.

Thanks for your help!
-Hugues
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130313/4e8538e6/attachment.html>

From Raul.Rangel at disney.com  Wed Mar 13 22:05:39 2013
From: Raul.Rangel at disney.com (Rangel, Raul)
Date: Wed, 13 Mar 2013 15:05:39 -0700
Subject: return(restart); and ESI
In-Reply-To: <CAL_utwY8HDh_xTne1DBjU9_TPghSDY_9rSTXQTNW-sC5r3iSgg@mail.gmail.com>
References: <CAL_utwY8HDh_xTne1DBjU9_TPghSDY_9rSTXQTNW-sC5r3iSgg@mail.gmail.com>
Message-ID: <2465AAEEC8B8A242B26ED5F44BCA805F260D8DD62A@SM-CALA-VXMB04A.swna.wdpr.disney.com>

A restart in an esi will restart only the ESI. There is no way to restart the parent. The parent could have been sent to the client already so a restart is impossible.

Raul

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Hugues Alary
Sent: Wednesday, March 13, 2013 3:40 PM
To: varnish-misc at varnish-cache.org
Subject: return(restart); and ESI

Hi,

I'm unclear about the behavior that Varnish will have if there's a return(restart); happening when Varnish is processing an ESI include.

If I do a return(restart); in vcl_fetch(), when req.esi_level > 0 (meaning that the current request is an <esi:include src="" /> being processed by Varnish), will Varnish restart the parent request, or will it restart only the <esi:include /> ?

My gut tells me that Varnish will only restart the esi processing, can someone confirm that?

In the case the previous statement is true, is there any way I could instruct Varnish to restart the parent?

I feel like restarting the parent request should be possible considering that when Varnish processes an <esi:include />, the req.esi_level > 0, meaning that Varnish is aware that this request has a parent. Although it certainly is way more complicated than that.

Thanks for your help!
-Hugues
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130313/49d3eb71/attachment.html>

From souvikbhattacharyas at gmail.com  Thu Mar 14 07:39:01 2013
From: souvikbhattacharyas at gmail.com (Souvik Bhattacharya)
Date: Thu, 14 Mar 2013 13:09:01 +0530
Subject: How PURGE works
Message-ID: <CAAOnAtqdtR6rzoa2mikxBf4hzG61xCf9ZZs2dNH4RkEzpt45LA@mail.gmail.com>

Hi,
    I am new to Varnish and during cache invalidation I have a confusion.

a) As per the guide line when I will write 'purge' content management
system send a special HTTP request to Varnish. So, is this request sending
done automatically or my own program need to send a request with
request.method ='PURGE'

b) In the below program to purge a object VCL check is the request is with
'PURGE'. But if I am initiating purge for the first time then how the
req.request will be 'PURGE'?

sub vcl_recv {
        if (req.request == "PURGE") {
                if (!client.ip ~ purge) {
                        error 405 "Not allowed.";
                }
                return (lookup);
        }
}

sub vcl_hit {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged.";
        }
}

sub vcl_miss {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged.";
        }


Please help me to understand the same or if you have any better view please
share the same.
-- 
Thanks & Regards,

Souvik Bhattacharya
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130314/26f064d8/attachment.html>

From nkinkade at creativecommons.org  Thu Mar 14 14:49:53 2013
From: nkinkade at creativecommons.org (Nathan Kinkade)
Date: Thu, 14 Mar 2013 10:49:53 -0400
Subject: Varnish logging problems
In-Reply-To: <CAEcTaTX28GOni01R=v=HMfPv4_+TxG5zuyVSswwU_T5mGm+uxw@mail.gmail.com>
References: <CAEcTaTUwkbJU4b9EEdjMGPjnojcLYSSc+OZz1aw9NGVa7ieU4g@mail.gmail.com>
 <CAF5oByxU53suhhCuRAZ=PRY2zebhDg8iArOCxnRByE-BDxDTTg@mail.gmail.com>
 <CAEcTaTX28GOni01R=v=HMfPv4_+TxG5zuyVSswwU_T5mGm+uxw@mail.gmail.com>
Message-ID: <CAF5oByy+tvfA=yATLZX2qna-as+yqDpzNJZU00ngigKzhBrtoQ@mail.gmail.com>

If /tmp/varnishlog.log is actually growing in size and yet using the
-r flag in varnishncsa doesn't work, then I wouldn't know what to say.

Nathan

On Wed, Mar 13, 2013 at 5:39 PM, Juan Andrade <juanandrade1 at gmail.com> wrote:
> Hello Nathan,
> So I tried that and no output even reading from file
>
> varnishlog -w /tmp/varnishlog.log  // This writes
>
> varnishncsa -r /tmp/varnishlog // No output
>
> On Wed, Mar 13, 2013 at 12:56 PM, Nathan Kinkade
> <nkinkade at creativecommons.org> wrote:
>>
>> Have you tried using the -r option to make varnishncsa read from a
>> file instead of shared memory?  It could be one way to help narrow
>> down the problem.  In our setup varnishlog logs to
>> /var/log/varnish/varnish.log, so I can always use:
>>
>> # varnishncsa -r /var/log/varnish/varnish.log
>>
>> ... this effectively gives me the same output as varnishncsa without
>> any options.
>>
>> Nathan
>>
>> On Tue, Mar 12, 2013 at 8:04 PM, Juan Andrade <juanandrade1 at gmail.com>
>> wrote:
>> > Varnishlog works fine but varnishncsa does not output anything.  Also
>> > tried
>> > std.syslog but messages are empty
>> >
>> > Here is my config:
>> >
>> > sub vcl_recv {
>> >     if (req.http.host) {
>> >         std.syslog(191, "vcl_recv(" + req.xid + ") Req: " +
>> >                     req.request + " http://" + req.http.host +
>> >                     req.url + " " + req.proto);
>> >         set req.backend = stage;
>> >     }
>> > }
>> >
>> > Log messages: No message as set above
>> > 2013-03-12T16:55:23.725466-07:00 stage-cache1 varnishd[10191]:
>> > 2013-03-12T16:56:43.413369-07:00 stage-cache1 varnishd[10530]:
>> >
>> > Running varnish version
>> > varnishd (varnish-trunk revision 5582b35)
>> >
>> > Anybody run into this problem?  Thanks for any help.
>> >
>> > Juan
>> >
>> > _______________________________________________
>> > varnish-misc mailing list
>> > varnish-misc at varnish-cache.org
>> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>



From pprocacci at datapipe.com  Thu Mar 14 16:44:01 2013
From: pprocacci at datapipe.com (Paul A. Procacci)
Date: Thu, 14 Mar 2013 11:44:01 -0500
Subject: How PURGE works
In-Reply-To: <CAAOnAtqdtR6rzoa2mikxBf4hzG61xCf9ZZs2dNH4RkEzpt45LA@mail.gmail.com>
References: <CAAOnAtqdtR6rzoa2mikxBf4hzG61xCf9ZZs2dNH4RkEzpt45LA@mail.gmail.com>
Message-ID: <20130314164401.GP63316@nat.myhome>

> a) As per the guide line when I will write 'purge' content management
> system send a special HTTP request to Varnish. So, is this request sending
> done automatically or my own program need to send a request with
> request.method ='PURGE'

Whether the request is done automatically or not depends on the CMS.  I assume
the CMS has built in functionality to make the request.  You'll need to read
the documentation that came with the CMS to determine how it's handled.
Assume it's done for you though.

> b) In the below program to purge a object VCL check is the request is with
> 'PURGE'. But if I am initiating purge for the first time then how the
> req.request will be 'PURGE'?

You can simulate a purge by sending the following bytes to varnish:

PURGE /url HTTP/1.0

You can send this via telnet, or a programming language of your choice, but
ultimately this is how request.method gets set to 'PURGE'.

~Paul

________________________________

This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.



From souvikbhattacharyas at gmail.com  Thu Mar 14 18:27:20 2013
From: souvikbhattacharyas at gmail.com (Souvik Bhattacharya)
Date: Thu, 14 Mar 2013 23:57:20 +0530
Subject: How PURGE works
In-Reply-To: <20130314164401.GP63316@nat.myhome>
References: <CAAOnAtqdtR6rzoa2mikxBf4hzG61xCf9ZZs2dNH4RkEzpt45LA@mail.gmail.com>
 <20130314164401.GP63316@nat.myhome>
Message-ID: <CAAOnAtoBNezPUeCbfGqqEeUXp6V=g-=1NyyAh3ukMjWtyNy80w@mail.gmail.com>

Hi,
    That means, whenever I need to request for "PURGE" I need to fire the
Url by my self through container. Then it will be a extra round trip of
client to server. Is there anyway to avoid that

On Thu, Mar 14, 2013 at 10:14 PM, Paul A. Procacci
<pprocacci at datapipe.com>wrote:

> > a) As per the guide line when I will write 'purge' content management
> > system send a special HTTP request to Varnish. So, is this request
> sending
> > done automatically or my own program need to send a request with
> > request.method ='PURGE'
>
> Whether the request is done automatically or not depends on the CMS.  I
> assume
> the CMS has built in functionality to make the request.  You'll need to
> read
> the documentation that came with the CMS to determine how it's handled.
> Assume it's done for you though.
>
> > b) In the below program to purge a object VCL check is the request is
> with
> > 'PURGE'. But if I am initiating purge for the first time then how the
> > req.request will be 'PURGE'?
>
> You can simulate a purge by sending the following bytes to varnish:
>
> PURGE /url HTTP/1.0
>
> You can send this via telnet, or a programming language of your choice, but
> ultimately this is how request.method gets set to 'PURGE'.
>
> ~Paul
>
> ________________________________
>
> This message may contain confidential or privileged information. If you
> are not the intended recipient, please advise us immediately and delete
> this message. See http://www.datapipe.com/legal/email_disclaimer/ for
> further information on confidentiality and the risks of non-secure
> electronic communication. If you cannot access these links, please notify
> us by reply message and we will send the contents to you.
>



-- 
Thanks & Regards,

Souvik Bhattacharya
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130314/ec5a8431/attachment.html>

From smwood4 at gmail.com  Thu Mar 14 18:37:05 2013
From: smwood4 at gmail.com (Stephen Wood)
Date: Thu, 14 Mar 2013 11:37:05 -0700
Subject: How PURGE works
In-Reply-To: <CAAOnAtoBNezPUeCbfGqqEeUXp6V=g-=1NyyAh3ukMjWtyNy80w@mail.gmail.com>
References: <CAAOnAtqdtR6rzoa2mikxBf4hzG61xCf9ZZs2dNH4RkEzpt45LA@mail.gmail.com>
 <20130314164401.GP63316@nat.myhome>
 <CAAOnAtoBNezPUeCbfGqqEeUXp6V=g-=1NyyAh3ukMjWtyNy80w@mail.gmail.com>
Message-ID: <CA+pzux-3xFw9-3EYESYx9mgW5ZYUniX1TrN=K2z9jtPMx234ug@mail.gmail.com>

As stated earlier you can use telnet/nc. There may be a better option for
what you're attempting. For example, have you tried changing the TTL of
your objects? It's possible to avoid purges altogether.

On Thu, Mar 14, 2013 at 11:27 AM, Souvik Bhattacharya <
souvikbhattacharyas at gmail.com> wrote:

> Hi,
>     That means, whenever I need to request for "PURGE" I need to fire the
> Url by my self through container. Then it will be a extra round trip of
> client to server. Is there anyway to avoid that
>
>
> On Thu, Mar 14, 2013 at 10:14 PM, Paul A. Procacci <pprocacci at datapipe.com
> > wrote:
>
>> > a) As per the guide line when I will write 'purge' content management
>> > system send a special HTTP request to Varnish. So, is this request
>> sending
>> > done automatically or my own program need to send a request with
>> > request.method ='PURGE'
>>
>> Whether the request is done automatically or not depends on the CMS.  I
>> assume
>> the CMS has built in functionality to make the request.  You'll need to
>> read
>> the documentation that came with the CMS to determine how it's handled.
>> Assume it's done for you though.
>>
>> > b) In the below program to purge a object VCL check is the request is
>> with
>> > 'PURGE'. But if I am initiating purge for the first time then how the
>> > req.request will be 'PURGE'?
>>
>> You can simulate a purge by sending the following bytes to varnish:
>>
>> PURGE /url HTTP/1.0
>>
>> You can send this via telnet, or a programming language of your choice,
>> but
>> ultimately this is how request.method gets set to 'PURGE'.
>>
>> ~Paul
>>
>> ________________________________
>>
>> This message may contain confidential or privileged information. If you
>> are not the intended recipient, please advise us immediately and delete
>> this message. See http://www.datapipe.com/legal/email_disclaimer/ for
>> further information on confidentiality and the risks of non-secure
>> electronic communication. If you cannot access these links, please notify
>> us by reply message and we will send the contents to you.
>>
>
>
>
> --
> Thanks & Regards,
>
> Souvik Bhattacharya
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130314/1c16049b/attachment.html>

From sean at monkeysnatchbanana.com  Fri Mar 15 14:11:34 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Fri, 15 Mar 2013 10:11:34 -0400
Subject: vcl_miss and purge question
Message-ID: <CABm-Wmf0AVLrPOaxbKN_6=3Xch_N6h2oZN9RrcT-wuM9c3k1EQ@mail.gmail.com>

Using varnish 3. Our configuration works but when I was putting it
together, I was learning from docs and blog posts. I have:

sub vcl_hit {
  # 'manual' purge
  if (req.request == "PURGE") {
    purge;
    error 200 "Purged.";
  }
}

sub vcl_miss {
  # 'manual' purge
  if (req.request == "PURGE") {
    purge;
    error 404 "Not in cache.";
  }
}

Does the purge in vcl_miss need to be there?
Could it just be:

sub vcl_miss {
  # 'manual' purge
  if (req.request == "PURGE") {
    error 404 "Not in cache.";
  }
}

?

Thanks,
Sean

-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130315/3d427b99/attachment.html>

From Raul.Rangel at disney.com  Fri Mar 15 14:13:10 2013
From: Raul.Rangel at disney.com (Rangel, Raul)
Date: Fri, 15 Mar 2013 07:13:10 -0700
Subject: vcl_miss and purge question
In-Reply-To: <CABm-Wmf0AVLrPOaxbKN_6=3Xch_N6h2oZN9RrcT-wuM9c3k1EQ@mail.gmail.com>
References: <CABm-Wmf0AVLrPOaxbKN_6=3Xch_N6h2oZN9RrcT-wuM9c3k1EQ@mail.gmail.com>
Message-ID: <2465AAEEC8B8A242B26ED5F44BCA805F260D99D460@SM-CALA-VXMB04A.swna.wdpr.disney.com>

You need it in the vcl_miss because you need to clear our Very objects as well.

Raul

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Sean Allen
Sent: Friday, March 15, 2013 8:12 AM
To: varnish-misc at varnish-cache.org
Subject: vcl_miss and purge question

Using varnish 3. Our configuration works but when I was putting it together, I was learning from docs and blog posts. I have:

sub vcl_hit {
  # 'manual' purge
  if (req.request == "PURGE") {
    purge;
    error 200 "Purged.";
  }
}

sub vcl_miss {
  # 'manual' purge
  if (req.request == "PURGE") {
    purge;
    error 404 "Not in cache.";
  }
}

Does the purge in vcl_miss need to be there?
Could it just be:

sub vcl_miss {
  # 'manual' purge
  if (req.request == "PURGE") {
    error 404 "Not in cache.";
  }
}

?

Thanks,
Sean

--

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130315/21967a0c/attachment.html>

From sean at monkeysnatchbanana.com  Fri Mar 15 14:31:21 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Fri, 15 Mar 2013 10:31:21 -0400
Subject: vcl_miss and purge question
In-Reply-To: <2465AAEEC8B8A242B26ED5F44BCA805F260D99D460@SM-CALA-VXMB04A.swna.wdpr.disney.com>
References: <CABm-Wmf0AVLrPOaxbKN_6=3Xch_N6h2oZN9RrcT-wuM9c3k1EQ@mail.gmail.com>
 <2465AAEEC8B8A242B26ED5F44BCA805F260D99D460@SM-CALA-VXMB04A.swna.wdpr.disney.com>
Message-ID: <CABm-WmeG5KT30MetHEn+5J75aEDKNrMR5R5dXKdT9RgvxubC=w@mail.gmail.com>

Thank Raul.

I thought there was a reason but I couldn't remember one.


On Fri, Mar 15, 2013 at 10:13 AM, Rangel, Raul <Raul.Rangel at disney.com>wrote:

> You need it in the vcl_miss because you need to clear our Very objects as
> well.****
>
> ** **
>
> Raul****
>
> ** **
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Sean Allen
> *Sent:* Friday, March 15, 2013 8:12 AM
> *To:* varnish-misc at varnish-cache.org
> *Subject:* vcl_miss and purge question****
>
> ** **
>
> Using varnish 3. Our configuration works but when I was putting it
> together, I was learning from docs and blog posts. I have:****
>
> ** **
>
> sub vcl_hit {****
>
>   # 'manual' purge****
>
>   if (req.request == "PURGE") {****
>
>     purge;****
>
>     error 200 "Purged.";****
>
>   }****
>
> }****
>
> ** **
>
> sub vcl_miss {****
>
>   # 'manual' purge****
>
>   if (req.request == "PURGE") {****
>
>     purge;****
>
>     error 404 "Not in cache.";****
>
>   }****
>
> }****
>
> ** **
>
> Does the purge in vcl_miss need to be there?****
>
> Could it just be:****
>
> ** **
>
> sub vcl_miss {****
>
>   # 'manual' purge****
>
>   if (req.request == "PURGE") {****
>
>     error 404 "Not in cache.";****
>
>   }****
>
> }****
>
> ** **
>
> ?****
>
> ** **
>
> Thanks,****
>
> Sean****
>
> ** **
>
> -- ****
>
> ** **
>
> Ce n'est pas une signature****
>



-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130315/382e2684/attachment.html>

From simvirus at gmail.com  Sat Mar 16 19:29:46 2013
From: simvirus at gmail.com (Sim)
Date: Sat, 16 Mar 2013 20:29:46 +0100
Subject: Varnish config (any tips?)
Message-ID: <CAD=Xf-WJ=nW8uSxF3OEHSowb1O8OpJPdLxqsPpJsOzMe4LNo_Q@mail.gmail.com>

Dear users,
this is my Varnish config for use it at "100%" (to cache all) for about "5"
min.
Have you any comments or tips for me?
Are there other way or optimizations?
Why Varnish does not keep everything in cache by default?

Thanks
Regards


##############################################
backend [...]

##############################################
sub vcl_recv {

     if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For =
                req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
     }

     # Direct connections from 1.2.3.4 IP
     if (client.ip == "1.2.3.4"{
         return (pipe);
     }

     # CACHE ALL!
     if (req.request == "GET" || req.request == "HEAD") {
        return (lookup);
     }

     if (req.request != "GET" &&
         req.request != "HEAD" &&
         req.request != "PUT" &&
         req.request != "POST" &&
         req.request != "TRACE" &&
         req.request != "OPTIONS" &&
         req.request != "DELETE") {
         /* Non-RFC2616 or CONNECT which is weird. */
         return (pipe);
     }
     if (req.request != "GET" && req.request != "HEAD") {
         /* We only deal with GET and HEAD by default */
         return (pass);
     }
     if (req.http.Authorization || req.http.Cookie) {
         /* Not cacheable by default */
         return (pass);
     }
     return (lookup);
 }

##############################################
sub vcl_pipe {
     set bereq.http.connection = "close";
     return (pipe);
}

##############################################
sub vcl_hit {
    return (deliver);
}

##############################################
sub vcl_miss {
    return (fetch);
}

##############################################
sub vcl_fetch {
    if (beresp.ttl <= 0s ||
        beresp.http.Set-Cookie ||
        beresp.http.Vary == "*") {
                /*
                 * Mark as "Hit-For-Pass" for the next 5 minutes
                 */
                set beresp.ttl = 300s;
                return (hit_for_pass);
    }
    return (deliver);
}

##############################################
sub vcl_deliver {
   # Check if working
   if (obj.hits > 0) {
     set resp.http.X-Cache = "HIT";
  } else {
     set resp.http.X-Cache = "MISS";
  }
   return (deliver);
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130316/a1c47e9c/attachment.html>

From radecki.rafal at gmail.com  Tue Mar 19 12:51:50 2013
From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=)
Date: Tue, 19 Mar 2013 13:51:50 +0100
Subject: Varnish 3.0.3, CentOS 6.3 - how to properly monitor it with Nagios?
Message-ID: <CAHd9_iQF=KHmZK0x9sxEKzk_XpYH=cHqvtYOQ_+f8jz+aVRW2A@mail.gmail.com>

Hi All.

I am searching information on how to properly monitor varnish
parameters with Nagios+nrpe.
I would like to monitor backend health and various parameters
(client_conn, n_lru_moved, ... ). I know that I can use:

- https://github.com/varnish/varnish-nagios -> but it only gives me
cumulative number for every parameter

./check_varnish -p client_conn
VARNISH UNKNOWN: Client connections accepted (8,361)|client_conn=8361
and not a per second value which is what I would like to have;

- varnishstat -1 -> but it also gives me only cumulative number

varnishstat -f client_conn -1
client_conn               8409         0.49 Client connections accepted
(0.49 - the average since varnish startup also not usable, rather
infromational only).

I also know that there is no stats command in varnishadm for varnish 3
so I cannot use it.
I would like to monitor my parameters' per second (or other timeperiod
defined) change, do you know any solutions/scripts/tools?
Is my only option to write a script which (for example) calls
varnishstat -1 every second for some time and then computes the
average?

Best regards,
Rafal Radecki.



From sean at monkeysnatchbanana.com  Wed Mar 20 01:39:41 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Tue, 19 Mar 2013 21:39:41 -0400
Subject: Swapping & High Load
Message-ID: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>

We have a varnish 3 server that was performing fine that we just increased
malloc size to as we started pulling a lot more data through it.

Malloc is 6 gigs.
We gave the entire machine 7.5 gigs

And we started using a couple gigs of swap being used which lead to load
average rapidly climbing.

Does Varnish need additional memory beyond what is allocated via malloc?
For all our other services that are running on this version of cent/os, 1.5
gigs for OS has been fine.

Specific version of varnish is:

(varnish-3.0.3 revision 9e6a70f)

sysconfig/varnish is:

# -*- mode: conf-unix -*-
#
# Configuration file for varnish
#
# This file is managed by puppet.
# Local changes will be reverted.
#
# /etc/init.d/varnish expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#

# Maximum number of open files (for ulimit -n)
NFILES=131072

# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000

# Maximum size of corefile (for ulimit -c). Default in Fedora is 0
# DAEMON_COREFILE_LIMIT="unlimited"

# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request.  Use a
# fixed-size cache file.
#
# Without "ban_lurker_sleep," nothing banned from the cache ever gets
evicted.
#
DAEMON_OPTS="-a :6081 \
             -T :6082 \
             -f /etc/varnish/default.vcl \
             -u varnish -g varnish \
             -p thread_pool_min=500 \
             -p thread_pool_max=5000 \
             -s malloc,6G \
             -p thread_pools=2 \
             -p thread_pool_add_delay=1 \
             -p ban_lurker_sleep=60s"

If varnish/default.vcl is needed, please let me know.

Running on a vmware vm with 2 cores.


-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130319/db3f245d/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 01:46:37 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Tue, 19 Mar 2013 21:46:37 -0400
Subject: high iowait
Message-ID: <CABm-WmeAF015u=KZ6UJMdYGHKR0JzZYpv90-EKLusnA7w5+2Dg@mail.gmail.com>

One of our varrnish servers is spending about 40-50% of its time in iowait.
Is this just from the varnishlog getting written? Our IO performance is not
great and I'm looking to be able to get the amount of time we are spending
doing IO down. This occurs even when everything is running nicely in memory
and we aren't overflowing into swap ( which was detailed in a different
email as I think they might be different issues ).

varnishd (varnish-3.0.3 revision 9e6a70f)

sysconfig/varnish

# -*- mode: conf-unix -*-
#
# Configuration file for varnish
#
# This file is managed by puppet.
# Local changes will be reverted.
#
# /etc/init.d/varnish expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#

# Maximum number of open files (for ulimit -n)
NFILES=131072

# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000

# Maximum size of corefile (for ulimit -c). Default in Fedora is 0
# DAEMON_COREFILE_LIMIT="unlimited"

# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request.  Use a
# fixed-size cache file.
#
# Without "ban_lurker_sleep," nothing banned from the cache ever gets
evicted.
#
DAEMON_OPTS="-a :6081 \
             -T :6082 \
             -f /etc/varnish/default.vcl \
             -u varnish -g varnish \
             -p thread_pool_min=500 \
             -p thread_pool_max=5000 \
             -s malloc,6G \
             -p thread_pools=2 \
             -p thread_pool_add_delay=1 \
             -p ban_lurker_sleep=60s"

-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130319/28a8339c/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 03:41:54 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Tue, 19 Mar 2013 23:41:54 -0400
Subject: Swapping & High Load
In-Reply-To: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
Message-ID: <CABm-Wmed1vAeJgcGz5Ju61-ywX75fwB4ZnTO+4t-Wv54Si9rrg@mail.gmail.com>

We upped OS ram and started running a stress test. It shows how we ended up
swapping. I didn't appreciate how much extra ram beyond the malloc that
varnish would need. What is the best source of documentation on memory
usage?

What is the best source for learning about the memory used across threads
and thread pools?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130319/a6a8c68a/attachment.html>

From perbu at varnish-software.com  Wed Mar 20 12:06:53 2013
From: perbu at varnish-software.com (Per Buer)
Date: Wed, 20 Mar 2013 13:06:53 +0100
Subject: Swapping & High Load
In-Reply-To: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
Message-ID: <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>

On Wed, Mar 20, 2013 at 2:39 AM, Sean Allen <sean at monkeysnatchbanana.com>wrote:

>
> Does Varnish need additional memory beyond what is allocated via malloc?
>

Yes. Malloc is only the backend storage. In addition there are a few data
structures that are global. If I recall correctly the overhead is more or
less 1k per object.


-- 
 <http://www.varnish-software.com/> *Per Buer*
CEO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/7cb46b9e/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 12:28:54 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Wed, 20 Mar 2013 08:28:54 -0400
Subject: Swapping & High Load
In-Reply-To: <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
 <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
Message-ID: <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>

I'm seeing something higher like 5k per.

6g malloc.
600k objects cached right now
varnish has 9.7 gigs of memory resident


On Wed, Mar 20, 2013 at 8:06 AM, Per Buer <perbu at varnish-software.com>wrote:

>
>
>
> On Wed, Mar 20, 2013 at 2:39 AM, Sean Allen <sean at monkeysnatchbanana.com>wrote:
>
>>
>> Does Varnish need additional memory beyond what is allocated via malloc?
>>
>
> Yes. Malloc is only the backend storage. In addition there are a few data
> structures that are global. If I recall correctly the overhead is more or
> less 1k per object.
>
>
> --
>  <http://www.varnish-software.com/> *Per Buer*
> CEO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
>


-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/929c1c37/attachment.html>

From dridi.boukelmoune at zenika.com  Wed Mar 20 12:35:53 2013
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Wed, 20 Mar 2013 13:35:53 +0100
Subject: Swapping & High Load
In-Reply-To: <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
 <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
 <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
Message-ID: <CABtDKm5BWfkEKsCrzuS+Njzdk+cm2heOZJgSKwmaxtBmn454vQ@mail.gmail.com>

Doesn't worker threads also use memory for both their call stacks and
workspace ?
I believe this is not part of the storage, as well as the "naked" memory
footprint of varnishd.

Best Regards,
Dridi

On Wed, Mar 20, 2013 at 1:28 PM, Sean Allen <sean at monkeysnatchbanana.com>wrote:

> I'm seeing something higher like 5k per.
>
> 6g malloc.
> 600k objects cached right now
> varnish has 9.7 gigs of memory resident
>
>
> On Wed, Mar 20, 2013 at 8:06 AM, Per Buer <perbu at varnish-software.com>wrote:
>
>>
>>
>>
>> On Wed, Mar 20, 2013 at 2:39 AM, Sean Allen <sean at monkeysnatchbanana.com>wrote:
>>
>>>
>>> Does Varnish need additional memory beyond what is allocated via malloc?
>>>
>>
>> Yes. Malloc is only the backend storage. In addition there are a few data
>> structures that are global. If I recall correctly the overhead is more or
>> less 1k per object.
>>
>>
>> --
>>  <http://www.varnish-software.com/> *Per Buer*
>> CEO | Varnish Software AS
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>>
>
>
> --
>
> Ce n'est pas une signature
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/ae98ef7b/attachment.html>

From lkarsten at varnish-software.com  Wed Mar 20 12:30:23 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 20 Mar 2013 13:30:23 +0100
Subject: Swapping & High Load
In-Reply-To: <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
 <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
 <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
Message-ID: <20130320123022.GA16203@immer.varnish-software.com>

On Wed, Mar 20, 2013 at 08:28:54AM -0400, Sean Allen wrote:
> I'm seeing something higher like 5k per.
> 6g malloc.
> 600k objects cached right now
> varnish has 9.7 gigs of memory resident

Please post varnishstat.


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From sean at monkeysnatchbanana.com  Wed Mar 20 12:59:23 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Wed, 20 Mar 2013 08:59:23 -0400
Subject: Swapping & High Load
In-Reply-To: <20130320123022.GA16203@immer.varnish-software.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
 <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
 <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
 <20130320123022.GA16203@immer.varnish-software.com>
Message-ID: <CABm-WmfxuWBe4v5QHRVcxVfObNKvF9dZvnCrwC76O6ohvfn_sA@mail.gmail.com>

We ran a batch job last night that moves everything possible through the
cache. Thus the eviction numbers and the currently low hit rate as 'active'
records start to be pulled through at the start of our business day:

0+11:37:21
Hitrate ratio:       10      100     1000
Hitrate avg:     0.5494   0.5375   0.5230

      420499        18.18        10.05 client_conn - Client connections
accepted
    10210075    83.85 244.02 client_req - Client requests received
     2966776    30.31        70.91 cache_hit - Cache hits
     7238285    52.53 173.00 cache_miss - Cache misses
        5832         0.00         0.14 backend_conn - Backend conn. success
          26         0.00         0.00 backend_fail - Backend conn. failures
     7225040    38.39 172.68 backend_reuse - Backend conn. reuses
        5810         0.00         0.14 backend_toolate - Backend conn. was
closed
     7230871    38.39 172.82 backend_recycle - Backend conn. recycles
     7192721    38.39 171.91 fetch_length - Fetch with Length
       38146         0.00         0.91 fetch_chunked - Fetch chunked
         431          .            .   n_sess_mem - N struct sess_mem
          39          .            .   n_sess - N struct sess
      583562          .            .   n_object - N struct object
      583738          .            .   n_objectcore - N struct objectcore
      595457          .            .   n_objecthead - N struct objecthead
        9610          .            .   n_waitinglist - N struct waitinglist
          22          .            .   n_vbc - N struct vbc
        1000          .            .   n_wrk - N worker threads
        1000         0.00         0.02 n_wrk_create - N worker threads
created
          28         0.00         0.00 n_wrk_queued - N queued work requests
           4          .            .   n_backend - N backends
       10609          .            .   n_expired - N expired objects
     6631682          .            .   n_lru_nuked - N LRU nuked objects
      426916          .            .   n_lru_moved - N LRU moved objects
    10348344    83.85 247.33 n_objwrite - Objects sent with write
      420498        17.17        10.05 s_sess - Total Sessions
    10210075    83.85 244.02 s_req - Total Requests
        5014         1.01         0.12 s_pass - Total pass
     7230867    38.39 172.82 s_fetch - Total fetch
  2945392448     23715.74     70394.89 s_hdrbytes - Total header bytes
 82769021882    721855.66   1978179.82 s_bodybytes - Total body bytes
      315656        16.16         7.54 sess_closed - Session Closed
    10189788    67.69 243.54 sess_linger - Session Linger
     1036235        24.25        24.77 sess_herd - Session herd
   587715869      4231.96     14046.41 shm_records - SHM records
    27056594   243.47 646.65 shm_writes - SHM writes
          90         0.00         0.00 shm_flushes - SHM flushes due to
overflow
        5354         0.00         0.13 shm_cont - SHM MTX contention
         206         0.00         0.00 shm_cycles - SHM cycles through
buffer
       19939        16.16         0.48 sms_nreq - SMS allocator requests
     7860413          .            .   sms_balloc - SMS bytes allocated
     7860413          .            .   sms_bfree - SMS bytes freed
     7230872    38.39 172.82 backend_req - Backend requests made
           1         0.00         0.00 n_vcl - N vcl total
           1         0.00         0.00 n_vcl_avail - N vcl available
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/68439ba2/attachment.html>

From lkarsten at varnish-software.com  Wed Mar 20 13:12:29 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 20 Mar 2013 14:12:29 +0100
Subject: high iowait
In-Reply-To: <CABm-WmeAF015u=KZ6UJMdYGHKR0JzZYpv90-EKLusnA7w5+2Dg@mail.gmail.com>
References: <CABm-WmeAF015u=KZ6UJMdYGHKR0JzZYpv90-EKLusnA7w5+2Dg@mail.gmail.com>
Message-ID: <20130320131228.GA17754@immer.varnish-software.com>

On Tue, Mar 19, 2013 at 09:46:37PM -0400, Sean Allen wrote:
> One of our varrnish servers is spending about 40-50% of its time in iowait.
> Is this just from the varnishlog getting written? Our IO performance is not
> great and I'm looking to be able to get the amount of time we are spending
> doing IO down. This occurs even when everything is running nicely in memory
> and we aren't overflowing into swap ( which was detailed in a different
> email as I think they might be different issues ).
> varnishd (varnish-3.0.3 revision 9e6a70f)
[..]
> # Without "ban_lurker_sleep," nothing banned from the cache ever gets
> evicted.

Ban lurker default sleep is 10ms in 3.0.3. You are increasing it a lot, why?

Are you using smart bans?

	https://www.varnish-software.com/static/book/Cache_invalidation.html#smart-bans

> DAEMON_OPTS="-a :6081 \
>              -T :6082 \
>              -f /etc/varnish/default.vcl \
>              -u varnish -g varnish \
>              -p thread_pool_min=500 \
>              -p thread_pool_max=5000 \
>              -s malloc,6G \
>              -p thread_pools=2 \
>              -p thread_pool_add_delay=1 \
>              -p ban_lurker_sleep=60s"

Your thread count is a bit on the high side (10k); ~1 thread / concurrent connection is good.

But it doesn't explain why you are in iowait.
Is this running on a virtualised server? Do you have proper network drivers?


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From lkarsten at varnish-software.com  Wed Mar 20 13:22:51 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 20 Mar 2013 14:22:51 +0100
Subject: Swapping & High Load
In-Reply-To: <CABm-WmfxuWBe4v5QHRVcxVfObNKvF9dZvnCrwC76O6ohvfn_sA@mail.gmail.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
 <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
 <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
 <20130320123022.GA16203@immer.varnish-software.com>
 <CABm-WmfxuWBe4v5QHRVcxVfObNKvF9dZvnCrwC76O6ohvfn_sA@mail.gmail.com>
Message-ID: <20130320132250.GB17754@immer.varnish-software.com>

On Wed, Mar 20, 2013 at 08:59:23AM -0400, Sean Allen wrote:
> We ran a batch job last night that moves everything possible through the
> cache. Thus the eviction numbers and the currently low hit rate as 'active'
> records start to be pulled through at the start of our business day:
> 0+11:37:21
> Hitrate ratio:       10      100     1000
> Hitrate avg:     0.5494   0.5375   0.5230
[..]

If you use varnishstat -1 you get all the counters at once, not just
the ones that fit onto your screen. :)


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From lnsano at bol.com.br  Wed Mar 20 13:26:10 2013
From: lnsano at bol.com.br (lnsano at bol.com.br)
Date: Wed, 20 Mar 2013 10:26:10 -0300
Subject: Swapping & High Load
In-Reply-To: <CABm-WmfxuWBe4v5QHRVcxVfObNKvF9dZvnCrwC76O6ohvfn_sA@mail.gmail.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
 <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
 <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
 <20130320123022.GA16203@immer.varnish-software.com>
 <CABm-WmfxuWBe4v5QHRVcxVfObNKvF9dZvnCrwC76O6ohvfn_sA@mail.gmail.com>
Message-ID: <5149b8f27931_52192cbd46c566@a4-winter17.tmail>

An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/bf979604/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 13:27:58 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Wed, 20 Mar 2013 09:27:58 -0400
Subject: high iowait
In-Reply-To: <20130320131228.GA17754@immer.varnish-software.com>
References: <CABm-WmeAF015u=KZ6UJMdYGHKR0JzZYpv90-EKLusnA7w5+2Dg@mail.gmail.com>
 <20130320131228.GA17754@immer.varnish-software.com>
Message-ID: <CABm-Wmcouqb7v009ACMzfznCD2eq9+n-cp9NtNb0LtRT3n-biw@mail.gmail.com>

On Wed, Mar 20, 2013 at 9:12 AM, Lasse Karstensen <
lkarsten at varnish-software.com> wrote:

> On Tue, Mar 19, 2013 at 09:46:37PM -0400, Sean Allen wrote:
> > One of our varrnish servers is spending about 40-50% of its time in
> iowait.
> > Is this just from the varnishlog getting written? Our IO performance is
> not
> > great and I'm looking to be able to get the amount of time we are
> spending
> > doing IO down. This occurs even when everything is running nicely in
> memory
> > and we aren't overflowing into swap ( which was detailed in a different
> > email as I think they might be different issues ).
> > varnishd (varnish-3.0.3 revision 9e6a70f)
> [..]
> > # Without "ban_lurker_sleep," nothing banned from the cache ever gets
> > evicted.
>
> Ban lurker default sleep is 10ms in 3.0.3. You are increasing it a lot,
> why?
>
>
Lack of knowledge when we put it together about what would be good.
My understanding is the less frequently that runs the more memory that
would be used
but performance would be slightly better. Is that incorrect?



> Are you using smart bans?
>
>
> https://www.varnish-software.com/static/book/Cache_invalidation.html#smart-bans


A combination of smart bans and purges.

Any invalidation that comes from the backend is handled via a smart ban.
There is legacy code
that causes db updates that would send a purge via port 6081 to invalidate
entities that were
updated 'out of band'.

The vast majority come via the 'out of band' purges.



>
>
> > DAEMON_OPTS="-a :6081 \
> >              -T :6082 \
> >              -f /etc/varnish/default.vcl \
> >              -u varnish -g varnish \
> >              -p thread_pool_min=500 \
> >              -p thread_pool_max=5000 \
> >              -s malloc,6G \
> >              -p thread_pools=2 \
> >              -p thread_pool_add_delay=1 \
> >              -p ban_lurker_sleep=60s"
>
> Your thread count is a bit on the high side (10k); ~1 thread / concurrent
> connection is good.
>

So workers created says 1000. Nothing ever gets queued. On this varnish we
dont get to 1000 concurrent.
On another we regularly bounce up 3k concurrent connections but the workers
stays at 1000.

>From the documentation, Ive never been entirely clear about how the thread
pool sizes work.

I would have thought that as our 1 second rate for connections was 3k, we
would create more workers.
Would this be requests finishing in less than 1 second so 1000 workers can
handle the 3k without queueing?
If they were going to queue, we would start to spin up more workers yes?


> But it doesn't explain why you are in iowait.
> Is this running on a virtualised server? Do you have proper network
> drivers?
>

VMWare on Centos 5.x
Drivers are good.

I'm hoping that virtualized isn't the culprit but I won't be surprised if
it is.

Thanks,
Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/e5386cbc/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 13:29:12 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Wed, 20 Mar 2013 09:29:12 -0400
Subject: Swapping & High Load
In-Reply-To: <20130320132250.GB17754@immer.varnish-software.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
 <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
 <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
 <20130320123022.GA16203@immer.varnish-software.com>
 <CABm-WmfxuWBe4v5QHRVcxVfObNKvF9dZvnCrwC76O6ohvfn_sA@mail.gmail.com>
 <20130320132250.GB17754@immer.varnish-software.com>
Message-ID: <CABm-Wmf+3X05JxHTATQ7dB9zfhtDnKaJzicSZFz2TUH8a1C3YA@mail.gmail.com>

On Wed, Mar 20, 2013 at 9:22 AM, Lasse Karstensen <
lkarsten at varnish-software.com> wrote:

> varnishstat -1


Nice. Thanks. Here you go:

client_conn           28048769       659.06 Client connections accepted
client_drop                  0         0.00 Connection dropped, no sess/wrk
client_req            56031540      1316.56 Client requests received
cache_hit             55584119      1306.05 Cache hits
cache_hitpass               17         0.00 Cache hits for pass
cache_miss              447078        10.50 Cache misses
backend_conn               520         0.01 Backend conn. success
backend_unhealthy            0         0.00 Backend conn. not attempted
backend_busy                 0         0.00 Backend conn. too many
backend_fail                27         0.00 Backend conn. failures
backend_reuse           439571        10.33 Backend conn. reuses
backend_toolate            482         0.01 Backend conn. was closed
backend_recycle         440069        10.34 Backend conn. recycles
backend_retry               19         0.00 Backend conn. retry
fetch_head                   0         0.00 Fetch head
fetch_length            440060        10.34 Fetch with Length
fetch_chunked                9         0.00 Fetch chunked
fetch_eof                    0         0.00 Fetch EOF
fetch_bad                    0         0.00 Fetch had bad headers
fetch_close                  0         0.00 Fetch wanted close
fetch_oldhttp                0         0.00 Fetch pre HTTP/1.1 closed
fetch_zero                   0         0.00 Fetch zero len
fetch_failed                 0         0.00 Fetch failed
fetch_1xx                    0         0.00 Fetch no body (1xx)
fetch_204                    0         0.00 Fetch no body (204)
fetch_304                    0         0.00 Fetch no body (304)
n_sess_mem                 127          .   N struct sess_mem
n_sess                      20          .   N struct sess
n_object                387181          .   N struct object
n_vampireobject              0          .   N unresurrected objects
n_objectcore            387215          .   N struct objectcore
n_objecthead            387247          .   N struct objecthead
n_waitinglist              348          .   N struct waitinglist
n_vbc                       16          .   N struct vbc
n_wrk                     1000          .   N worker threads
n_wrk_create              1000         0.02 N worker threads created
n_wrk_failed                 0         0.00 N worker threads not created
n_wrk_max                    0         0.00 N worker threads limited
n_wrk_lqueue                 0         0.00 work request queue length
n_wrk_queued                 1         0.00 N queued work requests
n_wrk_drop                   0         0.00 N dropped work requests
n_backend                    3          .   N backends
n_expired                52545          .   N expired objects
n_lru_nuked                  0          .   N LRU nuked objects
n_lru_moved           49646103          .   N LRU moved objects
losthdr                      0         0.00 HTTP header overflows
n_objsendfile                0         0.00 Objects sent with sendfile
n_objwrite            28054198       659.18 Objects sent with write
n_objoverflow                0         0.00 Objects overflowing workspace
s_sess                28048766       659.06 Total Sessions
s_req                 56031540      1316.56 Total Requests
s_pipe                       0         0.00 Total pipe
s_pass                     343         0.01 Total pass
s_fetch                 440069        10.34 Total fetch
s_hdrbytes         18085670561    424955.25 Total header bytes
s_bodybytes       203280131380   4776431.10 Total body bytes
sess_closed           28011785       658.19 Session Closed
sess_pipeline                0         0.00 Session Pipeline
sess_readahead               0         0.00 Session Read Ahead
sess_linger           56023162      1316.36 Session Linger
sess_herd               102158         2.40 Session herd
shm_records         2031807374     47740.96 SHM records
shm_writes           141221068      3318.24 SHM writes
shm_flushes                  0         0.00 SHM flushes due to overflow
shm_cont                  5219         0.12 SHM MTX contention
shm_cycles                 680         0.02 SHM cycles through buffer
sms_nreq                  8324         0.20 SMS allocator requests
sms_nobj                     0          .   SMS outstanding allocations
sms_nbytes                   0          .   SMS outstanding bytes
sms_balloc             3320554          .   SMS bytes allocated
sms_bfree              3320554          .   SMS bytes freed
backend_req             440091        10.34 Backend requests made
n_vcl                        1         0.00 N vcl total
n_vcl_avail                  1         0.00 N vcl available
n_vcl_discard                0         0.00 N vcl discarded
n_ban                        1          .   N total active bans
n_ban_gone                   1          .   N total gone bans
n_ban_add                    1         0.00 N new bans added
n_ban_retire                 0         0.00 N old bans deleted
n_ban_obj_test               0         0.00 N objects tested
n_ban_re_test                0         0.00 N regexps tested against
n_ban_dups                   0         0.00 N duplicate bans removed
hcb_nolock            56031222      1316.55 HCB Lookups without lock
hcb_lock                425399        10.00 HCB Lookups with lock
hcb_insert              425399        10.00 HCB Inserts
esi_errors                   0         0.00 ESI parse errors (unlock)
esi_warnings                 0         0.00 ESI parse warnings (unlock)
accept_fail                  0         0.00 Accept failures
client_drop_late             0         0.00 Connection dropped late
uptime                   42559         1.00 Client uptime
dir_dns_lookups              0         0.00 DNS director lookups
dir_dns_failed               0         0.00 DNS director failed lookups
dir_dns_hit                  0         0.00 DNS director cached lookups hit
dir_dns_cache_full           0         0.00 DNS director full dnscache
vmods                        0          .   Loaded VMODs
n_gzip                       0         0.00 Gzip operations
n_gunzip                     0         0.00 Gunzip operations
LCK.sms.creat                1         0.00 Created locks
LCK.sms.destroy              0         0.00 Destroyed locks
LCK.sms.locks            24972         0.59 Lock Operations
LCK.sms.colls                0         0.00 Collisions
LCK.smp.creat                0         0.00 Created locks
LCK.smp.destroy              0         0.00 Destroyed locks
LCK.smp.locks                0         0.00 Lock Operations
LCK.smp.colls                0         0.00 Collisions
LCK.sma.creat                2         0.00 Created locks
LCK.sma.destroy              0         0.00 Destroyed locks
LCK.sma.locks           714028        16.78 Lock Operations
LCK.sma.colls                0         0.00 Collisions
LCK.smf.creat                0         0.00 Created locks
LCK.smf.destroy              0         0.00 Destroyed locks
LCK.smf.locks                0         0.00 Lock Operations
LCK.smf.colls                0         0.00 Collisions
LCK.hsl.creat                0         0.00 Created locks
LCK.hsl.destroy              0         0.00 Destroyed locks
LCK.hsl.locks                0         0.00 Lock Operations
LCK.hsl.colls                0         0.00 Collisions
LCK.hcb.creat                1         0.00 Created locks
LCK.hcb.destroy              0         0.00 Destroyed locks
LCK.hcb.locks           463854        10.90 Lock Operations
LCK.hcb.colls                0         0.00 Collisions
LCK.hcl.creat                0         0.00 Created locks
LCK.hcl.destroy              0         0.00 Destroyed locks
LCK.hcl.locks                0         0.00 Lock Operations
LCK.hcl.colls                0         0.00 Collisions
LCK.vcl.creat                1         0.00 Created locks
LCK.vcl.destroy              0         0.00 Destroyed locks
LCK.vcl.locks              869         0.02 Lock Operations
LCK.vcl.colls                0         0.00 Collisions
LCK.stat.creat               1         0.00 Created locks
LCK.stat.destroy             0         0.00 Destroyed locks
LCK.stat.locks        28048877       659.06 Lock Operations
LCK.stat.colls               0         0.00 Collisions
LCK.sessmem.creat            1         0.00 Created locks
LCK.sessmem.destroy            0         0.00 Destroyed locks
LCK.sessmem.locks       28296740       664.88 Lock Operations
LCK.sessmem.colls              0         0.00 Collisions
LCK.wstat.creat                1         0.00 Created locks
LCK.wstat.destroy              0         0.00 Destroyed locks
LCK.wstat.locks            84726         1.99 Lock Operations
LCK.wstat.colls                0         0.00 Collisions
LCK.herder.creat               1         0.00 Created locks
LCK.herder.destroy             0         0.00 Destroyed locks
LCK.herder.locks               1         0.00 Lock Operations
LCK.herder.colls               0         0.00 Collisions
LCK.wq.creat                   2         0.00 Created locks
LCK.wq.destroy                 0         0.00 Destroyed locks
LCK.wq.locks            56313822      1323.19 Lock Operations
LCK.wq.colls                   0         0.00 Collisions
LCK.objhdr.creat          425433        10.00 Created locks
LCK.objhdr.destroy         38186         0.90 Destroyed locks
LCK.objhdr.locks       224254162      5269.25 Lock Operations
LCK.objhdr.colls               0         0.00 Collisions
LCK.exp.creat                  1         0.00 Created locks
LCK.exp.destroy                0         0.00 Destroyed locks
LCK.exp.locks             535266        12.58 Lock Operations
LCK.exp.colls                  0         0.00 Collisions
LCK.lru.creat                  2         0.00 Created locks
LCK.lru.destroy                0         0.00 Destroyed locks
LCK.lru.locks             440745        10.36 Lock Operations
LCK.lru.colls                  0         0.00 Collisions
LCK.cli.creat                  1         0.00 Created locks
LCK.cli.destroy                0         0.00 Destroyed locks
LCK.cli.locks              14157         0.33 Lock Operations
LCK.cli.colls                  0         0.00 Collisions
LCK.ban.creat                  1         0.00 Created locks
LCK.ban.destroy                0         0.00 Destroyed locks
LCK.ban.locks             534216        12.55 Lock Operations
LCK.ban.colls                  0         0.00 Collisions
LCK.vbp.creat                  1         0.00 Created locks
LCK.vbp.destroy                0         0.00 Destroyed locks
LCK.vbp.locks              25070         0.59 Lock Operations
LCK.vbp.colls                  0         0.00 Collisions
LCK.vbe.creat                  1         0.00 Created locks
LCK.vbe.destroy                0         0.00 Destroyed locks
LCK.vbe.locks               1078         0.03 Lock Operations
LCK.vbe.colls                  0         0.00 Collisions
LCK.backend.creat              3         0.00 Created locks
LCK.backend.destroy            0         0.00 Destroyed locks
LCK.backend.locks         881747        20.72 Lock Operations
LCK.backend.colls              0         0.00 Collisions
SMA.s0.c_req              645811        15.17 Allocator requests
SMA.s0.c_fail                  0         0.00 Allocator failures
SMA.s0.c_bytes        1968349000     46249.89 Bytes allocated
SMA.s0.c_freed         335207639      7876.30 Bytes freed
SMA.s0.g_alloc            578975          .   Allocations outstanding
SMA.s0.g_bytes        1633141361          .   Bytes outstanding
SMA.s0.g_space        1588084111          .   Bytes available
SMA.Transient.c_req          686         0.02 Allocator requests
SMA.Transient.c_fail           0         0.00 Allocator failures
SMA.Transient.c_bytes      2793980        65.65 Bytes allocated
SMA.Transient.c_freed      2793980        65.65 Bytes freed
SMA.Transient.g_alloc            0          .   Allocations outstanding
SMA.Transient.g_bytes            0          .   Bytes outstanding
SMA.Transient.g_space            0          .   Bytes available
VBE.job_1(192.168.0.13,,8080).vcls            1          .   VCL references
VBE.job_1(192.168.0.13,,8080).happy           0          .   Happy health
probes
VBE.job_2(192.168.0.14,,8080).vcls            1          .   VCL references
VBE.job_2(192.168.0.14,,8080).happy18446744073709551615          .   Happy
health probes
VBE.job_3(192.168.0.43,,8080).vcls            1          .   VCL references
VBE.job_3(192.168.0.43,,8080).happy18446744073709551615          .   Happy
health probes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/b49ef05c/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 13:39:41 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Wed, 20 Mar 2013 09:39:41 -0400
Subject: Swapping & High Load
In-Reply-To: <CABm-Wmf+3X05JxHTATQ7dB9zfhtDnKaJzicSZFz2TUH8a1C3YA@mail.gmail.com>
References: <CABm-WmeUTpQH=f9ixTrTyWH61r1Npvz9AeWMM4-DAfceBxXOmg@mail.gmail.com>
 <CAOXZevAi9UQL+7fSyfuR2qg_d4XfaCq_qJTmvLuWirCEU=YzMw@mail.gmail.com>
 <CABm-WmeLE8KSnyUSpWSf+_rVVabRw4DnPMirFwARzocm0pWH4w@mail.gmail.com>
 <20130320123022.GA16203@immer.varnish-software.com>
 <CABm-WmfxuWBe4v5QHRVcxVfObNKvF9dZvnCrwC76O6ohvfn_sA@mail.gmail.com>
 <20130320132250.GB17754@immer.varnish-software.com>
 <CABm-Wmf+3X05JxHTATQ7dB9zfhtDnKaJzicSZFz2TUH8a1C3YA@mail.gmail.com>
Message-ID: <CABm-WmeuFQhgONC+fK99deRB_D_tg9sDT9unixWOm6FiXP6dsA@mail.gmail.com>

Sorry that last one for the varnish that is doing quite well. Apologies, it
has been a long night of working out performance issue as we scale a new
system.

Here is the correct varnishstat...

client_conn             436982         9.87 Client connections accepted
client_drop                  0         0.00 Connection dropped, no sess/wrk
client_req            10472769       236.62 Client requests received
cache_hit              3132035        70.77 Cache hits
cache_hitpass                0         0.00 Cache hits for pass
cache_miss             7335111       165.73 Cache misses
backend_conn              5874         0.13 Backend conn. success
backend_unhealthy            0         0.00 Backend conn. not attempted
backend_busy                 0         0.00 Backend conn. too many
backend_fail                26         0.00 Backend conn. failures
backend_reuse          7320973       165.41 Backend conn. reuses
backend_toolate           5853         0.13 Backend conn. was closed
backend_recycle        7326845       165.54 Backend conn. recycles
backend_retry                0         0.00 Backend conn. retry
fetch_head                   0         0.00 Fetch head
fetch_length           7288254       164.67 Fetch with Length
fetch_chunked            38586         0.87 Fetch chunked
fetch_eof                    0         0.00 Fetch EOF
fetch_bad                    0         0.00 Fetch had bad headers
fetch_close                  0         0.00 Fetch wanted close
fetch_oldhttp                0         0.00 Fetch pre HTTP/1.1 closed
fetch_zero                   0         0.00 Fetch zero len
fetch_failed                 0         0.00 Fetch failed
fetch_1xx                    0         0.00 Fetch no body (1xx)
fetch_204                    0         0.00 Fetch no body (204)
fetch_304                    0         0.00 Fetch no body (304)
n_sess_mem                 431          .   N struct sess_mem
n_sess                      27          .   N struct sess
n_object                570373          .   N struct object
n_vampireobject              0          .   N unresurrected objects
n_objectcore            570558          .   N struct objectcore
n_objecthead            580322          .   N struct objecthead
n_waitinglist             9800          .   N struct waitinglist
n_vbc                       21          .   N struct vbc
n_wrk                     1000          .   N worker threads
n_wrk_create              1000         0.02 N worker threads created
n_wrk_failed                 0         0.00 N worker threads not created
n_wrk_max                    0         0.00 N worker threads limited
n_wrk_lqueue                 0         0.00 work request queue length
n_wrk_queued                28         0.00 N queued work requests
n_wrk_drop                   0         0.00 N dropped work requests
n_backend                    4          .   N backends
n_expired                12029          .   N expired objects
n_lru_nuked            6738821          .   N LRU nuked objects
n_lru_moved             493769          .   N LRU moved objects
losthdr                      0         0.00 HTTP header overflows
n_objsendfile                0         0.00 Objects sent with sendfile
n_objwrite            10619899       239.95 Objects sent with write
n_objoverflow                0         0.00 Objects overflowing workspace
s_sess                  436982         9.87 Total Sessions
s_req                 10472769       236.62 Total Requests
s_pipe                       0         0.00 Total pipe
s_pass                    5623         0.13 Total pass
s_fetch                7326840       165.54 Total fetch
s_hdrbytes          3022382411     68288.54 Total header bytes
s_bodybytes        86894699524   1963322.70 Total body bytes
sess_closed             322071         7.28 Session Closed
sess_pipeline                0         0.00 Session Pipeline
sess_readahead               0         0.00 Session Read Ahead
sess_linger           10450028       236.11 Session Linger
sess_herd              1123126        25.38 Session herd
shm_records          599992632     13556.40 SHM records
shm_writes            27652580       624.79 SHM writes
shm_flushes                 90         0.00 SHM flushes due to overflow
shm_cont                  5414         0.12 SHM MTX contention
shm_cycles                 210         0.00 SHM cycles through buffer
sms_nreq                 22387         0.51 SMS allocator requests
sms_nobj                     0          .   SMS outstanding allocations
sms_nbytes                   0          .   SMS outstanding bytes
sms_balloc             8824313          .   SMS bytes allocated
sms_bfree              8824313          .   SMS bytes freed
backend_req            7326847       165.54 Backend requests made
n_vcl                        1         0.00 N vcl total
n_vcl_avail                  1         0.00 N vcl available
n_vcl_discard                0         0.00 N vcl discarded
n_ban                     4323          .   N total active bans
n_ban_gone                3013          .   N total gone bans
n_ban_add                 4424         0.10 N new bans added
n_ban_retire               101         0.00 N old bans deleted
n_ban_obj_test          555491        12.55 N objects tested
n_ban_re_test         24192370       546.61 N regexps tested against
n_ban_dups                2857         0.06 N duplicate bans removed
hcb_nolock            10467161       236.50 HCB Lookups without lock
hcb_lock               7325552       165.52 HCB Lookups with lock
hcb_insert             7325552       165.52 HCB Inserts
esi_errors                   0         0.00 ESI parse errors (unlock)
esi_warnings                 0         0.00 ESI parse warnings (unlock)
accept_fail                  0         0.00 Accept failures
client_drop_late             0         0.00 Connection dropped late
uptime                   44259         1.00 Client uptime
dir_dns_lookups              0         0.00 DNS director lookups
dir_dns_failed               0         0.00 DNS director failed lookups
dir_dns_hit                  0         0.00 DNS director cached lookups hit
dir_dns_cache_full           0         0.00 DNS director full dnscache
vmods                        0          .   Loaded VMODs
n_gzip                       0         0.00 Gzip operations
n_gunzip                     0         0.00 Gunzip operations
LCK.sms.creat                1         0.00 Created locks
LCK.sms.destroy              0         0.00 Destroyed locks
LCK.sms.locks            67161         1.52 Lock Operations
LCK.sms.colls                0         0.00 Collisions
LCK.smp.creat                0         0.00 Created locks
LCK.smp.destroy              0         0.00 Destroyed locks
LCK.smp.locks                0         0.00 Lock Operations
LCK.smp.colls                0         0.00 Collisions
LCK.sma.creat                2         0.00 Created locks
LCK.sma.destroy              0         0.00 Destroyed locks
LCK.sma.locks         35695962       806.52 Lock Operations
LCK.sma.colls                0         0.00 Collisions
LCK.smf.creat                0         0.00 Created locks
LCK.smf.destroy              0         0.00 Destroyed locks
LCK.smf.locks                0         0.00 Lock Operations
LCK.smf.colls                0         0.00 Collisions
LCK.hsl.creat                0         0.00 Created locks
LCK.hsl.destroy              0         0.00 Destroyed locks
LCK.hsl.locks                0         0.00 Lock Operations
LCK.hsl.colls                0         0.00 Collisions
LCK.hcb.creat                1         0.00 Created locks
LCK.hcb.destroy              0         0.00 Destroyed locks
LCK.hcb.locks         14080978       318.15 Lock Operations
LCK.hcb.colls                0         0.00 Collisions
LCK.hcl.creat                0         0.00 Created locks
LCK.hcl.destroy              0         0.00 Destroyed locks
LCK.hcl.locks                0         0.00 Lock Operations
LCK.hcl.colls                0         0.00 Collisions
LCK.vcl.creat                1         0.00 Created locks
LCK.vcl.destroy              0         0.00 Destroyed locks
LCK.vcl.locks            20318         0.46 Lock Operations
LCK.vcl.colls                0         0.00 Collisions
LCK.stat.creat               1         0.00 Created locks
LCK.stat.destroy             0         0.00 Destroyed locks
LCK.stat.locks          437385         9.88 Lock Operations
LCK.stat.colls               0         0.00 Collisions
LCK.sessmem.creat            1         0.00 Created locks
LCK.sessmem.destroy            0         0.00 Destroyed locks
LCK.sessmem.locks         439310         9.93 Lock Operations
LCK.sessmem.colls              0         0.00 Collisions
LCK.wstat.creat                1         0.00 Created locks
LCK.wstat.destroy              0         0.00 Destroyed locks
LCK.wstat.locks           659867        14.91 Lock Operations
LCK.wstat.colls                0         0.00 Collisions
LCK.herder.creat               1         0.00 Created locks
LCK.herder.destroy             0         0.00 Destroyed locks
LCK.herder.locks               1         0.00 Lock Operations
LCK.herder.colls               0         0.00 Collisions
LCK.wq.creat                   2         0.00 Created locks
LCK.wq.destroy                 0         0.00 Destroyed locks
LCK.wq.locks             3000715        67.80 Lock Operations
LCK.wq.colls                   0         0.00 Collisions
LCK.objhdr.creat         7325735       165.52 Created locks
LCK.objhdr.destroy       6745408       152.41 Destroyed locks
LCK.objhdr.locks        55415086      1252.06 Lock Operations
LCK.objhdr.colls               0         0.00 Collisions
LCK.exp.creat                  1         0.00 Created locks
LCK.exp.destroy                0         0.00 Destroyed locks
LCK.exp.locks           14134819       319.37 Lock Operations
LCK.exp.colls                  0         0.00 Collisions
LCK.lru.creat                  2         0.00 Created locks
LCK.lru.destroy                0         0.00 Destroyed locks
LCK.lru.locks           14079161       318.11 Lock Operations
LCK.lru.colls                  0         0.00 Collisions
LCK.cli.creat                  1         0.00 Created locks
LCK.cli.destroy                0         0.00 Destroyed locks
LCK.cli.locks              14720         0.33 Lock Operations
LCK.cli.colls                  0         0.00 Collisions
LCK.ban.creat                  1         0.00 Created locks
LCK.ban.destroy                0         0.00 Destroyed locks
LCK.ban.locks           14633724       330.64 Lock Operations
LCK.ban.colls                  0         0.00 Collisions
LCK.vbp.creat                  1         0.00 Created locks
LCK.vbp.destroy                0         0.00 Destroyed locks
LCK.vbp.locks              35291         0.80 Lock Operations
LCK.vbp.colls                  0         0.00 Collisions
LCK.vbe.creat                  1         0.00 Created locks
LCK.vbe.destroy                0         0.00 Destroyed locks
LCK.vbe.locks              11779         0.27 Lock Operations
LCK.vbe.colls                  0         0.00 Collisions
LCK.backend.creat              4         0.00 Created locks
LCK.backend.destroy            0         0.00 Destroyed locks
LCK.backend.locks       14671350       331.49 Lock Operations
LCK.backend.colls              0         0.00 Collisions
SMA.s0.c_req            22055417       498.33 Allocator requests
SMA.s0.c_fail       226907034699   5126799.85 Allocator failures
SMA.s0.c_bytes       62481788970   1411730.70 Bytes allocated
SMA.s0.c_freed       56039342774   1266168.30 Bytes freed
SMA.s0.g_alloc           1143047          .   Allocations outstanding
SMA.s0.g_bytes        6442446196          .   Bytes outstanding
SMA.s0.g_space              4748          .   Bytes available
SMA.Transient.c_req        12052         0.27 Allocator requests
SMA.Transient.c_fail           0         0.00 Allocator failures
SMA.Transient.c_bytes    186420941      4212.05 Bytes allocated
SMA.Transient.c_freed    186420941      4212.05 Bytes freed
SMA.Transient.g_alloc            0          .   Allocations outstanding
SMA.Transient.g_bytes            0          .   Bytes outstanding
SMA.Transient.g_space            0          .   Bytes available
VBE.jobseeker_1(192.168.0.17,,8080).vcls            1          .   VCL
references
VBE.jobseeker_1(192.168.0.17,,8080).happy18446744073709551615          .
Happy health probes
VBE.jobseeker_2(192.168.0.18,,8080).vcls            1          .   VCL
references
VBE.jobseeker_2(192.168.0.18,,8080).happy18446744073709551615          .
Happy health probes
VBE.jobseeker_3(192.168.0.19,,8080).vcls            1          .   VCL
references
VBE.jobseeker_3(192.168.0.19,,8080).happy18446744073709551615          .
Happy health probes
VBE.jobseeker_4(192.168.1.160,,8080).vcls           1          .   VCL
references
VBE.jobseeker_4(192.168.1.160,,8080).happy 18446744073709551615          .
  Happy health probes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/a548df4b/attachment.html>

From radecki.rafal at gmail.com  Wed Mar 20 14:23:56 2013
From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=)
Date: Wed, 20 Mar 2013 15:23:56 +0100
Subject: Varnish error page when all backends are gone?
Message-ID: <CAHd9_iTLOHuANi7URTq_nm2kvFf2o_uVP_WXzKMUR3Qy4Nd9_A@mail.gmail.com>

Hi All.

I use varnish with two backends. I have custom error pages for
standard errors returned from backends. But when both my backends are
down I get:

Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection
without sending any data.

Is there a way to return a custom "error" page when all backends are
down? Or should I configure a local (on the same server as varnish)
instance od web server with a static page and use it in case of
problems? Hot to achieve that in VCL?

Best regards,
Rafal Radecki.



From cornuwel at gmail.com  Wed Mar 20 17:01:08 2013
From: cornuwel at gmail.com (Julien Cornuwel)
Date: Wed, 20 Mar 2013 18:01:08 +0100
Subject: GeoIP-plugin : Varnish won't start
Message-ID: <CAOvShqp++Yr5VbNE1pqdVVW=wwje=g4zWdN7-N0ORvHsBx0ydg@mail.gmail.com>

Hi,

I installed Varnish 3.0.3 from repo.varnish-cache.org on Debian Squeeze.
I got the plugin from
https://www.varnish-cache.org/trac/wiki/GeoipUsingInlineC (
GeoIP-plugin-2.tar.gz ).

The plugin seems to work on its own :
# make test
gcc geoip_plugin_test.c -o geoip_plugin_test -ldl
LD_LIBRARY_PATH=$PWD ./geoip_plugin_test
GeoIP plugin loaded successfully.
Testing IPs:
195.110.128.11 => IT (OK)
157.166.224.25 => US (OK)
212.58.224.138 => GB (OK)
80.91.37.210 => NO (OK)
192.168.0.1 => Unknown (OK)
(null) => Unknown (OK)

However, Varnish fails to start :
# /etc/init.d/varnish start
Starting HTTP accelerator: varnishd failed!
Message from C-compiler:
./vcl.IoMQyW5q.c: In function ?VGC_function_vcl_hash?:
./vcl.IoMQyW5q.c:569: warning: implicit declaration of function
?VRT_l_req_hash?
Message from dlopen:
Compiled VCL program failed to load:
  ./vcl.IoMQyW5q.so: undefined symbol: VRT_l_req_hash
Running dlopen failed, exit 1

VCL compilation failed


Any idea what could cause that ?
If not, any alternative that is known to work ?


Below is my default.vcl configuration (which is the file provided with the
plugin, with just the correct path added) :

#################################
# This block of inline C loads the library
C{
  #include <dlfcn.h>
  #include <stdlib.h>
  #include <stdio.h>

static const char* (*get_country_code)(char* ip) = NULL;

__attribute__((constructor)) void
load_module()
{
    const char* symbol_name = "get_country_code";
    const char* plugin_name = "/usr/local/lib/geoip_plugin.so";
    void* handle = NULL;

    handle = dlopen( plugin_name, RTLD_NOW );
    if (handle != NULL) {
        get_country_code = dlsym( handle, symbol_name );
        if (get_country_code == NULL)
            fprintf( stderr, "\nError: Could not load GeoIP
plugin:\n%s\n\n", dlerror() );
        else
            printf( "GeoIP plugin loaded successfully.\n");
    }
    else
        fprintf( stderr, "\nError: Could not load GeoIP plugin:\n%s\n\n",
dlerror() );
}
}C

# A sample backend declaration used for testing
backend default {
    .host = "localhost";
    .port = "8080";
}

sub vcl_hash {
# This block of inline C adds the country code to the hash
    C{
        if (get_country_code)
            VRT_l_req_hash(sp, (*get_country_code)( VRT_IP_string(sp,
VRT_r_server_ip(sp))));
    }C

# Insert any custom hashing here

}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/a91c6350/attachment.html>

From ruben at varnish-software.com  Wed Mar 20 18:56:36 2013
From: ruben at varnish-software.com (=?UTF-8?Q?Rub=C3=A9n_Romero?=)
Date: Wed, 20 Mar 2013 19:56:36 +0100
Subject: GeoIP-plugin : Varnish won't start
In-Reply-To: <CAOvShqp++Yr5VbNE1pqdVVW=wwje=g4zWdN7-N0ORvHsBx0ydg@mail.gmail.com>
References: <CAOvShqp++Yr5VbNE1pqdVVW=wwje=g4zWdN7-N0ORvHsBx0ydg@mail.gmail.com>
Message-ID: <CAEfnkDZnUBdy1kHcDrvS4jOAGUnrCVYHzAwe-boWsYQD+FNKaw@mail.gmail.com>

On Wed, Mar 20, 2013 at 6:01 PM, Julien Cornuwel <cornuwel at gmail.com> wrote:

> Hi,
>
>
Hi Julien


> I installed Varnish 3.0.3 from repo.varnish-cache.org on Debian Squeeze.
> I got the plugin from
> https://www.varnish-cache.org/trac/wiki/GeoipUsingInlineC (
> GeoIP-plugin-2.tar.gz ).
>
>
In Varnish 3.0 you should use a VMOD for this, not Inline-C. Have a look at
these options that do geo-location look ups in slightly different ways:

https://github.com/leed25d/geoip-vmod
https://github.com/lampeh/libvmod-geoip

Good luck!



Best,
-- 
*Rub?n Romero*
Global Sales Executive | Varnish Software AS
Cell: +47 95964088 / Office: +47 21989260
Skype & Twitter: ruben_varnish
We Make Websites Fly!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/73845666/attachment.html>

From cornuwel at gmail.com  Wed Mar 20 19:25:31 2013
From: cornuwel at gmail.com (Julien Cornuwel)
Date: Wed, 20 Mar 2013 20:25:31 +0100
Subject: GeoIP-plugin : Varnish won't start
In-Reply-To: <CAEfnkDZnUBdy1kHcDrvS4jOAGUnrCVYHzAwe-boWsYQD+FNKaw@mail.gmail.com>
References: <CAOvShqp++Yr5VbNE1pqdVVW=wwje=g4zWdN7-N0ORvHsBx0ydg@mail.gmail.com>
 <CAEfnkDZnUBdy1kHcDrvS4jOAGUnrCVYHzAwe-boWsYQD+FNKaw@mail.gmail.com>
Message-ID: <CAOvShqrS4JruiWE-oy09NXghTJznjPbNb4AW0wqwVqXJchCmzA@mail.gmail.com>

That's what I found out later, and copiously cursed at Google for only
showing inline-C solutions on the first page for "varnish geoip".

Thanks for your pointers, I'll try this tomorrow.


2013/3/20 Rub?n Romero <ruben at varnish-software.com>

> On Wed, Mar 20, 2013 at 6:01 PM, Julien Cornuwel <cornuwel at gmail.com>wrote:
>
>> Hi,
>>
>>
> Hi Julien
>
>
>> I installed Varnish 3.0.3 from repo.varnish-cache.org on Debian Squeeze.
>> I got the plugin from
>> https://www.varnish-cache.org/trac/wiki/GeoipUsingInlineC (
>> GeoIP-plugin-2.tar.gz ).
>>
>>
> In Varnish 3.0 you should use a VMOD for this, not Inline-C. Have a look
> at these options that do geo-location look ups in slightly different ways:
>
> https://github.com/leed25d/geoip-vmod
> https://github.com/lampeh/libvmod-geoip
>
> Good luck!
>
>
>
> Best,
> --
> *Rub?n Romero*
> Global Sales Executive | Varnish Software AS
> Cell: +47 95964088 / Office: +47 21989260
> Skype & Twitter: ruben_varnish
> We Make Websites Fly!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/df67040e/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 20:23:56 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Wed, 20 Mar 2013 16:23:56 -0400
Subject: query string ordering and banning
Message-ID: <CABm-Wmd9CiQ3PqpB+JsXfm031KcTU2OMQmRq38xF+r8edUtNiw@mail.gmail.com>

I found this old mailing list entry for using query string parameters in an
'unordered' fashion for consistent hashing:

https://www.varnish-cache.org/lists/pipermail/varnish-misc/2011-May/020472.html

Can I use this same approach when building a url to store on the beresp to
do ban matching against?


-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/25bebd3e/attachment.html>

From jnerin at gmail.com  Wed Mar 20 21:17:16 2013
From: jnerin at gmail.com (Jorge)
Date: Wed, 20 Mar 2013 22:17:16 +0100
Subject: high iowait
In-Reply-To: <CABm-Wmcouqb7v009ACMzfznCD2eq9+n-cp9NtNb0LtRT3n-biw@mail.gmail.com>
References: <CABm-WmeAF015u=KZ6UJMdYGHKR0JzZYpv90-EKLusnA7w5+2Dg@mail.gmail.com>
 <20130320131228.GA17754@immer.varnish-software.com>
 <CABm-Wmcouqb7v009ACMzfznCD2eq9+n-cp9NtNb0LtRT3n-biw@mail.gmail.com>
Message-ID: <CAF-dNMZraCu7o61Z8dLNd-ZSSsnYxs+d6gM_oHSEvP_o4QdiSg@mail.gmail.com>

You are running linux, check that the path where the SHM log is written is
tmpfs mounted:
tmpfs on /usr/local/var/varnish type tmpfs (rw,size=256M)

A small one will be enough:

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           256M   82M  175M  32% /usr/local/var/varnish

Search for a file named "_.vsm".

If this file is on a "normal" filesystem (disk) linux will periodically
flush the contents to disk, which depending on your traffic can be some
tens of megabytes every some seconds.

-- 
Jorge Ner?n
<jnerin at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/97b462bf/attachment.html>

From R.Guest at gns.cri.nz  Wed Mar 20 22:09:53 2013
From: R.Guest at gns.cri.nz (Richard Guest)
Date: Thu, 21 Mar 2013 11:09:53 +1300
Subject: query string ordering and banning
In-Reply-To: <CABm-Wmd9CiQ3PqpB+JsXfm031KcTU2OMQmRq38xF+r8edUtNiw@mail.gmail.com>
References: <CABm-Wmd9CiQ3PqpB+JsXfm031KcTU2OMQmRq38xF+r8edUtNiw@mail.gmail.com>
Message-ID: <OF78829F80.AC555677-ONCC257B34.0079C17C-CC257B34.0079C17D@gns.cri.nz>

As long as you know what all the parameters are, sorting them "by hand" in your VCL should work.

Otherwise you could try?using libvmod-querystring to sort the URL parameters??https://github.com/Dridi/libvmod-querystring

Cheers,
Rich


-----varnish-misc-bounces at varnish-cache.org wrote: -----
To: "varnish-misc at varnish-cache.org" <varnish-misc at varnish-cache.org>
From: Sean Allen 
Sent by: varnish-misc-bounces at varnish-cache.org
Date: 21/03/2013 09:26
Subject: query string ordering and banning

I found this old mailing list entry for using query string parameters in an 'unordered' fashion for consistent hashing:

https://www.varnish-cache.org/lists/pipermail/varnish-misc/2011-May/020472.html

Can I use this same approach when building a url to store on the beresp to do ban matching against?


-- 

Ce n'est pas une signature
_________________________
______________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Notice: This email and any attachments are confidential.
If received in error please destroy and immediately notify us.
Do not copy or disclose the contents.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130321/ad0abe5d/attachment.html>

From dridi.boukelmoune at zenika.com  Wed Mar 20 22:52:22 2013
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Wed, 20 Mar 2013 23:52:22 +0100
Subject: query string ordering and banning
In-Reply-To: <OF78829F80.AC555677-ONCC257B34.0079C17C-CC257B34.0079C17D@gns.cri.nz>
References: <CABm-Wmd9CiQ3PqpB+JsXfm031KcTU2OMQmRq38xF+r8edUtNiw@mail.gmail.com>
 <OF78829F80.AC555677-ONCC257B34.0079C17C-CC257B34.0079C17D@gns.cri.nz>
Message-ID: <CABtDKm6H+1gS_0p34g3HyCGDi3Y+RGBP3DBAsTYDw1RDZcgXmA@mail.gmail.com>

Hi,

I actually am the author of this vmod, you can use it with any version of
varnish 3.

Feedback is welcomed, and I hope this will solve your issue.

Envoy? de mon smartphone
Le 20 mars 2013 23:10, "Richard Guest" <R.Guest at gns.cri.nz> a ?crit :

> As long as you know what all the parameters are, sorting them "by hand" in
> your VCL should work.
>
> Otherwise you could try using libvmod-querystring to sort the URL
> parameters? https://github.com/Dridi/libvmod-querystring
>
> Cheers,
> Rich
>
>
> -----varnish-misc-bounces at varnish-cache.org wrote: -----
> To: "varnish-misc at varnish-cache.org" <varnish-misc at varnish-cache.org>
> From: Sean Allen **
> Sent by: varnish-misc-bounces at varnish-cache.org
> Date: 21/03/2013 09:26
> Subject: query string ordering and banning
>
> I found this old mailing list entry for using query string parameters in
> an 'unordered' fashion for consistent hashing:
>
>
> https://www.varnish-cache.org/lists/pipermail/varnish-misc/2011-May/020472.html
>
> Can I use this same approach when building a url to store on the beresp to
> do ban matching against?
>
>
> --
>
> Ce n'est pas une signature
>   _________________________ ______________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> **
> Notice: This email and any attachments are confidential. If received in
> error please destroy and immediately notify us. Do not copy or disclose the
> contents.
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/e1cc1ec0/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 23:01:24 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Wed, 20 Mar 2013 19:01:24 -0400
Subject: high iowait
In-Reply-To: <CAF-dNMZraCu7o61Z8dLNd-ZSSsnYxs+d6gM_oHSEvP_o4QdiSg@mail.gmail.com>
References: <CABm-WmeAF015u=KZ6UJMdYGHKR0JzZYpv90-EKLusnA7w5+2Dg@mail.gmail.com>
 <20130320131228.GA17754@immer.varnish-software.com>
 <CABm-Wmcouqb7v009ACMzfznCD2eq9+n-cp9NtNb0LtRT3n-biw@mail.gmail.com>
 <CAF-dNMZraCu7o61Z8dLNd-ZSSsnYxs+d6gM_oHSEvP_o4QdiSg@mail.gmail.com>
Message-ID: <CABm-WmcgBPquxtVBbwiy4528oRaB=wtf+J36_usohQDNwox_6Q@mail.gmail.com>

Thanks, we'll give that a try tomorrow.


On Wed, Mar 20, 2013 at 5:17 PM, Jorge <jnerin at gmail.com> wrote:

> You are running linux, check that the path where the SHM log is written is
> tmpfs mounted:
> tmpfs on /usr/local/var/varnish type tmpfs (rw,size=256M)
>
> A small one will be enough:
>
> Filesystem      Size  Used Avail Use% Mounted on
> tmpfs           256M   82M  175M  32% /usr/local/var/varnish
>
> Search for a file named "_.vsm".
>
> If this file is on a "normal" filesystem (disk) linux will periodically
> flush the contents to disk, which depending on your traffic can be some
> tens of megabytes every some seconds.
>
> --
> Jorge Ner?n
> <jnerin at gmail.com>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/c7e49062/attachment.html>

From sean at monkeysnatchbanana.com  Wed Mar 20 23:08:43 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Wed, 20 Mar 2013 19:08:43 -0400
Subject: query string ordering and banning
In-Reply-To: <CABtDKm6H+1gS_0p34g3HyCGDi3Y+RGBP3DBAsTYDw1RDZcgXmA@mail.gmail.com>
References: <CABm-Wmd9CiQ3PqpB+JsXfm031KcTU2OMQmRq38xF+r8edUtNiw@mail.gmail.com>
 <OF78829F80.AC555677-ONCC257B34.0079C17C-CC257B34.0079C17D@gns.cri.nz>
 <CABtDKm6H+1gS_0p34g3HyCGDi3Y+RGBP3DBAsTYDw1RDZcgXmA@mail.gmail.com>
Message-ID: <CABm-WmdpA5r6KqccCs8HnXo11wJp2V6A-00Jt6vndxG3HeWBPw@mail.gmail.com>

Looks excellent. Will give it a try. Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/c1ba37c2/attachment.html>

From hugo.cisneiros at gmail.com  Wed Mar 20 23:25:47 2013
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Wed, 20 Mar 2013 20:25:47 -0300
Subject: Article: An intro to Varnish (portuguese)
Message-ID: <CA+KACL=h4dXRVr2L6KuwC8qd13iqRLv3gzUsNh5Fc7AGGWNbKg@mail.gmail.com>

Hi,

I've made a tutorial about varnish introducing it in brazilian portuguese.
It's a quick guide to get the first impressions on varnish (how it works,
backends, vcl basics, what to cache). If you speak portuguese or know
someone that needs, check:

http://www.devin.com.br/uma-introducao-ao-varnish/

Thanks :-)

-- 
[]'s
Hugo
www.devin.com.br
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/3efefff0/attachment.html>

From ricardo.chaves.gnu at gmail.com  Thu Mar 21 00:23:36 2013
From: ricardo.chaves.gnu at gmail.com (Ricardo Chaves)
Date: Wed, 20 Mar 2013 21:23:36 -0300
Subject: Article: An intro to Varnish (portuguese)
In-Reply-To: <CA+KACL=h4dXRVr2L6KuwC8qd13iqRLv3gzUsNh5Fc7AGGWNbKg@mail.gmail.com>
References: <CA+KACL=h4dXRVr2L6KuwC8qd13iqRLv3gzUsNh5Fc7AGGWNbKg@mail.gmail.com>
Message-ID: <CAJ03aTZ-wmJwMfniGV2TzHaQ8cRVfRwZRF9aOt-_c9mX4uM+1w@mail.gmail.com>

really good, congratulations my friend :)

2013/3/20 Hugo Cisneiros (Eitch) <hugo.cisneiros at gmail.com>

> Hi,
>
> I've made a tutorial about varnish introducing it in brazilian portuguese.
> It's a quick guide to get the first impressions on varnish (how it works,
> backends, vcl basics, what to cache). If you speak portuguese or know
> someone that needs, check:
>
> http://www.devin.com.br/uma-introducao-ao-varnish/
>
> Thanks :-)
>
> --
> []'s
> Hugo
> www.devin.com.br
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130320/941a8254/attachment.html>

From cornuwel at gmail.com  Thu Mar 21 14:21:52 2013
From: cornuwel at gmail.com (Julien Cornuwel)
Date: Thu, 21 Mar 2013 15:21:52 +0100
Subject: varnishlog broken after reinstall
Message-ID: <CAOvShqqCjP40NE_xuAtMcTvVFtQWZid6MD6L7Dp=0nsmJJjsmQ@mail.gmail.com>

Hi,

I had varnish installed from debian packages. I removed it and compiled it
from source. Then I changed my mind again and reinstalled from package. I
know, I shouldn't have.

Varnish works fine, but varnishlog doesn't.

Lauched on its own, it looks for a file that was created by the source
install :
# /usr/bin/varnishlog
Cannot open /usr/local/var/varnish/test-jcornuwe/_.vsm: No such file or
directory

If I try to force the correct file path :
# varnishlog -r /var/lib/varnish/test-jcornuwe/_.vsm
    0 Debug        - ""
    0 Debug        - ""
    0 Debug        - ""
    0 Debug        - ""
    0 Debug        - ""
    0 Debug        - ""
...

Said file is 84MB in size.

I can't find a config file that would have been overwritten and have no
idea how to get varnishlog to work again.
If there is a simple way to make varnish write its logs to a file, I'll
take it.

Any pointer?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130321/a532e03c/attachment.html>

From smstone at gmail.com  Thu Mar 21 15:34:28 2013
From: smstone at gmail.com (Steven Stone)
Date: Thu, 21 Mar 2013 11:34:28 -0400
Subject: varnishlog broken after reinstall
In-Reply-To: <CAOvShqqCjP40NE_xuAtMcTvVFtQWZid6MD6L7Dp=0nsmJJjsmQ@mail.gmail.com>
References: <CAOvShqqCjP40NE_xuAtMcTvVFtQWZid6MD6L7Dp=0nsmJJjsmQ@mail.gmail.com>
Message-ID: <CAJ0ynBiD44j1QE11G_YS=vaAfq0Z_aWYyYGF6rShRvBkugsh=g@mail.gmail.com>

Hi,

You should be able to symlink /var/lib/varnish/test-jcornuwe/_.vsm
to /usr/local/var/varnish/test-jcornuwe/_.vsm and Varnishlog will read it.
I had the same problem recently but did not find how to point Varnishlog to
the correct file without the temporary symlink fix.

Steven


On Thu, Mar 21, 2013 at 10:21 AM, Julien Cornuwel <cornuwel at gmail.com>wrote:

> Hi,
>
> I had varnish installed from debian packages. I removed it and compiled it
> from source. Then I changed my mind again and reinstalled from package. I
> know, I shouldn't have.
>
> Varnish works fine, but varnishlog doesn't.
>
> Lauched on its own, it looks for a file that was created by the source
> install :
> # /usr/bin/varnishlog
> Cannot open /usr/local/var/varnish/test-jcornuwe/_.vsm: No such file or
> directory
>
> If I try to force the correct file path :
> # varnishlog -r /var/lib/varnish/test-jcornuwe/_.vsm
>     0 Debug        - ""
>     0 Debug        - ""
>     0 Debug        - ""
>     0 Debug        - ""
>     0 Debug        - ""
>     0 Debug        - ""
> ...
>
> Said file is 84MB in size.
>
> I can't find a config file that would have been overwritten and have no
> idea how to get varnishlog to work again.
> If there is a simple way to make varnish write its logs to a file, I'll
> take it.
>
> Any pointer?
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130321/e835d7ea/attachment.html>

From Travis.Crowder at penton.com  Thu Mar 21 18:18:09 2013
From: Travis.Crowder at penton.com (Crowder, Travis)
Date: Thu, 21 Mar 2013 18:18:09 +0000
Subject: Determine if ESI object is cached
In-Reply-To: <CAJ0ynBiD44j1QE11G_YS=vaAfq0Z_aWYyYGF6rShRvBkugsh=g@mail.gmail.com>
References: <CAOvShqqCjP40NE_xuAtMcTvVFtQWZid6MD6L7Dp=0nsmJJjsmQ@mail.gmail.com>
 <CAJ0ynBiD44j1QE11G_YS=vaAfq0Z_aWYyYGF6rShRvBkugsh=g@mail.gmail.com>
Message-ID: <61E9173DF681274790BA29C89A72200C3EA60227@ksopexchm02>

Is there a way to determine if an ESI call came from cache or a backend?

I am trying to set a HTTP header on an ESI cache hit or miss, but am not having as much success as when checking for a non-ESI request.

In the recv, I set do_esi to true and the x-esi-request value, in deliver I have the code below which doesn't seem to work at all.  I don't see any X-ESI-Cache headers in varnishlog

In vcl_deliver:

if(obj.hits > 0){
        if(resp.http.x-esi-request == "Yes"){ set resp.http.X-ESI-Cache = "Cached"; }
        set resp.http.X-Cache = "Cached";
        set resp.http.X-Hits = obj.hits;
    } else {
        if(resp.http.x-esi-request == "Yes"){ set resp.http.X-ESI-Cache = "Fresh"; }
        set resp.http.X-Cache = "Fresh";
    }

    return (deliver);

Any assistance is appreciated.

Thanks,
Travis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130321/6f76d759/attachment.html>

From brentgclarklist at gmail.com  Fri Mar 22 10:48:19 2013
From: brentgclarklist at gmail.com (Brent Clark)
Date: Fri, 22 Mar 2013 12:48:19 +0200
Subject: Wordpress updates timeout
Message-ID: <514C36F3.8060602@gmail.com>

Good day.

I wonder if someone can please help me.

A client wordpress site is timing out when he does wordpress upgrades.

Wordpress sits on two backend apache web servers. And in front the 
Varnish server.
Varnish is making use of the load balancing facility.

To help trouble shoot, I have a very basic non caching config.

backend web1 { .host = "server1.host.com"; .probe = { .url = 
"/checker.html"; .interval = 5s; .timeout = 1s; .window = 5;.threshold = 
3; }
// we include time outs so uploads don't time out
  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;
}
backend web2 { .host = "server2.host.com"; .probe = { .url = 
"/checker.html"; .interval = 5s; .timeout = 1s; .window = 5;.threshold = 
3; }
// we include time outs so uploads don't time out
  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;
}

director default_director random {
   { .backend = web1; .weight = 1; }
   { .backend = web2; .weight = 1; }
}

sub vcl_recv {
    set req.backend = default_director;

   # Add a unique header containing the client address
   remove req.http.X-Forwarded-For;
   set    req.http.X-Forwarded-For = client.ip;
}

As you may see, Ive tried adding, the following to the backend directive.

  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;

But the problem still persists.

If anyone can help or point me in the right direction, it would be 
appreciated.

Kind Regards
Brent Clark




From radecki.rafal at gmail.com  Fri Mar 22 10:56:48 2013
From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=)
Date: Fri, 22 Mar 2013 11:56:48 +0100
Subject: Varnish 3.0.3 - segfault in libvarnish.so.
Message-ID: <CAHd9_iSVA-q7jm7yMUjEj0Y0NOrOUSMjXewyFak9EJ7pLk3J7A@mail.gmail.com>

Hi All.

I use varnish 3.0.3 on my production server:

rpm -qa | grep varnish
varnish-3.0.3-1.el6.x86_64
varnish-libs-3.0.3-1.el6.x86_64
varnish-libs-devel-3.0.3-1.el6.x86_64

uname -r
2.6.32-279.19.1.el6.x86_64

cat /etc/redhat-release
CentOS release 6.3 (Final)

My varnish process suddenly dies and logs:

Mar 22 09:00:07 server.local kernel: : varnishd[2085]: segfault at 0
ip 0000003f60c0c234 sp 00007fa9217cc2e0 error 4 in
libvarnish.so[3f60c00000+13000]
Mar 22 08:00:14 server.local varnishd[28424]: Child (2043) not
responding to CLI, killing it.
Mar 22 08:00:14 server.local varnishd[28424]: Child (2043) not
responding to CLI, killing it.
Mar 22 08:00:14 server.local varnishd[28424]: Child (2043) died
signal=11 (core dumped)
Mar 22 08:00:14 server.local varnishd[28424]: Child cleanup complete
Mar 22 08:00:14 server.local varnishd[28424]: child (1129) Started
Mar 22 08:00:14 server.local varnishd[28424]: Child (1129) said Child starts

I have found a similar problem:
https://www.varnish-cache.org/lists/pipermail/varnish-bugs/2013-March/005285.html
but with no answer.
Can anyone help? What steps should I make to resolve the problem?

Best regards,
Rafal Radecki.



From rabelo at raphaelr.com.br  Fri Mar 22 21:16:28 2013
From: rabelo at raphaelr.com.br (Raphael Rabelo)
Date: Fri, 22 Mar 2013 18:16:28 -0300
Subject: Varnish URL rewrite
Message-ID: <CAMOH6nD60G-Qh__w83YQytuMnA3yRqkxk9Bj1+opth4f8kCLCw@mail.gmail.com>

Hi guys!

I'm tried to rewrite an url with varnish, and i need the help of yours:

The URL www.meysite.com/something redirect to subpage "Something"

but

the url mysite.com/something redirect to home :S

How can i save all after "/" and rewrite the url with "www"
Ex.

request url = mysite.com/something
after rewrite = www.mysite.com/something

Thanks!

rabeloo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130322/228d6d8f/attachment.html>

From paulojjs at gmail.com  Sat Mar 23 03:01:51 2013
From: paulojjs at gmail.com (Paulo Silva)
Date: Sat, 23 Mar 2013 03:01:51 +0000
Subject: Varnish URL rewrite
In-Reply-To: <CAMOH6nD60G-Qh__w83YQytuMnA3yRqkxk9Bj1+opth4f8kCLCw@mail.gmail.com>
References: <CAMOH6nD60G-Qh__w83YQytuMnA3yRqkxk9Bj1+opth4f8kCLCw@mail.gmail.com>
Message-ID: <CAHJdQr=PvmpLntnm3y5FxKy6o-cd7D71JwkUGOC5Q3PxQCmwcg@mail.gmail.com>

Hi,

I think what you want is described in the wiki (see the req.http.host
example):

https://www.varnish-cache.org/trac/wiki/RedirectsAndRewrites

If you want to send a redirect to the browser try something like this:

https://www.varnish-cache.org/trac/wiki/VCLExampleRedirectInVCL


2013/3/22 Raphael Rabelo <rabelo at raphaelr.com.br>

> Hi guys!
>
> I'm tried to rewrite an url with varnish, and i need the help of yours:
>
> The URL www.meysite.com/something redirect to subpage "Something"
>
> but
>
> the url mysite.com/something redirect to home :S
>
> How can i save all after "/" and rewrite the url with "www"
> Ex.
>
> request url = mysite.com/something
> after rewrite = www.mysite.com/something
>
> Thanks!
>
> rabeloo
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
Paulo Silva <paulojjs at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130323/65963a3a/attachment.html>

From lkarsten at varnish-software.com  Sat Mar 23 11:42:55 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Sat, 23 Mar 2013 12:42:55 +0100
Subject: Varnish 3.0.3 - segfault in libvarnish.so.
In-Reply-To: <CAHd9_iSVA-q7jm7yMUjEj0Y0NOrOUSMjXewyFak9EJ7pLk3J7A@mail.gmail.com>
References: <CAHd9_iSVA-q7jm7yMUjEj0Y0NOrOUSMjXewyFak9EJ7pLk3J7A@mail.gmail.com>
Message-ID: <20130323114254.GB16897@immer.varnish-software.com>

On Fri, Mar 22, 2013 at 11:56:48AM +0100, Rafa? Radecki wrote:
> I use varnish 3.0.3 on my production server:
[..]
> Mar 22 09:00:07 server.local kernel: : varnishd[2085]: segfault at 0
> ip 0000003f60c0c234 sp 00007fa9217cc2e0 error 4 in
> libvarnish.so[3f60c00000+13000]

This should never happen and may require a bit of effort to figure out.

Please file a bug report on this in the main bugtracker:

	https://www.varnish-cache.org/trac/

-- 
With regards,
Lasse Karstensen
Varnish Software AS



From tobias.eichelbroenner at lamp-solutions.de  Sat Mar 23 15:00:24 2013
From: tobias.eichelbroenner at lamp-solutions.de (=?ISO-8859-1?Q?Tobias_Eichelbr=F6nner?=)
Date: Sat, 23 Mar 2013 16:00:24 +0100
Subject: Varnish 3.0.3 - segfault in libvarnish.so.
In-Reply-To: <CAHd9_iSVA-q7jm7yMUjEj0Y0NOrOUSMjXewyFak9EJ7pLk3J7A@mail.gmail.com>
References: <CAHd9_iSVA-q7jm7yMUjEj0Y0NOrOUSMjXewyFak9EJ7pLk3J7A@mail.gmail.com>
Message-ID: <514DC388.1070703@lamp-solutions.de>

Hi Rafal,

> Mar 22 09:00:07 server.local kernel: : varnishd[2085]: segfault at 0
> ip 0000003f60c0c234 sp 00007fa9217cc2e0 error 4 in

do you have any inline c or third party c module on use?

Sincerly,

Tobias



From mattias at nucleus.be  Sun Mar 24 21:18:52 2013
From: mattias at nucleus.be (Mattias Geniar)
Date: Sun, 24 Mar 2013 21:18:52 +0000
Subject: Varnish error page when all backends are gone?
In-Reply-To: <CAHd9_iTLOHuANi7URTq_nm2kvFf2o_uVP_WXzKMUR3Qy4Nd9_A@mail.gmail.com>
Message-ID: <CD752BE7.485A6%mattias@nucleus.be>

>Is there a way to return a custom "error" page when all backends are
>down? Or should I configure a local (on the same server as varnish)
>instance od web server with a static page and use it in case of
>problems? Hot to achieve that in VCL?

Hi Rafal,


Check out the use cases for the vcl_error() routine, that can output a
synthetic{} response that contains flat HTML for a "maintenance" or status
page. When all backends fail, the vcl_error() routine will be used.

Regards,
Mattias




From radecki.rafal at gmail.com  Mon Mar 25 10:04:09 2013
From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=)
Date: Mon, 25 Mar 2013 11:04:09 +0100
Subject: Varnish 3.0.3 - segfault in libvarnish.so.
In-Reply-To: <514DC388.1070703@lamp-solutions.de>
References: <CAHd9_iSVA-q7jm7yMUjEj0Y0NOrOUSMjXewyFak9EJ7pLk3J7A@mail.gmail.com>
 <514DC388.1070703@lamp-solutions.de>
Message-ID: <CAHd9_iTNkfTNVvg-sisuRVJEYQ5xWE+JeUTsOTCNzCWqEbvozg@mail.gmail.com>

No, we do not use any inline C. Just standard varnish from varnish
repository. My config is attached. Is there something suspicious in
it?


2013/3/23 Tobias Eichelbr?nner <tobias.eichelbroenner at lamp-solutions.de>:
> Hi Rafal,
>
>> Mar 22 09:00:07 server.local kernel: : varnishd[2085]: segfault at 0
>> ip 0000003f60c0c234 sp 00007fa9217cc2e0 error 4 in
>
> do you have any inline c or third party c module on use?
>
> Sincerly,
>
> Tobias
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: default.vcl
Type: application/octet-stream
Size: 5313 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130325/f39c6757/attachment.obj>

From radecki.rafal at gmail.com  Mon Mar 25 11:00:15 2013
From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=)
Date: Mon, 25 Mar 2013 12:00:15 +0100
Subject: Varnish 3.0.3 - segfault in libvarnish.so.
In-Reply-To: <20130323114254.GB16897@immer.varnish-software.com>
References: <CAHd9_iSVA-q7jm7yMUjEj0Y0NOrOUSMjXewyFak9EJ7pLk3J7A@mail.gmail.com>
 <20130323114254.GB16897@immer.varnish-software.com>
Message-ID: <CAHd9_iTZ_p2k8ZQN6sf=sV5C4tKfSg0bNn64OsNw986q=kXgfA@mail.gmail.com>

Hi.

I've added a ticket: https://www.varnish-cache.org/trac/ticket/1287 .

Best regards,
Rafal.

2013/3/23 Lasse Karstensen <lkarsten at varnish-software.com>:
> On Fri, Mar 22, 2013 at 11:56:48AM +0100, Rafa? Radecki wrote:
>> I use varnish 3.0.3 on my production server:
> [..]
>> Mar 22 09:00:07 server.local kernel: : varnishd[2085]: segfault at 0
>> ip 0000003f60c0c234 sp 00007fa9217cc2e0 error 4 in
>> libvarnish.so[3f60c00000+13000]
>
> This should never happen and may require a bit of effort to figure out.
>
> Please file a bug report on this in the main bugtracker:
>
>         https://www.varnish-cache.org/trac/
>
> --
> With regards,
> Lasse Karstensen
> Varnish Software AS



From mike_noren2002 at yahoo.co.uk  Mon Mar 25 17:07:04 2013
From: mike_noren2002 at yahoo.co.uk (Mike Noren)
Date: Mon, 25 Mar 2013 17:07:04 +0000 (GMT)
Subject: All calls redirecting to backend port
Message-ID: <1364231224.48157.YahooMailClassic@web172306.mail.ir2.yahoo.com>

Hello all,
I'm completely new to Varnish, if this is a silly question I apologize and ask to please be directed to a newbie forum.

Anyway, I've a LAMP server on which I'm testing Varnish. Apache is set to port 8008 in ports-conf and 000-default, Varnish is set to listen on port 80 in /etc/default/varnish and told that the backend is on port 8008 in default.vcl. 

Everything seems to be working, Varnishhist and Live Headers confirm that Varnish is caching the frontpage, but clicking on any link redirects the URL to the backend port, bypassing Varnish. 
E.g. clicking on link from the front page to http://www.mysite.com/mypage.php?item=1234 sends the browser to http://www.mysite.com:8008/mypage.php?item=1234. 
If I manually enter the URL in the browser without the port number then the page loads through Varnish just fine, but clicking any link automatically appends the port number.

I assume this is a configuration error, any ideas what's going on and how I can fix it?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130325/32dbd6e5/attachment.html>

From Raul.Rangel at disney.com  Mon Mar 25 17:41:43 2013
From: Raul.Rangel at disney.com (Rangel, Raul)
Date: Mon, 25 Mar 2013 10:41:43 -0700
Subject: All calls redirecting to backend port
In-Reply-To: <1364231224.48157.YahooMailClassic@web172306.mail.ir2.yahoo.com>
References: <1364231224.48157.YahooMailClassic@web172306.mail.ir2.yahoo.com>
Message-ID: <2465AAEEC8B8A242B26ED5F44BCA805F2610319D55@SM-CALA-VXMB04A.swna.wdpr.disney.com>

You need to make sure your application support being behind a reverse proxy. Two solutions I can think of to your problem are 1) Make all urls relative. i.e) /mypage.php?item=1234 instead of using the full url. 2) Have varnish append a X-Forwarded-Host header in vcl_recv. Then make your app use that as the host for the urls.

Raul

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Noren
Sent: Monday, March 25, 2013 11:07 AM
To: varnish-misc at varnish-cache.org
Subject: All calls redirecting to backend port

Hello all,
I'm completely new to Varnish, if this is a silly question I apologize and ask to please be directed to a newbie forum.

Anyway, I've a LAMP server on which I'm testing Varnish. Apache is set to port 8008 in ports-conf and 000-default, Varnish is set to listen on port 80 in /etc/default/varnish and told that the backend is on port 8008 in default.vcl.

Everything seems to be working, Varnishhist and Live Headers confirm that Varnish is caching the frontpage, but clicking on any link redirects the URL to the backend port, bypassing Varnish.
E.g. clicking on link from the front page to http://www.mysite.com/mypage.php?item=1234 sends the browser to http://www.mysite.com:8008/mypage.php?item=1234.
If I manually enter the URL in the browser without the port number then the page loads through Varnish just fine, but clicking any link automatically appends the port number.

I assume this is a configuration error, any ideas what's going on and how I can fix it?


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130325/cd198e59/attachment.html>

From StephenG at glam.com  Mon Mar 25 18:00:05 2013
From: StephenG at glam.com (Stephen Gazard)
Date: Mon, 25 Mar 2013 18:00:05 +0000
Subject: All calls redirecting to backend port
In-Reply-To: <2465AAEEC8B8A242B26ED5F44BCA805F2610319D55@SM-CALA-VXMB04A.swna.wdpr.disney.com>
References: <1364231224.48157.YahooMailClassic@web172306.mail.ir2.yahoo.com>
 <2465AAEEC8B8A242B26ED5F44BCA805F2610319D55@SM-CALA-VXMB04A.swna.wdpr.disney.com>
Message-ID: <D19C0500E58C3F42B2C121A7BDABB730521AA3CD@gmmxmb03z03.projecty.com>

Hello,

From: varnish-misc-bounces at varnish-cache.org<mailto:varnish-misc-bounces at varnish-cache.org> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Noren
Sent: Monday, March 25, 2013 11:07 AM
To: varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
Subject: All calls redirecting to backend port

Hello all,
I'm completely new to Varnish, if this is a silly question I apologize and ask to please be directed to a newbie forum.

Anyway, I've a LAMP server on which I'm testing Varnish. Apache is set to port 8008 in ports-conf and 000-default, Varnish is set to listen on port 80 in /etc/default/varnish and told that the backend is on port 8008 in default.vcl.

Everything seems to be working, Varnishhist and Live Headers confirm that Varnish is caching the frontpage, but clicking on any link redirects the URL to the backend port, bypassing Varnish.
E.g. clicking on link from the front page to http://www.mysite.com/mypage.php?item=1234 sends the browser to http://www.mysite.com:8008/mypage.php?item=1234.
If I manually enter the URL in the browser without the port number then the page loads through Varnish just fine, but clicking any link automatically appends the port number.

I assume this is a configuration error, any ideas what's going on and how I can fix it?
It?s a server configuration error.  Assuming varnish & apache are on the same server

-          varnish listening on the public IP address port 80

-          apache listening on 127.0.0.1 on port 80.  If it?s on a separate server, modify Apache to listen on port 80 on the secondary server?s IP
Otherwise you get apache serving links via Varnish and it will put the non-standard port it thinks it sits on.

See http://ocaoimh.ie/2011/08/09/speed-up-wordpress-with-apache-and-varnish/

Regards,
Stephen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130325/60353360/attachment.html>

From ddierker at stltoday.com  Mon Mar 25 18:20:36 2013
From: ddierker at stltoday.com (Dave Dierker)
Date: Mon, 25 Mar 2013 13:20:36 -0500
Subject: How to you display the 503 error as sent from the backend....
Message-ID: <5BF7D4003607C04BAF666AEBCCD8B54D09C2DF23@LEESTL-EX1.stl.leeent.net>

vcl_error, for 503 errors from the backend, appears to cloak what the
backend server sends.

 There are cases where presenting to user the full 503 response, as sent
by the backend, makes sense.

As a trivial example: a PHP script that returns a custom 503 error
response is completely cloaked/hidden by the varnish code:
===
<?php
header('HTTP/1.1 503 Smeagol ate your puppy!');
header('Status: 503 Smeagol ate your puppy!');header('Retry-After:
300');//300 seconds
header('Content-Type: text/plain');
?>503 Smeagol ate your puppy!
===

Whatever the vcl_error generates in the synthetic response is sent...
and if a synthetic response isn't provided nothing gets sent to the
client (last I tried it.)

Even the status text from the obj.response code returns "Service
Unavailable" instead of "Smeagol ate your puppy!".

Is there any way to get varnish to pass the 503 through as it came from
the backend?




From apj at mutt.dk  Mon Mar 25 18:53:22 2013
From: apj at mutt.dk (Andreas Plesner Jacobsen)
Date: Mon, 25 Mar 2013 19:53:22 +0100
Subject: How to you display the 503 error as sent from the backend....
In-Reply-To: <5BF7D4003607C04BAF666AEBCCD8B54D09C2DF23@LEESTL-EX1.stl.leeent.net>
References: <5BF7D4003607C04BAF666AEBCCD8B54D09C2DF23@LEESTL-EX1.stl.leeent.net>
Message-ID: <20130325185322.GR19694@nerd.dk>

On Mon, Mar 25, 2013 at 01:20:36PM -0500, Dave Dierker wrote:

> vcl_error, for 503 errors from the backend, appears to cloak what the
> backend server sends.

Nope. If the backend 503s, that will be sent to the client.

-- 
Andreas



From radecki.rafal at gmail.com  Tue Mar 26 07:45:55 2013
From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=)
Date: Tue, 26 Mar 2013 08:45:55 +0100
Subject: Varnish error page when all backends are gone?
In-Reply-To: <CD752BE7.485A6%mattias@nucleus.be>
References: <CAHd9_iTLOHuANi7URTq_nm2kvFf2o_uVP_WXzKMUR3Qy4Nd9_A@mail.gmail.com>
 <CD752BE7.485A6%mattias@nucleus.be>
Message-ID: <CAHd9_iT0PbXeoaBKD0Yjk4L7dF8m8spAyJ5nZFPOa+LZuDzXzA@mail.gmail.com>

We are using in vcl_error:

sub vcl_error {
        if (obj.status == 700) {
                set obj.http.Location = obj.response;
                set obj.status = 301;
                set obj.response = "Moved Permanently";
        } else if ( obj.status >= 500 && obj.status <= 505) {
                set obj.http.Content-Type = "text/html; charset=utf-8";
                synthetic
std.fileread("/etc/varnish/error_pages/503/index.html");
        } else if (obj.status == 403 || obj.status == 404 ||
obj.status == 400) {
                set obj.http.Location = "http://" + req.http.host +
"/no-route/";
                set obj.status = 301;
        }

        return (deliver);
}

But it does not work as expected. Any ideas why?

Best regards,
Rafal.

2013/3/24 Mattias Geniar <mattias at nucleus.be>:
>>Is there a way to return a custom "error" page when all backends are
>>down? Or should I configure a local (on the same server as varnish)
>>instance od web server with a static page and use it in case of
>>problems? Hot to achieve that in VCL?
>
> Hi Rafal,
>
>
> Check out the use cases for the vcl_error() routine, that can output a
> synthetic{} response that contains flat HTML for a "maintenance" or status
> page. When all backends fail, the vcl_error() routine will be used.
>
> Regards,
> Mattias
>



From matthewm at boedicker.org  Tue Mar 26 08:08:08 2013
From: matthewm at boedicker.org (Matthew M. Boedicker)
Date: Tue, 26 Mar 2013 01:08:08 -0700
Subject: getting near real time request counts
Message-ID: <CANfC_4Lh0ODUCKngJP603Z1u77cJXELSu3uJLCDc-gp2NYa7eQ@mail.gmail.com>

What is the best way to get real time or near real time request counts per
url? I want to maintain this data in a database, redis or something else
that my app can use.

I have considered:

- writing something to parse the log generated by varnishncsa

- writing a replacement for varnishncsa that reads the shared memory logs
and does something else instead of writing a file

- using a varnish module to talk to redis or memcache

- some VCL hack to pass data through to the backend
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130326/352d6c48/attachment.html>

From mattias at nucleus.be  Tue Mar 26 08:10:05 2013
From: mattias at nucleus.be (Mattias Geniar)
Date: Tue, 26 Mar 2013 08:10:05 +0000
Subject: Varnish error page when all backends are gone?
In-Reply-To: <CAHd9_iT0PbXeoaBKD0Yjk4L7dF8m8spAyJ5nZFPOa+LZuDzXzA@mail.gmail.com>
Message-ID: <CD7715FA.48C88%mattias@nucleus.be>

>But it does not work as expected. Any ideas why?

You want to catch the situation when your backends are all unavailable?
Then the obj.status probably won't be > 500 (error codes returned by the
backend).

End your if/else block with a final else statement to catch all other
situations (where the status code is unknown), use the std.fileread()
there and that'll most likely catch the situation where your backends are
unavailable.

Regards,
Mattias




From mike_noren2002 at yahoo.co.uk  Tue Mar 26 08:38:35 2013
From: mike_noren2002 at yahoo.co.uk (Mike Noren)
Date: Tue, 26 Mar 2013 08:38:35 +0000 (GMT)
Subject: All calls redirecting to backend port (Stephen Gazard)
In-Reply-To: <mailman.164.1364234416.2321.varnish-misc@varnish-cache.org>
Message-ID: <1364287115.25250.YahooMailClassic@web172306.mail.ir2.yahoo.com>

Thank you to everyone, and especially Stephen Gazard - you were right, the problem was that Apache was set to use *:8080, and was solved when I specified that Apache should use 127.0.0.1:80.

A final quick note for future users who might read this: remember to also clear your _browser_ cache when testing any changes you've made. Had me puzzled for a bit.

And again, thanks!

--- On Mon, 25/3/13, varnish-misc-request at varnish-cache.org <varnish-misc-request at varnish-cache.org> wrote:

From: varnish-misc-request at varnish-cache.org <varnish-misc-request at varnish-cache.org>
Subject: varnish-misc Digest, Vol 84, Issue 26
To: varnish-misc at varnish-cache.org
Date: Monday, 25 March, 2013, 19:00

Send varnish-misc mailing list submissions to
??? varnish-misc at varnish-cache.org

To subscribe or unsubscribe via the World Wide Web, visit
??? https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
or, via email, send a message with subject or body 'help' to
??? varnish-misc-request at varnish-cache.org

You can reach the person managing the list at
??? varnish-misc-owner at varnish-cache.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of varnish-misc digest..."


Today's Topics:

???1. Re: Varnish 3.0.3 - segfault in libvarnish.so. (Rafa? Radecki)
???2. All calls redirecting to backend port (Mike Noren)
???3. RE: All calls redirecting to backend port (Rangel, Raul)
???4. RE: All calls redirecting to backend port (Stephen Gazard)


----------------------------------------------------------------------

Message: 1
Date: Mon, 25 Mar 2013 12:00:15 +0100
From: Rafa? Radecki <radecki.rafal at gmail.com>
To: Lasse Karstensen <lkarsten at varnish-software.com>
Cc: varnish-misc at varnish-cache.org
Subject: Re: Varnish 3.0.3 - segfault in libvarnish.so.
Message-ID:
??? <CAHd9_iTZ_p2k8ZQN6sf=sV5C4tKfSg0bNn64OsNw986q=kXgfA at mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-2

Hi.

I've added a ticket: https://www.varnish-cache.org/trac/ticket/1287 .

Best regards,
Rafal.

2013/3/23 Lasse Karstensen <lkarsten at varnish-software.com>:
> On Fri, Mar 22, 2013 at 11:56:48AM +0100, Rafa? Radecki wrote:
>> I use varnish 3.0.3 on my production server:
> [..]
>> Mar 22 09:00:07 server.local kernel: : varnishd[2085]: segfault at 0
>> ip 0000003f60c0c234 sp 00007fa9217cc2e0 error 4 in
>> libvarnish.so[3f60c00000+13000]
>
> This should never happen and may require a bit of effort to figure out.
>
> Please file a bug report on this in the main bugtracker:
>
>? ? ? ???https://www.varnish-cache.org/trac/
>
> --
> With regards,
> Lasse Karstensen
> Varnish Software AS



------------------------------

Message: 2
Date: Mon, 25 Mar 2013 17:07:04 +0000 (GMT)
From: Mike Noren <mike_noren2002 at yahoo.co.uk>
To: varnish-misc at varnish-cache.org
Subject: All calls redirecting to backend port
Message-ID:
??? <1364231224.48157.YahooMailClassic at web172306.mail.ir2.yahoo.com>
Content-Type: text/plain; charset="utf-8"

Hello all,
I'm completely new to Varnish, if this is a silly question I apologize and ask to please be directed to a newbie forum.

Anyway, I've a LAMP server on which I'm testing Varnish. Apache is set to port 8008 in ports-conf and 000-default, Varnish is set to listen on port 80 in /etc/default/varnish and told that the backend is on port 8008 in default.vcl. 

Everything seems to be working, Varnishhist and Live Headers confirm that Varnish is caching the frontpage, but clicking on any link redirects the URL to the backend port, bypassing Varnish. 
E.g. clicking on link from the front page to http://www.mysite.com/mypage.php?item=1234 sends the browser to http://www.mysite.com:8008/mypage.php?item=1234. 
If I manually enter the URL in the browser without the port number then the page loads through Varnish just fine, but clicking any link automatically appends the port number.

I assume this is a configuration error, any ideas what's going on and how I can fix it?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20130325/32dbd6e5/attachment-0001.html>

------------------------------

Message: 3
Date: Mon, 25 Mar 2013 10:41:43 -0700
From: "Rangel, Raul" <Raul.Rangel at disney.com>
To: "'Mike Noren'" <mike_noren2002 at yahoo.co.uk>,
??? "'varnish-misc at varnish-cache.org'" <varnish-misc at varnish-cache.org>
Subject: RE: All calls redirecting to backend port
Message-ID:
??? <2465AAEEC8B8A242B26ED5F44BCA805F2610319D55 at SM-CALA-VXMB04A.swna.wdpr.disney.com>
??? 
Content-Type: text/plain; charset="utf-8"

You need to make sure your application support being behind a reverse proxy. Two solutions I can think of to your problem are 1) Make all urls relative. i.e) /mypage.php?item=1234 instead of using the full url. 2) Have varnish append a X-Forwarded-Host header in vcl_recv. Then make your app use that as the host for the urls.

Raul

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Noren
Sent: Monday, March 25, 2013 11:07 AM
To: varnish-misc at varnish-cache.org
Subject: All calls redirecting to backend port

Hello all,
I'm completely new to Varnish, if this is a silly question I apologize and ask to please be directed to a newbie forum.

Anyway, I've a LAMP server on which I'm testing Varnish. Apache is set to port 8008 in ports-conf and 000-default, Varnish is set to listen on port 80 in /etc/default/varnish and told that the backend is on port 8008 in default.vcl.

Everything seems to be working, Varnishhist and Live Headers confirm that Varnish is caching the frontpage, but clicking on any link redirects the URL to the backend port, bypassing Varnish.
E.g. clicking on link from the front page to http://www.mysite.com/mypage.php?item=1234 sends the browser to http://www.mysite.com:8008/mypage.php?item=1234.
If I manually enter the URL in the browser without the port number then the page loads through Varnish just fine, but clicking any link automatically appends the port number.

I assume this is a configuration error, any ideas what's going on and how I can fix it?


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20130325/cd198e59/attachment-0001.html>

------------------------------

Message: 4
Date: Mon, 25 Mar 2013 18:00:05 +0000
From: Stephen Gazard <StephenG at glam.com>
To: "'varnish-misc at varnish-cache.org'"
??? <varnish-misc at varnish-cache.org>
Subject: RE: All calls redirecting to backend port
Message-ID:
??? <D19C0500E58C3F42B2C121A7BDABB730521AA3CD at gmmxmb03z03.projecty.com>
Content-Type: text/plain; charset="utf-8"

Hello,

From: varnish-misc-bounces at varnish-cache.org<mailto:varnish-misc-bounces at varnish-cache.org> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Noren
Sent: Monday, March 25, 2013 11:07 AM
To: varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
Subject: All calls redirecting to backend port

Hello all,
I'm completely new to Varnish, if this is a silly question I apologize and ask to please be directed to a newbie forum.

Anyway, I've a LAMP server on which I'm testing Varnish. Apache is set to port 8008 in ports-conf and 000-default, Varnish is set to listen on port 80 in /etc/default/varnish and told that the backend is on port 8008 in default.vcl.

Everything seems to be working, Varnishhist and Live Headers confirm that Varnish is caching the frontpage, but clicking on any link redirects the URL to the backend port, bypassing Varnish.
E.g. clicking on link from the front page to http://www.mysite.com/mypage.php?item=1234 sends the browser to http://www.mysite.com:8008/mypage.php?item=1234.
If I manually enter the URL in the browser without the port number then the page loads through Varnish just fine, but clicking any link automatically appends the port number.

I assume this is a configuration error, any ideas what's going on and how I can fix it?
It?s a server configuration error.? Assuming varnish & apache are on the same server

-? ? ? ? ? varnish listening on the public IP address port 80

-? ? ? ? ? apache listening on 127.0.0.1 on port 80.? If it?s on a separate server, modify Apache to listen on port 80 on the secondary server?s IP
Otherwise you get apache serving links via Varnish and it will put the non-standard port it thinks it sits on.

See http://ocaoimh.ie/2011/08/09/speed-up-wordpress-with-apache-and-varnish/

Regards,
Stephen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20130325/60353360/attachment.html>

------------------------------

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

End of varnish-misc Digest, Vol 84, Issue 26
********************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130326/151b7b9f/attachment.html>

From spam at onlineconcepts.com  Tue Mar 26 18:00:12 2013
From: spam at onlineconcepts.com (John)
Date: Tue, 26 Mar 2013 14:00:12 -0400
Subject: Geolocation without compiling Varnish from source
Message-ID: <f877a3bc6f95b5bc9422e1e3c97ec58e.squirrel@sm.webmail.pair.com>

I would like to geolocate our website visitors by country. Varnish can
send the ISO country code to our backend, which will then serve the users
content specific to their region of the world.

VERSION INFO
We're running Red Hat Enterprise Linux 6.2 (RHEL).

"varnishd -V" produces:

varnishd (varnish-3.0.2 revision 55e70a4)

STEPS TAKEN THUS FAR
I'm attempting to follow the solution described here.
https://github.com/guybedford/varnish-geoip

1) Installed the MaxMind Geolocation API for C

2) Installed the MaxMind Binary GeoIP Country DB

3) Downloaded GeoIP Plugin for Varnish per
https://www.varnish-cache.org/trac/wiki/GeoipUsingInlineC

I don't think #3 is the solution we ultimately want to use, but running
"make test" produced the following output, indicating that the MaxMind
Geolocation API is probably (?) installed correctly.

Testing IPs:
195.110.128.11 => IT (OK)
157.166.224.25 => US (OK)
212.58.224.138 => GB (OK)
80.91.37.210 => NO (OK)
192.168.0.1 => Unknown (OK)
(null) => Unknown (OK)

AVOIDING COMPILATION
I'm more of a developer than a Linux sysadmin. I would like to continue to
maintain Varnish versions using standard yum commands, rather than
compiling from source.

I followed these instructions https://github.com/guybedford/varnish-geoip
and then ran this command.

varnishd -f /etc/varnish/default.vcl -p 'cc_command=exec cc -fpic -shared
-Wl,-x -L/usr/local/lib -lGeoIP -o %o %s'

This produces an error.

----------------------
Message from dlopen:
Compiled VCL program failed to load:
  ./vcl.8BAo5_pH.so: undefined symbol: vcl_geoip_set_header
Running dlopen failed, exit 1
VCL compilation failed
----------------------

I found the links below useful, but none of them fixed my problem. The one
solution I have not tried is creating a VMOD because it seems to require
recompiling Varnish.

https://github.com/guybedford/varnish-geoip
https://github.com/cosimo/varnish-geoip/
http://drcarter.info/2010/07/another-way-to-link-varnish-and-maxmind-geoip/
https://www.varnish-cache.org/trac/wiki/GeoipUsingInlineC
http://www.mursalat.net/tag/varnish3
https://plus.google.com/112740765000604900893/posts/DnomVJRMG4E

Thanks for any advice,
John



From james at ifixit.com  Tue Mar 26 18:06:08 2013
From: james at ifixit.com (James Pearson)
Date: Tue, 26 Mar 2013 11:06:08 -0700
Subject: getting near real time request counts
In-Reply-To: <CANfC_4Lh0ODUCKngJP603Z1u77cJXELSu3uJLCDc-gp2NYa7eQ@mail.gmail.com>
References: <CANfC_4Lh0ODUCKngJP603Z1u77cJXELSu3uJLCDc-gp2NYa7eQ@mail.gmail.com>
Message-ID: <1364320861-sup-8757@geror.local>

Excerpts from Matthew M. Boedicker's message of 2013-03-26 01:08:08 -0700:
> What is the best way to get real time or near real time request counts per
> url? I want to maintain this data in a database, redis or something else
> that my app can use.
> 
> I have considered:
> 
> - writing something to parse the log generated by varnishncsa
> 
> - writing a replacement for varnishncsa that reads the shared memory logs
> and does something else instead of writing a file
> 
> - using a varnish module to talk to redis or memcache
> 
> - some VCL hack to pass data through to the backend

While there is at least a memcached module, I tend to be wary of doing
reporting directly within varnish, because a slow connection to memcached could
slow down request responses.  OTOH, a separate log-parsing utility could get
behind and never catch up, if you're not careful.

How close to realtime do you need?  What sorts of grouping are you wanting
(e.g. the number of requests in the last minute)?  How long do you need to
store this data?  Are you ok with losing precision as time passes, as happens
in rrdtool and friends?  Are you concerned with all urls, or only certain ones,
or only the top x%?  How important is it to you to not lose any data (is it ok
to send stats around via UDP)?

 - P



From r.egglestone at auckland.ac.nz  Wed Mar 27 02:06:30 2013
From: r.egglestone at auckland.ac.nz (Robert Egglestone)
Date: Wed, 27 Mar 2013 02:06:30 +0000
Subject: Queue at 130 concurrent requests?
Message-ID: <EDA44FFDD9129945880F2759A34FF53E66CB6D37@uxcn10-tdc01.UoA.auckland.ac.nz>

Hi,

I have Varnish 3.0.3 on RHEL6 configured to use 2000 worker threads (both min and max). While testing, at the point of exactly 130 concurrent requests, I consistently see the maximum response time jump up, and continue to grow as the number of concurrent requests increases.  I'm wondering if it's hitting a limit at this point causing the requests to be queued.

I need to be able to process up to 500 concurrent requests with fairly consistent performance to take this through to production. The material I've seen online suggests that this should be no problem, so I'm wondering if I've overlooked a configuration setting.

The backend is an Apache server returning a redirect. I've tested directly against the backend too and that shows the maximum response times increasing fairly linear with the number of concurrent requests - the maximum response time does not jump up like this.

I've included details below of the test results and configuration below.

Thanks,
Robert Egglestone | Application Engineering Team Lead | The University of Auckland
r.egglestone at auckland.ac.nz<mailto:r.egglestone at auckland.ac.nz> | ext: 84624 | m: +64 21 240 8992 | Level 2, 58 Symonds St, Auckland, NZ

The test results are:

$ ab -n 10000 -c 110 localhost:6081/

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   2.0      1      10
Processing:     2    7   1.9      7      24
Waiting:        0    6   2.1      6      24
Total:          5    8   1.9      8      24

$ ab -n 10000 -c 129 localhost:6081/

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    4   3.1      5      10
Processing:     2    8   2.0      8      25
Waiting:        0    7   2.5      6      24
Total:          6   12   2.5     13      25

$ ab -n 10000 -c 130 localhost:6081/

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   2.6      1      10
Processing:     2    8   3.6      7     215
Waiting:        0    7   3.7      7     214
Total:          6   10   3.8     10     215

$ ab -n 10000 -c 150 localhost:6081/

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    3   2.7      2      12
Processing:     2    8  12.2      7     523
Waiting:        0    7  12.3      6     523
Total:          6   11  12.3     10     523


The Varnish startup parameters I'm using are:

/usr/sbin/varnishd -F -a :6081
-f /etc/varnish/webroute.vcl
-T 127.0.0.1:6082
-t 120
-w 2000,2000,120
-u varnish -g varnish
-S /etc/varnish/secret
-s file,/var/lib/varnish/varnish_storage.bin,1G

The Varnish configuration is:

backend apache {
  .host = "127.0.0.1";
  .port = "8080";
}

#
# Serve stale content while retrieving content from backend
# https://www.varnish-cache.org/trac/wiki/VCLExampleGrace
#

sub vcl_recv {
  set req.grace = 30s;
}

sub vcl_fetch {
  set beresp.grace = 30s;
}

#
# Normalize cookies
#

sub vcl_recv {
  # Remove has_js and Google Analytics __* cookies.
  set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");
  # Remove a ";" prefix, if present.
  set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");

  # Cookies
  if (req.http.Cookie == "") {
    remove req.http.Cookie;
  }
}

#
# Normalize Accept-Encoding header
#

sub vcl_recv {
  if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
      # No point in compressing these
      remove req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate") {
      set req.http.Accept-Encoding = "deflate";
    } else {
      # unknown algorithm
      remove req.http.Accept-Encoding;
    }
  }
}

#
# Allow items to be purged from the cache
#

acl purgers {
  "127.0.0.1";
}

sub vcl_recv {
  if (req.request == "PURGE") {
    if (!client.ip ~ purgers) {
      error 405 "Method not allowed";
    }
    return (lookup);
  }
}

sub vcl_hit {
  if (req.request == "PURGE") {
    purge;
    error 200 "Purged";
  }
}

sub vcl_miss {
  if (req.request == "PURGE") {
    purge;
    error 404 "Not in cache";
  }
}

sub vcl_pass {
  if (req.request == "PURGE") {
    error 502 "PURGE on a passed object";
  }
}

#
# ESI support
# http://www.w3.org/TR/edge-arch
#
# http://symfony.com/doc/2.0/cookbook/cache/varnish.html
#

sub vcl_recv {
  set req.http.Surrogate-Capability = "webroute=ESI/1.0";
}

sub vcl_fetch {
  if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
    unset beresp.http.Surrogate-Control;
    set beresp.http.X-ESI = "true";
    set beresp.do_esi = true;
  }
}

#
# Debugging
#

sub vcl_deliver {
  if (obj.hits > 0) {
    set resp.http.X-Cache = "HIT";
    set resp.http.X-Cache-Hits = obj.hits;
  } else {
    set resp.http.X-Cache = "MISS";
  }
}


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130327/ac738c79/attachment.html>

From matthewm at boedicker.org  Wed Mar 27 06:17:07 2013
From: matthewm at boedicker.org (Matthew M. Boedicker)
Date: Tue, 26 Mar 2013 23:17:07 -0700
Subject: getting near real time request counts
In-Reply-To: <1364320861-sup-8757@geror.local>
References: <CANfC_4Lh0ODUCKngJP603Z1u77cJXELSu3uJLCDc-gp2NYa7eQ@mail.gmail.com>
 <1364320861-sup-8757@geror.local>
Message-ID: <CANfC_4JwaxyztUOmrKvX5doDVLz1AiOPFSW6GYnfM=TcQaFRKg@mail.gmail.com>

> How close to realtime do you need?  What sorts of grouping are you wanting
> (e.g. the number of requests in the last minute)?  How long do you need to
> store this data?  Are you ok with losing precision as time passes, as happens
> in rrdtool and friends?  Are you concerned with all urls, or only certain ones,
> or only the top x%?  How important is it to you to not lose any data (is it ok
> to send stats around via UDP)?

I had the same concern about slowing down responses, so it would have
to be very lightweight. Maybe UDP as you said or putting something
into a queue.

I want to track total hit counts of image urls to track their
popularity. A single value per url is good enough and it can be pretty
far from real time.

Almost all of them are cached in varnish so varnish is the only thing
that knows that they have been requested.



From ross at trademe.co.nz  Wed Mar 27 10:35:31 2013
From: ross at trademe.co.nz (Ross Brown)
Date: Wed, 27 Mar 2013 10:35:31 +0000
Subject: getting near real time request counts
In-Reply-To: <CANfC_4JwaxyztUOmrKvX5doDVLz1AiOPFSW6GYnfM=TcQaFRKg@mail.gmail.com>
References: <CANfC_4Lh0ODUCKngJP603Z1u77cJXELSu3uJLCDc-gp2NYa7eQ@mail.gmail.com>
 <1364320861-sup-8757@geror.local>
 <CANfC_4JwaxyztUOmrKvX5doDVLz1AiOPFSW6GYnfM=TcQaFRKg@mail.gmail.com>
Message-ID: <6ED006BA952A6E4F802D61A21A5D4F054C90AF@TMMAIL1.trademe.local>

This may be what you need, even if not it may send you off in an interesting direction :)

http://jiboumans.wordpress.com/2013/02/27/realtime-stats-from-varnish/


-----Original Message-----
From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Matthew M. Boedicker
Sent: Wednesday, 27 March 2013 7:17 p.m.
To: James Pearson
Cc: varnish-misc
Subject: Re: getting near real time request counts

> How close to realtime do you need?  What sorts of grouping are you wanting
> (e.g. the number of requests in the last minute)?  How long do you need to
> store this data?  Are you ok with losing precision as time passes, as happens
> in rrdtool and friends?  Are you concerned with all urls, or only certain ones,
> or only the top x%?  How important is it to you to not lose any data (is it ok
> to send stats around via UDP)?

I had the same concern about slowing down responses, so it would have
to be very lightweight. Maybe UDP as you said or putting something
into a queue.

I want to track total hit counts of image urls to track their
popularity. A single value per url is good enough and it can be pretty
far from real time.

Almost all of them are cached in varnish so varnish is the only thing
that knows that they have been requested.

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From mhettwer at team.mobile.de  Wed Mar 27 14:21:39 2013
From: mhettwer at team.mobile.de (Hettwer, Marian)
Date: Wed, 27 Mar 2013 14:21:39 +0000
Subject: backend.set_health bug
Message-ID: <6DD1CF2A31C9724E9425CDB5511A92FC1C683DC7@DUB-EXDDA-S32.corp.ebay.com>

Hi Ho,

seems like the parser for specifying the backend is buggy.
Should I file a bug report?

But first the details. Look what's happening.

We have this backend list:

varnish> backend.list
200        
Backend name                   Refs   Admin      Probe
speedy(10.47.28.60,,9062)      1      probe      Healthy 5/5
speedy(10.47.28.61,,9062)      1      probe      Sick 0/5
speedy47_1(10.47.60.1,,9062)   1      probe      Healthy 8/8
speedy47_10(10.47.60.10,,9062) 1      probe      Healthy 8/8
speedy47_11(10.47.60.11,,9062) 1      probe      Healthy 8/8
speedy47_12(10.47.60.12,,9062) 1      probe      Healthy 8/8
speedy47_13(10.47.60.13,,9062) 1      probe      Healthy 8/8
speedy47_14(10.47.60.14,,9062) 1      probe      Healthy 8/8
speedy47_15(10.47.60.15,,9062) 1      probe      Healthy 8/8
speedy47_16(10.47.60.16,,9062) 1      probe      Healthy 8/8
speedy47_17(10.47.60.17,,9062) 1      probe      Healthy 8/8
speedy47_18(10.47.60.18,,9062) 1      probe      Healthy 8/8
speedy47_2(10.47.60.2,,9062)   1      probe      Healthy 8/8
speedy47_3(10.47.60.3,,9062)   1      probe      Healthy 8/8
speedy47_4(10.47.60.4,,9062)   1      probe      Healthy 8/8
speedy47_5(10.47.60.5,,9062)   1      probe      Healthy 8/8
speedy47_6(10.47.60.6,,9062)   1      probe      Healthy 8/8
speedy47_7(10.47.60.7,,9062)   1      probe      Healthy 8/8
speedy47_8(10.47.60.8,,9062)   1      probe      Healthy 8/8
speedy47_9(10.47.60.9,,9062)   1      probe      Healthy 8/8

And are now setting speedy47_1 to sick:

varnish> backend.set_health speedy47_1(10.47.60.1) sick
200        


Now let's have a look at the backend list:

varnish> backend.list
200        
Backend name                   Refs   Admin      Probe
speedy(10.47.28.60,,9062)      1      probe      Healthy 5/5
speedy(10.47.28.61,,9062)      1      probe      Sick 0/5
speedy47_1(10.47.60.1,,9062)   1      sick       Healthy 8/8
speedy47_10(10.47.60.10,,9062) 1      sick       Healthy 8/8
speedy47_11(10.47.60.11,,9062) 1      sick       Healthy 8/8
speedy47_12(10.47.60.12,,9062) 1      sick       Healthy 8/8
speedy47_13(10.47.60.13,,9062) 1      sick       Healthy 8/8
speedy47_14(10.47.60.14,,9062) 1      sick       Healthy 8/8
speedy47_15(10.47.60.15,,9062) 1      sick       Healthy 8/8
speedy47_16(10.47.60.16,,9062) 1      sick       Healthy 8/8
speedy47_17(10.47.60.17,,9062) 1      sick       Healthy 8/8
speedy47_18(10.47.60.18,,9062) 1      sick       Healthy 8/8
speedy47_2(10.47.60.2,,9062)   1      probe      Healthy 8/8
speedy47_3(10.47.60.3,,9062)   1      probe      Healthy 8/8
speedy47_4(10.47.60.4,,9062)   1      probe      Healthy 8/8
speedy47_5(10.47.60.5,,9062)   1      probe      Healthy 8/8
speedy47_6(10.47.60.6,,9062)   1      probe      Healthy 8/8
speedy47_7(10.47.60.7,,9062)   1      probe      Healthy 8/8
speedy47_8(10.47.60.8,,9062)   1      probe      Healthy 8/8
speedy47_9(10.47.60.9,,9062)   1      probe      Healthy 8/8



Oh No! I only wanted one backend to go sick. Damn.

Let's try Name, IP address and port instead...

varnish> backend.list
200        
Backend name                   Refs   Admin      Probe
speedy(10.47.28.60,,9062)      1      probe      Healthy 5/5
speedy(10.47.28.61,,9062)      1      probe      Sick 0/5
speedy47_1(10.47.60.1,,9062)   1      sick       Healthy 8/8
speedy47_10(10.47.60.10,,9062) 1      sick       Healthy 8/8
speedy47_11(10.47.60.11,,9062) 1      sick       Healthy 8/8
speedy47_12(10.47.60.12,,9062) 1      sick       Healthy 8/8
speedy47_13(10.47.60.13,,9062) 1      sick       Healthy 8/8
speedy47_14(10.47.60.14,,9062) 1      sick       Healthy 8/8
speedy47_15(10.47.60.15,,9062) 1      sick       Healthy 8/8
speedy47_16(10.47.60.16,,9062) 1      sick       Healthy 8/8
speedy47_17(10.47.60.17,,9062) 1      sick       Healthy 8/8
speedy47_18(10.47.60.18,,9062) 1      sick       Healthy 8/8
speedy47_2(10.47.60.2,,9062)   1      probe      Healthy 8/8
speedy47_3(10.47.60.3,,9062)   1      probe      Healthy 8/8
speedy47_4(10.47.60.4,,9062)   1      probe      Healthy 8/8
speedy47_5(10.47.60.5,,9062)   1      probe      Healthy 8/8
speedy47_6(10.47.60.6,,9062)   1      probe      Healthy 8/8
speedy47_7(10.47.60.7,,9062)   1      probe      Healthy 8/8
speedy47_8(10.47.60.8,,9062)   1      probe      Healthy 8/8
speedy47_9(10.47.60.9,,9062)   1      probe      Healthy 8/8
varnish> backend.set_health speedy47_1(10.47.60.1:9062) auto
200        

varnish> backend.list
200        
Backend name                   Refs   Admin      Probe
speedy(10.47.28.60,,9062)      1      probe      Healthy 5/5
speedy(10.47.28.61,,9062)      1      probe      Sick 0/5
speedy47_1(10.47.60.1,,9062)   1      probe      Healthy 8/8
speedy47_10(10.47.60.10,,9062) 1      probe      Healthy 8/8
speedy47_11(10.47.60.11,,9062) 1      probe      Healthy 8/8
speedy47_12(10.47.60.12,,9062) 1      probe      Healthy 8/8
speedy47_13(10.47.60.13,,9062) 1      probe      Healthy 8/8
speedy47_14(10.47.60.14,,9062) 1      probe      Healthy 8/8
speedy47_15(10.47.60.15,,9062) 1      probe      Healthy 8/8
speedy47_16(10.47.60.16,,9062) 1      probe      Healthy 8/8
speedy47_17(10.47.60.17,,9062) 1      probe      Healthy 8/8
speedy47_18(10.47.60.18,,9062) 1      probe      Healthy 8/8
speedy47_2(10.47.60.2,,9062)   1      probe      Healthy 8/8
speedy47_3(10.47.60.3,,9062)   1      probe      Healthy 8/8
speedy47_4(10.47.60.4,,9062)   1      probe      Healthy 8/8
speedy47_5(10.47.60.5,,9062)   1      probe      Healthy 8/8
speedy47_6(10.47.60.6,,9062)   1      probe      Healthy 8/8
speedy47_7(10.47.60.7,,9062)   1      probe      Healthy 8/8
speedy47_8(10.47.60.8,,9062)   1      probe      Healthy 8/8
speedy47_9(10.47.60.9,,9062)   1      probe      Healthy 8/8

Again, matched all. Meeeeep!



hm. Looks like something is wrong down here in the code:

bin/varnishd/cache/cache_backend_cfg.c
367     if (port_b != NULL && strncmp(b->port, port_b, port_l) != 0)
369     if (name_b != NULL && strncmp(b->vcl_name, name_b, name_l) != 0)
373           strncmp(b->ipv4_addr, ip_b, ip_l)) &&
375           strncmp(b->ipv6_addr, ip_b, ip_l)))

should also take the strlen(b->port)... into account.

Example:
367     if (port_b != NULL && strlen(b->port) == port_l &&
strncmp(b->port, port_b, port_l) != 0)



best regards,
Marian




From mhettwer at team.mobile.de  Wed Mar 27 15:38:37 2013
From: mhettwer at team.mobile.de (Hettwer, Marian)
Date: Wed, 27 Mar 2013 15:38:37 +0000
Subject: backend.set_health bug
In-Reply-To: <6DD1CF2A31C9724E9425CDB5511A92FC1C683DC7@DUB-EXDDA-S32.corp.ebay.com>
Message-ID: <6DD1CF2A31C9724E9425CDB5511A92FC1C683EF2@DUB-EXDDA-S32.corp.ebay.com>

Follow up. We're not sure what the code want's to achieve here, but at
least the attached patch let's us use names the way we think it should be
;-)


FWIW,
./Marian

patch by seffenberg at team.mobile.de

On 27.03.13 15:21, "Hettwer, Marian" <mhettwer at team.mobile.de> wrote:

>Hi Ho,
>
>seems like the parser for specifying the backend is buggy.
>Should I file a bug report?
>
>But first the details. Look what's happening.
>
>We have this backend list:
>
>varnish> backend.list
>200        
>Backend name                   Refs   Admin      Probe
>speedy(10.47.28.60,,9062)      1      probe      Healthy 5/5
>speedy(10.47.28.61,,9062)      1      probe      Sick 0/5
>speedy47_1(10.47.60.1,,9062)   1      probe      Healthy 8/8
>speedy47_10(10.47.60.10,,9062) 1      probe      Healthy 8/8
>speedy47_11(10.47.60.11,,9062) 1      probe      Healthy 8/8
>speedy47_12(10.47.60.12,,9062) 1      probe      Healthy 8/8
>speedy47_13(10.47.60.13,,9062) 1      probe      Healthy 8/8
>speedy47_14(10.47.60.14,,9062) 1      probe      Healthy 8/8
>speedy47_15(10.47.60.15,,9062) 1      probe      Healthy 8/8
>speedy47_16(10.47.60.16,,9062) 1      probe      Healthy 8/8
>speedy47_17(10.47.60.17,,9062) 1      probe      Healthy 8/8
>speedy47_18(10.47.60.18,,9062) 1      probe      Healthy 8/8
>speedy47_2(10.47.60.2,,9062)   1      probe      Healthy 8/8
>speedy47_3(10.47.60.3,,9062)   1      probe      Healthy 8/8
>speedy47_4(10.47.60.4,,9062)   1      probe      Healthy 8/8
>speedy47_5(10.47.60.5,,9062)   1      probe      Healthy 8/8
>speedy47_6(10.47.60.6,,9062)   1      probe      Healthy 8/8
>speedy47_7(10.47.60.7,,9062)   1      probe      Healthy 8/8
>speedy47_8(10.47.60.8,,9062)   1      probe      Healthy 8/8
>speedy47_9(10.47.60.9,,9062)   1      probe      Healthy 8/8
>
>And are now setting speedy47_1 to sick:
>
>varnish> backend.set_health speedy47_1(10.47.60.1) sick
>200        
>
>
>Now let's have a look at the backend list:
>
>varnish> backend.list
>200        
>Backend name                   Refs   Admin      Probe
>speedy(10.47.28.60,,9062)      1      probe      Healthy 5/5
>speedy(10.47.28.61,,9062)      1      probe      Sick 0/5
>speedy47_1(10.47.60.1,,9062)   1      sick       Healthy 8/8
>speedy47_10(10.47.60.10,,9062) 1      sick       Healthy 8/8
>speedy47_11(10.47.60.11,,9062) 1      sick       Healthy 8/8
>speedy47_12(10.47.60.12,,9062) 1      sick       Healthy 8/8
>speedy47_13(10.47.60.13,,9062) 1      sick       Healthy 8/8
>speedy47_14(10.47.60.14,,9062) 1      sick       Healthy 8/8
>speedy47_15(10.47.60.15,,9062) 1      sick       Healthy 8/8
>speedy47_16(10.47.60.16,,9062) 1      sick       Healthy 8/8
>speedy47_17(10.47.60.17,,9062) 1      sick       Healthy 8/8
>speedy47_18(10.47.60.18,,9062) 1      sick       Healthy 8/8
>speedy47_2(10.47.60.2,,9062)   1      probe      Healthy 8/8
>speedy47_3(10.47.60.3,,9062)   1      probe      Healthy 8/8
>speedy47_4(10.47.60.4,,9062)   1      probe      Healthy 8/8
>speedy47_5(10.47.60.5,,9062)   1      probe      Healthy 8/8
>speedy47_6(10.47.60.6,,9062)   1      probe      Healthy 8/8
>speedy47_7(10.47.60.7,,9062)   1      probe      Healthy 8/8
>speedy47_8(10.47.60.8,,9062)   1      probe      Healthy 8/8
>speedy47_9(10.47.60.9,,9062)   1      probe      Healthy 8/8
>
>
>
>Oh No! I only wanted one backend to go sick. Damn.
>
>Let's try Name, IP address and port instead...
>
>varnish> backend.list
>200        
>Backend name                   Refs   Admin      Probe
>speedy(10.47.28.60,,9062)      1      probe      Healthy 5/5
>speedy(10.47.28.61,,9062)      1      probe      Sick 0/5
>speedy47_1(10.47.60.1,,9062)   1      sick       Healthy 8/8
>speedy47_10(10.47.60.10,,9062) 1      sick       Healthy 8/8
>speedy47_11(10.47.60.11,,9062) 1      sick       Healthy 8/8
>speedy47_12(10.47.60.12,,9062) 1      sick       Healthy 8/8
>speedy47_13(10.47.60.13,,9062) 1      sick       Healthy 8/8
>speedy47_14(10.47.60.14,,9062) 1      sick       Healthy 8/8
>speedy47_15(10.47.60.15,,9062) 1      sick       Healthy 8/8
>speedy47_16(10.47.60.16,,9062) 1      sick       Healthy 8/8
>speedy47_17(10.47.60.17,,9062) 1      sick       Healthy 8/8
>speedy47_18(10.47.60.18,,9062) 1      sick       Healthy 8/8
>speedy47_2(10.47.60.2,,9062)   1      probe      Healthy 8/8
>speedy47_3(10.47.60.3,,9062)   1      probe      Healthy 8/8
>speedy47_4(10.47.60.4,,9062)   1      probe      Healthy 8/8
>speedy47_5(10.47.60.5,,9062)   1      probe      Healthy 8/8
>speedy47_6(10.47.60.6,,9062)   1      probe      Healthy 8/8
>speedy47_7(10.47.60.7,,9062)   1      probe      Healthy 8/8
>speedy47_8(10.47.60.8,,9062)   1      probe      Healthy 8/8
>speedy47_9(10.47.60.9,,9062)   1      probe      Healthy 8/8
>varnish> backend.set_health speedy47_1(10.47.60.1:9062) auto
>200        
>
>varnish> backend.list
>200        
>Backend name                   Refs   Admin      Probe
>speedy(10.47.28.60,,9062)      1      probe      Healthy 5/5
>speedy(10.47.28.61,,9062)      1      probe      Sick 0/5
>speedy47_1(10.47.60.1,,9062)   1      probe      Healthy 8/8
>speedy47_10(10.47.60.10,,9062) 1      probe      Healthy 8/8
>speedy47_11(10.47.60.11,,9062) 1      probe      Healthy 8/8
>speedy47_12(10.47.60.12,,9062) 1      probe      Healthy 8/8
>speedy47_13(10.47.60.13,,9062) 1      probe      Healthy 8/8
>speedy47_14(10.47.60.14,,9062) 1      probe      Healthy 8/8
>speedy47_15(10.47.60.15,,9062) 1      probe      Healthy 8/8
>speedy47_16(10.47.60.16,,9062) 1      probe      Healthy 8/8
>speedy47_17(10.47.60.17,,9062) 1      probe      Healthy 8/8
>speedy47_18(10.47.60.18,,9062) 1      probe      Healthy 8/8
>speedy47_2(10.47.60.2,,9062)   1      probe      Healthy 8/8
>speedy47_3(10.47.60.3,,9062)   1      probe      Healthy 8/8
>speedy47_4(10.47.60.4,,9062)   1      probe      Healthy 8/8
>speedy47_5(10.47.60.5,,9062)   1      probe      Healthy 8/8
>speedy47_6(10.47.60.6,,9062)   1      probe      Healthy 8/8
>speedy47_7(10.47.60.7,,9062)   1      probe      Healthy 8/8
>speedy47_8(10.47.60.8,,9062)   1      probe      Healthy 8/8
>speedy47_9(10.47.60.9,,9062)   1      probe      Healthy 8/8
>
>Again, matched all. Meeeeep!
>
>
>
>hm. Looks like something is wrong down here in the code:
>
>bin/varnishd/cache/cache_backend_cfg.c
>367     if (port_b != NULL && strncmp(b->port, port_b, port_l) != 0)
>369     if (name_b != NULL && strncmp(b->vcl_name, name_b, name_l) != 0)
>373           strncmp(b->ipv4_addr, ip_b, ip_l)) &&
>375           strncmp(b->ipv6_addr, ip_b, ip_l)))
>
>should also take the strlen(b->port)... into account.
>
>Example:
>367     if (port_b != NULL && strlen(b->port) == port_l &&
>strncmp(b->port, port_b, port_l) != 0)
>
>
>
>best regards,
>Marian
>
>
>_______________________________________________
>varnish-misc mailing list
>varnish-misc at varnish-cache.org
>https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-backend-matcher.patch
Type: application/octet-stream
Size: 1113 bytes
Desc: fix-backend-matcher.patch
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130327/ed1694ef/attachment.obj>

From wxz19861013 at gmail.com  Thu Mar 28 02:44:19 2013
From: wxz19861013 at gmail.com (Xianzhe Wang)
Date: Thu, 28 Mar 2013 10:44:19 +0800
Subject: varnish-3.0.2-streaming crashed when Condition((w->vfp->end(sp)) ==
 0) not true.
Message-ID: <CAAh4gyjdjg7tZPtxTauhJw9KxWNuEbcjgX35k=JavpzHCtf3Qw@mail.gmail.com>

I use varnish-3.0.2-streaming for my application.
I found the child thread crashed sometimes.And this is my panic.show log.
Are there somebody  met this issue before?
Any advice will be appreciated, Thank you for help.

Last panic at: Thu, 21 Mar 2013 12:05:12 GMT
Missing errorhandling code in FetchBody(), cache_fetch.c line 517:
  Condition((w->vfp->end(sp)) == 0) not true.thread = (cache-worker)
ident =
Linux,2.6.32.45-0.3.2.3813.0.PTF-default,x86_64,-smalloc,-smalloc,-hcritbit,epoll
Backtrace:
  0x42ffc3: pan_ic+d3
  0x424482: FetchBody+532
  0x418d17: cnt_streambody+287
  0x4199ed: CNT_Session+4fd
  0x4327d3: wrk_do_cnt_sess+93
  0x431a1a: wrk_thread_real+3ea
  0x7f3c9155f5f0: _end+7f3c90ee32d8
  0x7f3c912ce84d: _end+7f3c90c52535
sp = 0x7f3c1ef59008 {
  fd = 13, id = 13, xid = 0,
  client = 192.168.5.143 15380,
  step = STP_WAIT,
  handling = deliver,
  err_code = 200, err_reason = (null),
  restarts = 0, esi_level = 0
  flags =
  bodystatus = 4
  ws = 0x7f3c1ef59080 {
    id = "sess",
    {s,f,r,e} = {0x7f3c1ef5b628,+24,+32792,+65536},
  },
  http[req] = {
    ws = 0x7f3c1ef59080[sess]
      "",
      "http://img02.taobaocdn.com/tps/i2/wT1brifXk0AXXcMLR6o-81-18.png",
      "HTTP/1.1",
      "Host: img02.taobaocdn.com",
      "Proxy-Connection: keep-alive",
      "User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.11 (KHTML,
like Gecko) Chrome/23.0.1271.95 Safari/537.11",
      "Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
      "Accept-Language: zh-CN,zh;q=0.8",
      "Accept-Charset: GBK,utf-8;q=0.7,*;q=0.3",
      "X-Image-Compress-Level: 2",
      "If-None-Match: W/"PSA-nT-Js4Pc7j"",
      "If-Modified-Since: Tue, 08 May 2012 05:26:11 GMT",
      "X-Forwarded-For: 192.168.5.143",
  },
  worker = 0x7f3bbec40a90 {
    ws = 0x7f3bbec40ce0 {
      id = "wrk",
      {s,f,r,e} = {0x7f3bbec2ea20,0x7f3bbec2ea20,(nil),+65536},
    },
    },
},


Regards,
Shawn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130328/ac64a444/attachment.html>

From eliezer at ngtech.co.il  Thu Mar 28 05:31:31 2013
From: eliezer at ngtech.co.il (Eliezer Croitoru)
Date: Thu, 28 Mar 2013 07:31:31 +0200
Subject: Caching MS updates using varnish.
Message-ID: <5153D5B3.5040100@ngtech.co.il>

I want to cache MS updates using varnish.

The setup I was thinking about is to catch couple domains in varnish and 
pass it to one backhand proxy server.
The main problem is how to do a cache all domains and pass it to one 
backhand??

Eliezer



From kelvin1111111 at gmail.com  Thu Mar 28 07:07:57 2013
From: kelvin1111111 at gmail.com (Kelvin Loke)
Date: Thu, 28 Mar 2013 15:07:57 +0800
Subject: Why "pass" object goes to vcl_hash?
Message-ID: <CAAct57wgvHbpZ2SGtkmxy7CxedDeKG+qCvW2=3U1A=zNoOim+A@mail.gmail.com>

This was bothering me for quite some time. I looked through the Varnish
Book detailed request flow diagram, an object that is "pass" in vcl_recv()
will go straight to vcl_pass() -> vcl_fetch() -> vcl_deliver().

But the varnishlog below tells me otherwise, why an object that is "pass"
in vcl_recv still needs to go to vcl_hash()?

   34 ReqStart     c 1.1.1.1 39816 1065872559
   34 RxRequest    c POST
   34 RxURL        c /marina/s/CheckSession
   34 RxProtocol   c HTTP/1.1
   34 RxHeader     c Host: localhost:8082
   34 RxHeader     c Accept: text/plain, */*; q=0.01
   34 RxHeader     c Origin: http://localhost:8082
   34 RxHeader     c X-Requested-With: XMLHttpRequest
   34 RxHeader     c User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22
   34 RxHeader     c Referer: http://localhost:8082/member/w.aspx
   34 RxHeader     c Accept-Encoding: gzip,deflate,sdch
   34 RxHeader     c Accept-Language: en-US,en;q=0.8
   34 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
   34 RxHeader     c Cookie: JSESSIONID=5651fba1110cfdd2edf035494eec;
sl=-1237168048
   34 RxHeader     c Connection: close
   34 RxHeader     c Content-Length: 0
   34 VCL_call     c recv
   34 VCL_return   c pass
   34 VCL_call     c hash
   34 Hash         c /marina/s/CheckSession
   34 Hash         c
   34 Hash         c
   34 VCL_return   c hash
   34 VCL_call     c pass pass
   34 Backend      c 26 member_micro member_micro
   34 TTL          c 1065872559 RFC 0 -1 -1 1364453898 0 1364453897 0 0
   34 VCL_call     c fetch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130328/f376ac10/attachment.html>

From perbu at varnish-software.com  Thu Mar 28 09:11:01 2013
From: perbu at varnish-software.com (Per Buer)
Date: Thu, 28 Mar 2013 10:11:01 +0100
Subject: Geolocation without compiling Varnish from source
In-Reply-To: <f877a3bc6f95b5bc9422e1e3c97ec58e.squirrel@sm.webmail.pair.com>
References: <f877a3bc6f95b5bc9422e1e3c97ec58e.squirrel@sm.webmail.pair.com>
Message-ID: <CAOXZevBtrSd3ioo7=JWt=9gUGO1_2cDQgqisdFnE2PQofxv8fg@mail.gmail.com>

Hi John,

On Tue, Mar 26, 2013 at 7:00 PM, John <spam at onlineconcepts.com> wrote:

> I would like to geolocate our website visitors by country. Varnish can
> send the ISO country code to our backend, which will then serve the users
> content specific to their region of the world.
>

Right. What you want to take a look at is one of the GeoIP vmods that are
out there (there are at least two). A vmod is a module extending Varnish in
some way.

Inline C isn't pretty or easy to maintain and would recommend avoiding it
if you can.


-- 
 <http://www.varnish-software.com/> *Per Buer*
CEO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130328/a8ed3d81/attachment.html>

From spam at onlineconcepts.com  Thu Mar 28 19:44:29 2013
From: spam at onlineconcepts.com (John)
Date: Thu, 28 Mar 2013 15:44:29 -0400
Subject: Geolocation without compiling Varnish from source
In-Reply-To: <f877a3bc6f95b5bc9422e1e3c97ec58e.squirrel@sm.webmail.pair.com>
References: <f877a3bc6f95b5bc9422e1e3c97ec58e.squirrel@sm.webmail.pair.com>
Message-ID: <c3b8f693ff333a48d5a9c0b75eb72fa6.squirrel@sm.webmail.pair.com>

I received this advice off-list.

> You need the varnish sources to compile the vmod (like
> https://github.com/leed25d/geoip-vmod), but you can use it from your
> packaged varnish version.

That convinced me that it was possible to do what I wanted. Now I have
geolocation up and running in Varnish!

I hit a number of surprising snags, chiefly around version issues. I
followed this process (mostly based on
http://www.mursalat.net/ahmed-mursalat/varnish-3-geoip-mod-in-debian-squeeze-and-redirects.html)

* Download Varnish source code
* Download the VMOD source code
* Compile VMOD against the Varnish source
* Copy VMOD to Varnish's vmod directory on Test server

On my system running RHEL 6.2, the location was:
/usr/lib64/varnish/vmods/libvmod_geoip.so

VERSION ISSUES
The VMOD was *very* particular about which version of the Varnish source
you used.

git clone https://github.com/varnish/Varnish-Cache.git
cd Varnish-Cache

On my system, running "git checkout varnish-3.0.2" was not specific
enough. I ran "git log" to see the recent commit history. As of late March
2013, the current version was Varnish 3.0.2 670f56f, where 670f56f refers
to the most recent Git commit. I ran this on my server:

varnishd -V

It reported "varnishd (varnish-3.0.2 revision 55e70a4)." So the current
source was one commit ahead of the binary version of Varnish binaries that
were already deployed. I saw what was happening, but on a lark, I deployed
in Dev a version of the VMOD that matched commit 670f56f. That produced
this error when starting Varnish: "ABI mismatch, expected <Varnish 3.0.2
55e70a4>, got <Varnish 3.0.2 670f56f>"

THE FIX
Start over again, and instead of running "git checkout varnish-3.0.2" run:

git checkout 55e70a4

You have to check out the version at the correct commit because you'll get
an error if you compile your VMOD against the wrong code version.

STABILITY CONCERNS
Thankfully, all my relevant servers run the same version of Varnish. I'm
worried what will happen the next time a server admin runs:

"yum update-minimal --security"

I'm sure that could overwrite the Varnish binaries. Then my VMOD would be
out of sync with the binaries, and my website would go offline until I
either:

* Recompiled the VMOD; or
* Commented out the VCL code that includes the VMOD.

This seems a little fragile to me, but maybe I have been working too long
with scripting languages and APIs.

DOCUMENTATION
If the group considers it useful, I can document this further for the
Varnish community. Would this go in the wiki?

Thanks,
John



From james at ifixit.com  Thu Mar 28 21:06:47 2013
From: james at ifixit.com (James Pearson)
Date: Thu, 28 Mar 2013 14:06:47 -0700
Subject: Caching MS updates using varnish.
In-Reply-To: <5153D5B3.5040100@ngtech.co.il>
References: <5153D5B3.5040100@ngtech.co.il>
Message-ID: <1364504704-sup-5990@geror.local>

Excerpts from Eliezer Croitoru's message of 2013-03-27 22:31:31 -0700:
> I want to cache MS updates using varnish.

Varnish is designed from the start as a reverse-proxy (well, http accelerator,
I guess).  The kind of thing you're describing is much more suited to tools
developed to solve that problem, e.g. Squid.

 - P



From sean at monkeysnatchbanana.com  Fri Mar 29 18:45:56 2013
From: sean at monkeysnatchbanana.com (Sean Allen)
Date: Fri, 29 Mar 2013 14:45:56 -0400
Subject: number of thread pools
Message-ID: <CABm-WmczaunbbUrA-RQuRm+5N4LZQWq4oNz9gonP1v1+L-sO0w@mail.gmail.com>

Why is it advised for Varnish 3 to only run with 2 thread pools rather than
a thread pool per cpu core?


-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130329/c4cee788/attachment.html>

From ross at trademe.co.nz  Sun Mar 31 02:31:53 2013
From: ross at trademe.co.nz (Ross Brown)
Date: Sun, 31 Mar 2013 02:31:53 +0000
Subject: number of thread pools
In-Reply-To: <CABm-WmczaunbbUrA-RQuRm+5N4LZQWq4oNz9gonP1v1+L-sO0w@mail.gmail.com>
References: <CABm-WmczaunbbUrA-RQuRm+5N4LZQWq4oNz9gonP1v1+L-sO0w@mail.gmail.com>
Message-ID: <6ED006BA952A6E4F802D61A21A5D4F054CEAC9@TMMAIL1.trademe.local>

> Why is it advised for Varnish 3 to only run with 2 thread pools rather than a thread pool per cpu core?

The thread model allows you to use multiple thread pools, but time and experience has shown that as long as you have 2 thread pools, adding more will not increase performance.

The most important thread setting is the number of worker threads.

Note If you run across tuning advice that suggests running one thread pool for each CPU core, res assured that this is old advice. Experiments and data from production environments have revealed that as long as you have two thread pools (which is the default), there is nothing to gain by increasing the number of thread pools.

From https://www.varnish-software.com/static/book/Tuning.html#threading-model 

From mail at caseler.com  Mon Mar  4 18:52:19 2013
From: mail at caseler.com (Michael Finke)
Date: Mon, 04 Mar 2013 18:52:19 -0000
Subject: 503 Error
Message-ID: <BE7B61FB262141DDBDD24884464D1D1A@DESKTOPPC>

Dear Varnish Community,

I get an 503 error 
Service Unavailable

Guru Meditation:
XID: 1059580559


--------------------------------------------------------------------------------

Varnish cache server



everytime i want to import products in my magentosystem, and even if i delete products.

Can you tell me how to deal with this problem? Is it possible to deactivate the varnish cache server?



greetings 



Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130304/c214a10f/attachment.html>

From lnsano at bol.com.br  Wed Mar  6 21:23:34 2013
From: lnsano at bol.com.br (lnsano at bol.com.br)
Date: Wed, 06 Mar 2013 21:23:34 -0000
Subject: varnish 3 - thread_pools
Message-ID: <5137b3ca67d00_71b77e144784b6@a4-winter14.tmail>

An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130306/3a1b01a0/attachment.html>