From de.techno at gmail.com Sun May 1 04:03:46 2016 From: de.techno at gmail.com (dE) Date: Sun, 1 May 2016 09:33:46 +0530 Subject: Minimalistic varnish does not compile. In-Reply-To: <20160427164425.21456a6b@gdr-desktop.gdr.name> References: <5719DD20.1050805@gmail.com> <27407.1461313243@critter.freebsd.dk> <5719F3C7.207@gmail.com> <20160427164425.21456a6b@gdr-desktop.gdr.name> Message-ID: <57258022.1020707@gmail.com> On 04/27/2016 08:14 PM, Andrzej Godziuk wrote: >> gcc 5.3.0 > Is this the same gcc you've built Varnish with? No. Trying that too. Are there any specific GCC build time options which varnish requires? From de.techno at gmail.com Sun May 1 05:17:15 2016 From: de.techno at gmail.com (dE) Date: Sun, 1 May 2016 10:47:15 +0530 Subject: Minimalistic varnish does not compile. In-Reply-To: References: <5719DD20.1050805@gmail.com> Message-ID: <5725915B.9060508@gmail.com> It's 4.0.3 On 04/22/2016 02:04 PM, Glenn Slaven wrote: > What version of varnish are you running? > > It compiles fine on varnish 4.1.1: > http://www.vclfiddle.net/160422-1ef61d5/0 > > Glenn > > On 22 April 2016 at 18:13, dE > wrote: > > Hello, > I got this minimal VCL -- > > |vcl 4.0;|| > ||backend neuro {|| > ||.host = "129.170.30.193";|| > ||.port = "http";|| > ||}|| > || > ||sub vcl_recv {|| > ||set req.backend_hint = neuro;|| > ||}| > > Which fails to compile for no reason. > > |Running VCC-compiler failed, exited with 2|| > || > ||VCL compilation failed| > > Can anyone help? > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From de.techno at gmail.com Sun May 1 05:19:02 2016 From: de.techno at gmail.com (dE) Date: Sun, 1 May 2016 10:49:02 +0530 Subject: Minimalistic varnish does not compile. In-Reply-To: <57258022.1020707@gmail.com> References: <5719DD20.1050805@gmail.com> <27407.1461313243@critter.freebsd.dk> <5719F3C7.207@gmail.com> <20160427164425.21456a6b@gdr-desktop.gdr.name> <57258022.1020707@gmail.com> Message-ID: <572591C6.2080609@gmail.com> On 05/01/2016 09:33 AM, dE wrote: > On 04/27/2016 08:14 PM, Andrzej Godziuk wrote: >>> gcc 5.3.0 >> Is this the same gcc you've built Varnish with? > > No. > > Trying that too. > > Are there any specific GCC build time options which varnish requires? No, it does not help. I dont think this's a GCC problem. First the code must be converted to C source, it fails during that. There is no C code produced. From rgelfand2 at gmail.com Sun May 1 11:25:49 2016 From: rgelfand2 at gmail.com (Roman Gelfand) Date: Sun, 01 May 2016 11:25:49 +0000 Subject: Accelerating wordpress site In-Reply-To: <5722129E.8090004@lamp-solutions.de> References: <5722129E.8090004@lamp-solutions.de> Message-ID: Thanks a lot. Stupid me, I never thought it could be link not found. Are you saying that even after I remove link not found I would see a tremendous improvement in how fast the page is brought up? Could you elaborate on the cookie part. Keep in mind, I am using wordpress cms. Would you have any info on how to do this? On Thu, Apr 28, 2016 at 10:17 AM Tobias Eichelbr?nner < tobias.eichelbroenner at lamp-solutions.de> wrote: > Hi, > > > > My wordpress site www.pharmscript.com is > > slow to load full height full width page/image. Interestingly enough, > > the load is only slow when a full menu is showing. > > first of all, my browser tries to fetch some content from a private IP > when connection to you page. > > http://192.168.100.122/wordpress/wp-content/uploads/2016/01/menuDivbar.png > > This causes the browser to wait for a timeout before showing the page > fully loaded. > > The main page is rendered in about 1 second. Varnish can speed this up > dramatically, but you are setting a session cookie. You will have to get > rid of this first. > > Sincerely, > > Tobias > > > -- > LAMP solutions GmbH > Gostenhofer Hauptstrasse 35 > 90443 Nuernberg > > Amtsgericht Nuernberg: HRB 22366 > Geschaeftsfuehrer: Heiko Schubert > > Es gelten unsere allgemeinen Geschaeftsbedingungen. > http://www.lamp-solutions.de/agb/ > > Telefon : 0911 / 376 516 0 > Fax : 0911 / 376 516 11 > E-Mail : support at lamp-solutions.de > Web : www.lamp-solutions.de > Facebook : http://www.facebook.com/LAMPsolutions > Twitter : http://twitter.com/#!/lampsolutions > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Mon May 2 05:00:11 2016 From: subharaj.manna at gmail.com (Debraj Manna) Date: Mon, 2 May 2016 10:30:11 +0530 Subject: Collecting varnish stats at the Url Level In-Reply-To: References: Message-ID: Thanks Thomas. How can I maintain a counter per url. In vcl_deliver obj.hits will again give me the overall hits. On Sat, Apr 30, 2016 at 4:01 PM, Thomas Lecomte < thomas.lecomte at virtual-expo.com> wrote: > On Sat, Apr 30, 2016 at 12:19 PM, Debraj Manna > wrote: > > Right now we were able to get varnish stats like cache_hit, cache_miss > for > > the overall varnish hits. Is there a way we can get the stats at the url > > level like cache_hit, cache_miss, etc for /discovery/v1/search, > > /discovery/v1/suggest separately? > > Hi, > > Varnish can't do that out of the box. You will have to use std.log() > to log wether the request generated a cache miss or a cache hit, and > then compute the ratio by parsing the output of varnishlog/varnishncsa > (see %{VCL_log:key}x in the varnishncsa manpage). > > -- > Thomas Lecomte / Virtual Expo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Mon May 2 05:33:50 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Mon, 2 May 2016 07:33:50 +0200 Subject: Minimalistic varnish does not compile. In-Reply-To: <572591C6.2080609@gmail.com> References: <5719DD20.1050805@gmail.com> <27407.1461313243@critter.freebsd.dk> <5719F3C7.207@gmail.com> <20160427164425.21456a6b@gdr-desktop.gdr.name> <57258022.1020707@gmail.com> <572591C6.2080609@gmail.com> Message-ID: On Sun, May 1, 2016 at 7:19 AM, dE wrote: > On 05/01/2016 09:33 AM, dE wrote: >> >> On 04/27/2016 08:14 PM, Andrzej Godziuk wrote: >>>> >>>> gcc 5.3.0 >>> >>> Is this the same gcc you've built Varnish with? >> >> >> No. >> >> Trying that too. >> >> Are there any specific GCC build time options which varnish requires? No that I'm aware of, I'm running GCC 5.3.1 on this machine. Can you please run this command and attach the ouput? varnishd -n $(mktemp -d) -C -f /etc/varnish/default.vcl 2>/tmp/output.c Thanks, Dridi From thomas.lecomte at virtual-expo.com Mon May 2 05:52:11 2016 From: thomas.lecomte at virtual-expo.com (Thomas Lecomte) Date: Mon, 2 May 2016 07:52:11 +0200 Subject: Collecting varnish stats at the Url Level In-Reply-To: References: Message-ID: On Mon, May 2, 2016 at 7:00 AM, Debraj Manna wrote: > Thanks Thomas. > > How can I maintain a counter per url. In vcl_deliver obj.hits will again > give me the overall hits. As I said, you have to implement this yourself using VCL and varnishncsa. Example of VCL: # 38. match rule for PDF pages if (req.url ~ "^\/pdf\/([^\/]+)\-(\d+)\.html$") { set req.http.X-MatchRule = "38"; return (hash); } [...] std.log("MR:" + req.http.X-MatchRule); Then, call varnishncsa with -F and %{VCL_Log:MR} plus %{Varnish:handling} to log the match rule and the handling of the request (miss, hit, pipe, pass, whatever). You will have to parse this output yourself to compute a ratio. -- Thomas Lecomte / Virtual Expo From g.schoenberger at xortex.com Mon May 2 07:14:41 2016 From: g.schoenberger at xortex.com (Georg Schoenberger) Date: Mon, 2 May 2016 07:14:41 +0000 Subject: Varnish 4.0 Ubuntu 16.04 Xenial Message-ID: <5726FE61.9040400@xortex.com> Hi folks! Will there be a Varnish repo for Ubuntu 16.04 Xenial soon? I would like to use Varnish 4.0 with Ubuntu 16.04, but the Ubuntu repo only provides 4.1 in the universe section. I would prefer 4.0 from varnish-cache.org, but currently I can only find the Trusty repo URL: * https://www.varnish-cache.org/releases/install_ubuntu.html#install-ubuntu THX, Georg -- =================================== Georg Sch?nberger - XorteX eBusiness GmbH Network und Hosting Veldner Str. 29 | A-4120 Neufelden Tel: +43 7282 20797 4450 | Fax: +43 7282 20797 4490 FB-Nr: 272690s FB-Gericht: Landesgericht Linz Beh?rde gem. ECG: BH Rohrbach Mail: g.schoenberger at xortex.com Web: http://www.xortex.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Mon May 2 08:28:09 2016 From: perbu at varnish-software.com (Per Buer) Date: Mon, 2 May 2016 10:28:09 +0200 Subject: Varnish 4.0 Ubuntu 16.04 Xenial In-Reply-To: <5726FE61.9040400@xortex.com> References: <5726FE61.9040400@xortex.com> Message-ID: On Mon, May 2, 2016 at 9:14 AM, Georg Schoenberger < g.schoenberger at xortex.com> wrote: > Hi folks! > > Will there be a Varnish repo for Ubuntu 16.04 Xenial soon? > I would like to use Varnish 4.0 with Ubuntu 16.04, but the Ubuntu repo > only provides 4.1 in the universe section. > I would prefer 4.0 from varnish-cache.org, but currently I can only find > the Trusty repo URL: > * > https://www.varnish-cache.org/releases/install_ubuntu.html#install-ubuntu > > THX, Georg > The Trusty (14.04) packages should work fine on Xenial. AFAIK there are no plans to provide native packages for Xenial unless there is a compelling reason to do so. Could you try the Trusty packages and report any problems here, please? Cheers, Per. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.schoenberger at xortex.com Tue May 3 06:00:58 2016 From: g.schoenberger at xortex.com (Georg Schoenberger) Date: Tue, 3 May 2016 06:00:58 +0000 Subject: Varnish 4.0 Ubuntu 16.04 Xenial In-Reply-To: References: <5726FE61.9040400@xortex.com> Message-ID: <57283E9A.7060403@xortex.com> On 2016-05-02 10:28, Per Buer wrote: On Mon, May 2, 2016 at 9:14 AM, Georg Schoenberger > wrote: Hi folks! Will there be a Varnish repo for Ubuntu 16.04 Xenial soon? I would like to use Varnish 4.0 with Ubuntu 16.04, but the Ubuntu repo only provides 4.1 in the universe section. I would prefer 4.0 from varnish-cache.org, but currently I can only find the Trusty repo URL: * https://www.varnish-cache.org/releases/install_ubuntu.html#install-ubuntu THX, Georg The Trusty (14.04) packages should work fine on Xenial. AFAIK there are no plans to provide native packages for Xenial unless there is a compelling reason to do so. Could you try the Trusty packages and report any problems here, please? Cheers, Per. THX for the answer, what I did was: * Install varnish 4.1 from Ubuntu repos to get systemd files in /lib/systemd * Removed varnish 4.1 * Install varnish 4.0 from varnish-cache (trusty packages installed fine) * Adjust startup values in /lib/systemd varnish service files as systemd does not honor /etc/default/varnish like /etc/init.d/varnish did * Start varnish via systemd Just in case anyone needs to run varnish with systemd and is puzzled why /etc/default/varnish does not take into account. All the best, Georg -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkarsten at varnish-software.com Tue May 3 15:08:46 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 3 May 2016 17:08:46 +0200 Subject: Varnish modules package released (Was: Re: Update on vmods and packaging) In-Reply-To: <20160304133039.GA32277@immer.varnish-software.com> References: <20151202144041.GB19950@immer.varnish-software.com> <20160304133039.GA32277@immer.varnish-software.com> Message-ID: <20160503150844.GA23674@immer.varnish-software.com> On Fri, Mar 04, 2016 at 02:30:40PM +0100, Lasse Karstensen wrote: > [crosspost to both -dev and -misc] > On Wed, Dec 02, 2015 at 03:40:42PM +0100, Lasse Karstensen wrote: [cut] > > Further down the line it is likely that we will be combining the simple > > vmods (with no third party/library dependencies) into a single > > collection source package. Updates on that will come later. > We have now made an initial release of this collection. > Tarball distribution is here: > http://files.varnish-software.com/vmod/varnish-modules-0.9.0.tar.gz For the record, tarball distribution of varnish-modules has been moved to a new and more permanent home: https://download.varnish-software.com/varnish-modules/ We'll be doing a 0.9.1 release of it soon. [cut] > 3.0 and 4.0 versions of the old vmods will be archived and made > available in a similar manner. When that has been done, the old github > projects for the vmods will be retired. The mod_index site on https://download.varnish-software.com/ also contain archived versions of assorted other vmods for 3.0 and 4.0, if anyone needs those. More will come as time permits. Git branches will be removed over time. -- Lasse Karstensen Varnish Software AS From subharaj.manna at gmail.com Tue May 3 17:29:43 2016 From: subharaj.manna at gmail.com (Debraj Manna) Date: Tue, 3 May 2016 22:59:43 +0530 Subject: Collecting varnish stats at the Url Level In-Reply-To: References: Message-ID: Thanks Thomas. On May 2, 2016 11:22 AM, "Thomas Lecomte" wrote: > On Mon, May 2, 2016 at 7:00 AM, Debraj Manna > wrote: > > Thanks Thomas. > > > > How can I maintain a counter per url. In vcl_deliver obj.hits will again > > give me the overall hits. > > As I said, you have to implement this yourself using VCL and varnishncsa. > > Example of VCL: > > # 38. match rule for PDF pages > if (req.url ~ "^\/pdf\/([^\/]+)\-(\d+)\.html$") > { > set req.http.X-MatchRule = "38"; > return (hash); > } > [...] > std.log("MR:" + req.http.X-MatchRule); > > Then, call varnishncsa with -F and %{VCL_Log:MR} plus > %{Varnish:handling} to log the match rule and the handling of the > request (miss, hit, pipe, pass, whatever). You will have to parse this > output yourself to compute a ratio. > > -- > Thomas Lecomte / Virtual Expo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From de.techno at gmail.com Sat May 7 09:21:06 2016 From: de.techno at gmail.com (dE) Date: Sat, 7 May 2016 14:51:06 +0530 Subject: Minimalistic varnish does not compile. In-Reply-To: References: <5719DD20.1050805@gmail.com> <27407.1461313243@critter.freebsd.dk> <5719F3C7.207@gmail.com> <20160427164425.21456a6b@gdr-desktop.gdr.name> <57258022.1020707@gmail.com> <572591C6.2080609@gmail.com> Message-ID: <572DB382.8060302@gmail.com> It says |Running VCC-compiler failed, exited with 2|| || ||VCL compilation failed| On 05/02/16 11:03, Dridi Boukelmoune wrote: > On Sun, May 1, 2016 at 7:19 AM, dE wrote: >> On 05/01/2016 09:33 AM, dE wrote: >>> On 04/27/2016 08:14 PM, Andrzej Godziuk wrote: >>>>> gcc 5.3.0 >>>> Is this the same gcc you've built Varnish with? >>> >>> No. >>> >>> Trying that too. >>> >>> Are there any specific GCC build time options which varnish requires? > No that I'm aware of, I'm running GCC 5.3.1 on this machine. > > Can you please run this command and attach the ouput? > > varnishd -n $(mktemp -d) -C -f /etc/varnish/default.vcl 2>/tmp/output.c > > Thanks, > Dridi -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Mon May 9 07:49:15 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Mon, 9 May 2016 09:49:15 +0200 Subject: Minimalistic varnish does not compile. In-Reply-To: <572DB382.8060302@gmail.com> References: <5719DD20.1050805@gmail.com> <27407.1461313243@critter.freebsd.dk> <5719F3C7.207@gmail.com> <20160427164425.21456a6b@gdr-desktop.gdr.name> <57258022.1020707@gmail.com> <572591C6.2080609@gmail.com> <572DB382.8060302@gmail.com> Message-ID: On Sat, May 7, 2016 at 11:21 AM, dE wrote: > It says > > Running VCC-compiler failed, exited with 2 > > VCL compilation failed Hello, Is it the full contents of output.c? We may have to resort to strace to understand this one. Dridi From de.techno at gmail.com Mon May 9 08:08:18 2016 From: de.techno at gmail.com (dE) Date: Mon, 9 May 2016 13:38:18 +0530 Subject: Minimalistic varnish does not compile. In-Reply-To: References: <5719DD20.1050805@gmail.com> <27407.1461313243@critter.freebsd.dk> <5719F3C7.207@gmail.com> <20160427164425.21456a6b@gdr-desktop.gdr.name> <57258022.1020707@gmail.com> <572591C6.2080609@gmail.com> <572DB382.8060302@gmail.com> Message-ID: <57304572.8040909@gmail.com> Yes. On 05/09/16 13:19, Dridi Boukelmoune wrote: > On Sat, May 7, 2016 at 11:21 AM, dE wrote: >> It says >> >> Running VCC-compiler failed, exited with 2 >> >> VCL compilation failed > Hello, > > Is it the full contents of output.c? > > We may have to resort to strace to understand this one. > > Dridi From S.Garau at liguriadigitale.it Mon May 9 08:38:25 2016 From: S.Garau at liguriadigitale.it (Garau Simone) Date: Mon, 9 May 2016 10:38:25 +0200 Subject: [NEED HELP] Varnish stops waiting backend after 1 second Message-ID: this problem is gonna make me crazy: my varnish istance stops waiting for a backend response after exactly 1 second. Every first call to a page is a 503 Backend Daemon is configured this way: DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -p thread_pool_add_delay=2 \ -p thread_pools=4 \ -p thread_pool_min=200 \ -p thread_pool_max=4000 \ -p timeout_linger=50 \ -p connect_timeout=300 \ -p first_byte_timeout=300 \ -p between_bytes_timeout=300 \ -p send_timeout=900 \ -s malloc,3G? and the VCL backend: backend default { # Define one backend .host = "127.0.0.1"; # IP or Hostname of backend .port = "8080"; # Port Apache or whatever is listening .probe = { .url = "/"; .timeout = 1s; .interval = 1s; .window = 10; .threshold = 8; } .first_byte_timeout = 60s; # How long to wait before we receive a first byte from our backend? .connect_timeout = 60s; # How long to wait for a backend connection? .between_bytes_timeout = 60s; # How long to wait between bytes received from our backend? } Here is the one call in the log: * << Request >> 3440734 - Begin req 3440733 rxreq - Timestamp Start: 1462781837.623325 0.000000 0.000000 - Timestamp Req: 1462781837.623325 0.000000 0.000000 - ReqStart 10.20.129.118 58572 - ReqMethod GET - ReqURL xxxxx.html - ReqProtocol HTTP/1.1 - ReqHeader Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, image/pjpeg, application/x-shockwave-flash, */* - ReqHeader Referer: http://xxxxxx.html - ReqHeader Accept-Language: it-IT - ReqHeader User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E) - ReqHeader Accept-Encoding: gzip, deflate - ReqHeader Host: xxxxxx - ReqHeader DNT: 1 - ReqHeader Connection: Keep-Alive - ReqHeader Cookie: fc_uid=p; __utma=127650066.830977012.1423064118.1426582505.1426588086.20; _ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; ZNPCQ003-38303300=71a0f671; ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; _ga=GA1.2.830977012.142306411 - ReqHeader X-Forwarded-For: 10.20.129.118 - VCL_call RECV - ReqUnset Host: xxx - ReqHeader Host: xxx - ReqURL /xxxx.html - ReqUnset Cookie: fc_uid=p; __utma=127650066.830977012.1423064118.1426582505.1426588086.20; _ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; ZNPCQ003-38303300=71a0f671; ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; _ga=GA1.2.830977012.142306411 - ReqHeader Cookie: fc_uid=p; __utma=127650066.830977012.1423064118.1426582505.1426588086.20; _ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; ZNPCQ003-38303300=71a0f671; ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; _ga=GA1.2.830977012.142306411 - ReqUnset Cookie: fc_uid=p; __utma=127650066.830977012.1423064118.1426582505.1426588086.20; _ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; ZNPCQ003-38303300=71a0f671; ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; _ga=GA1.2.830977012.142306411 - ReqHeader Cookie: fc_uid=p; __utma=127650066.830977012.1423064118.1426582505.1426588086.20; _ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; ZNPCQ003-38303300=71a0f671; ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; _ga=GA1.2.830977012.142306411 - ReqHeader Surrogate-Capability: key=ESI/1.0 - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate - ReqHeader Accept-Encoding: gzip - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 3440735 fetch - Timestamp Fetch: 1462781838.492085 0.868760 0.868760 - Timestamp Process: 1462781838.492101 0.868776 0.000016 - RespHeader Date: Mon, 09 May 2016 08:17:18 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 3440734 - RespProtocol HTTP/1.1 - RespStatus 503 - RespReason Service Unavailable - RespReason Service Unavailable - VCL_call SYNTH - VCL_return deliver - RespHeader Content-Length: 0 - Storage malloc Transient - Debug "RES_MODE 2" - RespHeader Connection: keep-alive - Timestamp Resp: 1462781838.492145 0.868820 0.000044 - ReqAcct 985 0 985 153 0 153 - End Any precious help? Suggestion? From guillaume at varnish-software.com Mon May 9 09:14:55 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 9 May 2016 11:14:55 +0200 Subject: [NEED HELP] Varnish stops waiting backend after 1 second In-Reply-To: References: Message-ID: Hello, Would you mind showing us the corresponding BeReq? (bereq 3440735) -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From lcruzero at gmail.com Mon May 9 14:51:45 2016 From: lcruzero at gmail.com (L Cruzero) Date: Mon, 9 May 2016 10:51:45 -0400 Subject: varnish backend dns ttl Message-ID: In varnish 4.1 will initial DNS resolution during varnishD daemon start up adhere to a defined backend A record dns ttl, or does one in case of an IP change to that A record would still need to restart daemon to get the new IP of the defined backend A record ? -LC -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon May 9 15:21:57 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 9 May 2016 17:21:57 +0200 Subject: varnish backend dns ttl In-Reply-To: References: Message-ID: Hello there, DNS resolution is done once and for all at VCL load, so you can just reload instead of restart. IIUC, varnish will fail loading the VCL if more than one IP is found though. I would advise you to look ad vmod-named ( https://github.com/Dridi/libvmod-named) and vmod-gwist ( https://github.com/gquintard/libvmod-gwist) that could do what you need. -- Guillaume Quintard On Mon, May 9, 2016 at 4:51 PM, L Cruzero wrote: > In varnish 4.1 will initial DNS resolution during varnishD daemon start up > adhere to a defined backend A record dns ttl, or does one in case of an IP > change to that A record would still need to restart daemon to get the new > IP of the defined backend A record ? > > > > > -LC > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Nishant.Trivedi at dictionary.com Mon May 9 18:26:38 2016 From: Nishant.Trivedi at dictionary.com (Nishant Trivedi) Date: Mon, 9 May 2016 18:26:38 +0000 Subject: Problems with Varnish-LUA integration. Message-ID: Hello, I am trying to use Varnish in one of my projects and I need to parse some JSON and make some decisions as part of the design. Since Varnish supports LUA I decided to do the parsing and decision making in LUA. I am using libvmod-lua to invoke LUA scripts from Varnish. I setup a container with Apache2, Varnish 4.1, LUA 5.1.5, and LUARocks 2.3.0 along with all the dependencies. I then installed libvmod-lua in the container. I changed the default.vcl file to look like this: # Marker to tell the VCL compiler that this VCL has been adapted to the # new 4.0 format. vcl 4.0; import lua; import std; # Default backend definition. Set this to point to your content server. backend default { .host = "127.0.0.1"; .port = "8080"; } sub vcl_recv { lua.init("/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/local/lib/lua/5.1/?.lua;/usr/local/lib/lua/5.1/?/init.lua", "/usr/local/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so", "/my/foo.lua"); } sub vcl_deliver { set resp.http.x-json = lua.call("test_json"); set resp.http.x-now = lua.call("test_now"); return (deliver); } And following are the contents of foo.lua: local cjson = require("cjson") function test_json() local t = {} t[1] = "hello" t[2] = "world" return cjson.encode(t) end function test_now() return 1234 end But when I curl localhost:80 the x-json and x-now headers show up as empty in the response. I was wondering if I am missing something in the setup. I ran the LUA script independently and made sure its returning the right thing. I would appreciate any help on this issue. Also wanted to note that libvmod-lua doesn?t compile with LUA 5.3.2 which is the latest version of LUA. Are there plans to update the vmod? Thanks, ? Nishant. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Tue May 10 08:37:54 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 10 May 2016 10:37:54 +0200 Subject: ESI from remote host not working? In-Reply-To: <571F56F6.40103@andreas-loewen.de> References: <571F56F6.40103@andreas-loewen.de> Message-ID: That should work. Possible problems may be that you rewrote req.url before testing it, or you are setting req.backend_hint (or bereq.backend) later in the vcl. can you import std, and log the req.url right before the if-else, and log right before "set req.backend_hint = esi;"? It would also help if we had the client part of the log. -- Guillaume Quintard On Tue, Apr 26, 2016 at 1:54 PM, Andreas Loewen Internetservices < kontakt at andreas-loewen.de> wrote: > Dear all, > > i am using current varnish varnish 4. I have the following config: > > backend esi{ > .host = "x.x.x.x"; > .port = "8081"; > } > > ... > > sub vcl_recv { > set req.http.host = "xxx.yyy.de"; > ... > if (req.url ~ "^/typo3/.*" || req.url ~ "^/secure/.*") { > set req.backend_hint = typo3be; > return (synth(403, "Forbidden")); > } else if (req.url ~ "\.mesi\?" || req.url ~ "\.htm(\?.*)?$" || > req.url ~ "^/javax\.faces\..*" || req.url ~ "^/templates/.*" || req.url > ~"^/f/.*" || req.url ~"^/i/.*") { > set req.backend_hint = esi; > set req.hash_always_miss = true; > } else { > set req.backend_hint = typo3; > unset req.http.cookie; > } > > ... > } > > sub vcl_backend_response { > set beresp.do_esi = true; > ... > > } > > The esi backend is on a remote host, but it is reachable from this server. > > The funny thing is, that I keep getting 404's for my esi requests for > /fe_terminliste.mesi?pakid=1 . But they are comming from the default > backend (local Apache on port 81). varnishlog reads like: > > * << BeReq >> 5 > - Begin bereq 4 pass > - Timestamp Start: 1461670355.552619 0.000000 0.000000 > - BereqMethod GET > - BereqURL /fe_terminliste.mesi?pakid=1 > - BereqProtocol HTTP/1.1 > - BereqHeader Cache-Control: max-age=0 > - BereqHeader Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > - BereqHeader Upgrade-Insecure-Requests: 1 > - BereqHeader User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36 > - BereqHeader DNT: 1 > - BereqHeader Accept-Encoding: gzip, deflate > - BereqHeader Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4 > - BereqHeader Cookie: PHPSESSID=h7hk5n8pa95q4sd2m46ebbju14; > Typo3InstallTool=tgikum56sujjsv1cu6ra236kb0; > be_typo_user=5ad26762d12ad4b9fe2d321979fb75e4 > - BereqHeader Host: xxx:8081 > - BereqHeader X-Forwarded-For: 80.171.39.71 > - BereqHeader X-Varnish: 5 > - VCL_call BACKEND_FETCH > - VCL_return fetch > - BackendOpen 20 default(127.0.0.1,,81) 127.0.0.1 33017 > - Backend 20 default default(127.0.0.1,,81) > - Timestamp Bereq: 1461670355.552877 0.000257 0.000257 > - Timestamp Beresp: 1461670355.553530 0.000911 0.000654 > - BerespProtocol HTTP/1.1 > - BerespStatus 404 > - BerespReason Not Found > - BerespHeader Date: Tue, 26 Apr 2016 11:32:35 GMT > - BerespHeader Server: Apache/2.2.15 (CentOS) > - BerespHeader Content-Length: 310 > - BerespHeader Connection: close > - BerespHeader Content-Type: text/html; charset=iso-8859-1 > - TTL RFC 120 -1 -1 1461670356 1461670356 1461670355 0 0 > - VCL_call BACKEND_RESPONSE > - TTL VCL 120 10 0 1461670356 > - VCL_return deliver > - BerespHeader Content-Encoding: gzip > - BerespUnset Content-Length: 310 > - Storage malloc Transient > - ObjProtocol HTTP/1.1 > - ObjStatus 404 > - ObjReason Not Found > - ObjHeader Date: Tue, 26 Apr 2016 11:32:35 GMT > - ObjHeader Server: Apache/2.2.15 (CentOS) > - ObjHeader Content-Type: text/html; charset=iso-8859-1 > - ObjHeader Content-Encoding: gzip > - Fetch_Body 3 length - > - Gzip G F E 310 268 80 2064 2074 > - Timestamp BerespBody: 1461670355.554030 0.001411 0.000500 > - BackendClose 20 default(127.0.0.1,,81) > - Length 268 > - BereqAcct 623 0 623 180 310 490 > - End > > > So, does varnish restrict esi to localhost for some reason? Or what is > wrong? > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From S.Garau at liguriadigitale.it Fri May 13 07:48:10 2016 From: S.Garau at liguriadigitale.it (Garau Simone) Date: Fri, 13 May 2016 09:48:10 +0200 Subject: 503s for a non-cached host / Re: [SOLVED] Varnish stops waiting backend after 1 second Message-ID: Hi guys, Thank you guys for the help. I solved the main issue with another configuration that works?ish. I have two virtual host for the same site: one is cached, one is not. It is a Joomla site, so the frontend is the one cached and the backend is not. I still have an issue I can?t understand: I get some random 503 from the backend where Varnish isn?t caching. After a couple of refresh of the page is properly served. The Apache logs shows no activity when Varnish sends a 503, like Varnish doesn?t send the request.. Or why isn?t Apache logging? Can someone please give any hints? Thank you so much, this how the logs looks like: * << Request >> 2788942 - Begin req 2788933 rxreq - Timestamp Start: 1463124819.989770 0.000000 0.000000 - Timestamp Req: 1463124819.989770 0.000000 0.000000 - ReqStart 10.10.6.109 50229 - ReqMethod GET - ReqURL /administrator/index.php?option=com_installer&view=update&task=update.ajax& 1fb62ce4bf63027725460be31f1da764=1&eid=0&skip=700 - ReqProtocol HTTP/1.1 - ReqHeader Host: admin.not-the-real-url.it - ReqHeader Connection: keep-alive - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: */* - ReqHeader X-Requested-With: XMLHttpRequest - ReqHeader User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36 - ReqHeader Referer: http://admin.regione.liguria.it/administrator/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: it-IT,it;q=0.8,en-US;q=0.6,en;q=0.4,de;q=0.2,ru;q=0.2 - ReqHeader Cookie: fc_screen_resolution=1680x925; fc_uid=8e6c108858f91e4ca99ff688ce5f7638; ff607e18ab6c715f4bb35b5bbcbe1c56=9tv90q2et61f7l3kvff4d56v55; c25a506e7cbaa1047d8cf6231f8b6d85=atk1fmpnt3jlokc0jgar2foo23; smart_varnish_bypass=1 - ReqHeader X-Forwarded-For: 10.10.6.109 - VCL_call RECV - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 2788943 pass - Timestamp Fetch: 1463124824.986856 4.997086 4.997086 - RespProtocol HTTP/1.1 - RespStatus 503 - RespReason Backend fetch failed - RespHeader Date: Fri, 13 May 2016 07:33:44 GMT - RespHeader Server: Varnish - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - RespHeader X-Varnish: 2788942 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - RespHeader X-Domain-Status: EXCLUDED - RespHeader X-Cache: NOCACHE - VCL_return deliver - Timestamp Process: 1463124824.986913 4.997143 0.000057 - RespHeader Content-Length: 284 - Debug "RES_MODE 2" - RespHeader Connection: keep-alive - Timestamp Resp: 1463124824.986965 4.997195 0.000052 - ReqAcct 778 0 778 286 284 570 - End * << BeReq >> 1115881 - Begin bereq 1115880 pass - Timestamp Start: 1463124821.994498 0.000000 0.000000 - BereqMethod GET - BereqURL /administrator/index.php?option=com_installer&view=update&task=update.ajax& 1fb62ce4bf63027725460be31f1da764=1&eid=700&cache_timeout=3600 - BereqProtocol HTTP/1.1 - BereqHeader Host: admin.not-the-real-url.it - BereqHeader Accept: */* - BereqHeader X-Requested-With: XMLHttpRequest - BereqHeader User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36 - BereqHeader Referer: http://admin.regione.liguria.it/administrator/ - BereqHeader Accept-Encoding: gzip, deflate, sdch - BereqHeader Accept-Language: it-IT,it;q=0.8,en-US;q=0.6,en;q=0.4,de;q=0.2,ru;q=0.2 - BereqHeader Cookie: fc_screen_resolution=1680x925; fc_uid=8e6c108858f91e4ca99ff688ce5f7638; ff607e18ab6c715f4bb35b5bbcbe1c56=9tv90q2et61f7l3kvff4d56v55; c25a506e7cbaa1047d8cf6231f8b6d85=atk1fmpnt3jlokc0jgar2foo23; smart_varnish_bypass=1 - BereqHeader X-Forwarded-For: 10.10.6.109 - BereqHeader X-Varnish: 1115881 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 46 boot.default 127.0.0.1 8080 127.0.0.1 52937 - Timestamp Bereq: 1463124821.994727 0.000229 0.000229 - FetchError http first read error: EOF - BackendClose 46 boot.default - Timestamp Beresp: 1463124826.994768 5.000270 5.000041 - Timestamp Error: 1463124826.994777 5.000279 0.000009 - BerespProtocol HTTP/1.1 - BerespStatus 503 - BerespReason Service Unavailable - BerespReason Backend fetch failed - BerespHeader Date: Fri, 13 May 2016 07:33:46 GMT - BerespHeader Server: Varnish - VCL_call BACKEND_ERROR - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Retry-After: 5 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.1 - ObjStatus 503 - ObjReason Backend fetch failed - ObjHeader Date: Fri, 13 May 2016 07:33:46 GMT - ObjHeader Server: Varnish - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Retry-After: 5 - Length 284 - BereqAcct 790 0 790 0 0 0 - End Il giorno 09/05/16, 11:56, "Jaroslav Mit??" ha scritto: >Hi Simone, >1. try if you backend is realy working: put to your browser >127.0.0.1:8080/ - if there is no problem, check how long it takes to >response >2. if 1yes, try to call varnish admin console "varnishadm -T >localhost:6082 -S /etc/varnish/secret" and then use command >"backend.list" -> there you can see, if your backend(origin) is for >varnish valid and healthy backend. > >3. If your result will be 1yes and 2 sick, try to update your probe for >backend: >.timeout = 1s; #this is too strict, you can set 5s e.g. >.interval = 1s; #this is probably too often, I think every 60s is >enought. >-more info about setting probe >https://www.varnish-cache.org/trac/wiki/BackendPolling > >4. don't forget to restart varnish after probe changes. > >Hope it will help you. >jaroslav > >-----Original Message----- >From: varnish-misc-bounces+jaroslav.mitas=avg.com at varnish-cache.org >[mailto:varnish-misc-bounces+jaroslav.mitas=avg.com at varnish-cache.org] On >Behalf Of Garau Simone >Sent: 9. kv?tna 2016 10:38 >To: varnish-misc at varnish-cache.org >Subject: [NEED HELP] Varnish stops waiting backend after 1 second > >this problem is gonna make me crazy: my varnish istance stops waiting for >a backend response after exactly 1 second. >Every first call to a page is a 503 Backend > >Daemon is configured this way: >DAEMON_OPTS="-a :80 \ > -T localhost:6082 \ > -f /etc/varnish/default.vcl \ > -S /etc/varnish/secret \ > -p thread_pool_add_delay=2 \ > -p thread_pools=4 \ > -p thread_pool_min=200 \ > -p thread_pool_max=4000 \ > -p timeout_linger=50 \ > -p connect_timeout=300 \ > -p first_byte_timeout=300 \ > -p between_bytes_timeout=300 \ > -p send_timeout=900 \ > -s malloc,3G? > >and the VCL backend: >backend default { # Define one backend >.host = "127.0.0.1"; # IP or Hostname of backend >.port = "8080"; # Port Apache or whatever is listening >.probe = { > .url = "/"; > .timeout = 1s; > .interval = 1s; > .window = 10; > .threshold = 8; > } > .first_byte_timeout = 60s; # How long to wait before we receive a >first byte from our backend? > .connect_timeout = 60s; # How long to wait for a backend >connection? > .between_bytes_timeout = 60s; # How long to wait between bytes >received from our backend? >} > > >Here is the one call in the log: >* << Request >> 3440734 >- Begin req 3440733 rxreq >- Timestamp Start: 1462781837.623325 0.000000 0.000000 >- Timestamp Req: 1462781837.623325 0.000000 0.000000 >- ReqStart 10.20.129.118 58572 >- ReqMethod GET >- ReqURL xxxxx.html >- ReqProtocol HTTP/1.1 >- ReqHeader Accept: image/jpeg, application/x-ms-application, >image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, >application/vnd.ms-excel, application/vnd.ms-powerpoint, >application/msword, image/pjpeg, application/x-shockwave-flash, */* >- ReqHeader Referer: http://xxxxxx.html >- ReqHeader Accept-Language: it-IT >- ReqHeader User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows >NT 6.1; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET >CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E) >- ReqHeader Accept-Encoding: gzip, deflate >- ReqHeader Host: xxxxxx >- ReqHeader DNT: 1 >- ReqHeader Connection: Keep-Alive >- ReqHeader Cookie: fc_uid=p; >__utma=127650066.830977012.1423064118.1426582505.1426588086.20; >_ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; >ZNPCQ003-38303300=71a0f671; >ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; >_ga=GA1.2.830977012.142306411 >- ReqHeader X-Forwarded-For: 10.20.129.118 >- VCL_call RECV >- ReqUnset Host: xxx >- ReqHeader Host: xxx >- ReqURL /xxxx.html >- ReqUnset Cookie: fc_uid=p; >__utma=127650066.830977012.1423064118.1426582505.1426588086.20; >_ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; >ZNPCQ003-38303300=71a0f671; >ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; >_ga=GA1.2.830977012.142306411 >- ReqHeader Cookie: fc_uid=p; >__utma=127650066.830977012.1423064118.1426582505.1426588086.20; >_ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; >ZNPCQ003-38303300=71a0f671; >ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; >_ga=GA1.2.830977012.142306411 >- ReqUnset Cookie: fc_uid=p; >__utma=127650066.830977012.1423064118.1426582505.1426588086.20; >_ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; >ZNPCQ003-38303300=71a0f671; >ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; >_ga=GA1.2.830977012.142306411 >- ReqHeader Cookie: fc_uid=p; >__utma=127650066.830977012.1423064118.1426582505.1426588086.20; >_ga=GA1.3.830977012.1423064118; _gat_UA-13041322-1=1; >ZNPCQ003-38303300=71a0f671; >ff607e18ab6c715f4bb35b5bbcbe1c56=d82989olp2v0ur3gpl2ouprko6; >_ga=GA1.2.830977012.142306411 >- ReqHeader Surrogate-Capability: key=ESI/1.0 >- VCL_return hash >- ReqUnset Accept-Encoding: gzip, deflate >- ReqHeader Accept-Encoding: gzip >- VCL_call HASH >- VCL_return lookup >- VCL_call MISS >- VCL_return fetch >- Link bereq 3440735 fetch >- Timestamp Fetch: 1462781838.492085 0.868760 0.868760 >- Timestamp Process: 1462781838.492101 0.868776 0.000016 >- RespHeader Date: Mon, 09 May 2016 08:17:18 GMT >- RespHeader Server: Varnish >- RespHeader X-Varnish: 3440734 >- RespProtocol HTTP/1.1 >- RespStatus 503 >- RespReason Service Unavailable >- RespReason Service Unavailable >- VCL_call SYNTH >- VCL_return deliver >- RespHeader Content-Length: 0 >- Storage malloc Transient >- Debug "RES_MODE 2" >- RespHeader Connection: keep-alive >- Timestamp Resp: 1462781838.492145 0.868820 0.000044 >- ReqAcct 985 0 985 153 0 153 >- End > >Any precious help? Suggestion? > > >_______________________________________________ >varnish-misc mailing list >varnish-misc at varnish-cache.org >https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From geoff at uplex.de Fri May 13 08:12:07 2016 From: geoff at uplex.de (Geoff Simmons) Date: Fri, 13 May 2016 10:12:07 +0200 Subject: 503s for a non-cached host / Re: [SOLVED] Varnish stops waiting backend after 1 second In-Reply-To: References: Message-ID: <57358C57.6090809@uplex.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 05/13/2016 09:48 AM, Garau Simone wrote: > > I still have an issue I can?t understand: I get some random 503 > from the backend where Varnish isn?t caching. After a couple of > refresh of the page is properly served. > > The Apache logs shows no activity when Varnish sends a 503, like > Varnish doesn?t send the request.. Or why isn?t Apache logging? > * << BeReq >> 1115881 [..] > - Timestamp Bereq: 1463124821.994727 0.000229 0.000229 - > FetchError http first read error: EOF - BackendClose 46 > boot.default - Timestamp Beresp: 1463124826.994768 5.000270 > 5.000041 FetchError is the critical log entry for a 503 error, and in this case there was a failure reading the first bytes of the backend response after very close to 5 seconds. Looks very much like first_byte_timeout=5s elapsed. Varnish could also do that on a connection reset, meaning that the connection was closed by the backend or an intervening network component -- the latter might explain why Apache doesn't seem to receive the request. Since the time difference in the Beresp Timestamp is so close to exactly 5 seconds, I'd suspect that some component has hit a timeout. HTH, Geoff - -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJXNYxXAAoJEOUwvh9pJNURUIEP/RYgf223+oUrNe2ZLzwMdDE2 QGkrNNSQY6CleVW+lgIW2Y1gYZVUXsRaRP2qVwZVld6iTizVFiMFKP1ojQAhbWph 17k+JqVby2oggs2lRChFyWRpkwZigeMJmgopGTE3HJfm5Bhn1uY/qrCSHhQsxpRJ h63Cs6J0mlIoJREIAXxVBK0YS5LH/uoAjWemhtNksADCyir32uyxY2rREn7K7j8Q x/owTz9O76s2Xc1URk2DxhOk16k59AbfkwQSrFfBMubHhpa1wPXxXEUXnx4IKkTJ 3UDyXMK42vqiiZcECC2+Qgku9C4flxP+tCsIcKNTnYCya/rFKr7EesMx0UNUaLMX AesdDVDZRunIjZQM5B6yJePCE2eBxSsXZTXpwOthxo/p7D4aZGnaNqaS1zqcjDMO WvrVhWv76Nyq/OKZvuNjDRHxles2y5cNbjwOWGIrmvepSFbRgvQolc9JhXwIn3b9 mGwxGUA8n8oAhpPaYpWrJPnYW12lQuqcBJZGQBR0wjDgHNR9VmimB9UtAvSSI4nO ipwWjSH7LEhhv5k+Ao0o/bjZpadZA7F23L3onHxqI8+bBy3u+p0ul9wQr+Z5+Y+U i7kGbmPNOPgtvhfoV/84ax3YfsTp9fZBE07rk47s3iDhkwFcSZTIzmx0X8rt1Mxn 6nPvnDlUbL5F8VnE7YJN =3nFf -----END PGP SIGNATURE----- From webteknik.tcmb at gmail.com Wed May 18 07:11:08 2016 From: webteknik.tcmb at gmail.com (Web Teknik) Date: Wed, 18 May 2016 10:11:08 +0300 Subject: reset varnishstat counters? Message-ID: Hi, We are using Varnish 4.0 and it is working great. We are using varnishstat counters with icinga to monitor Varnish performance. For example, we monitor MAIN.sess_dropped counter to see if we need to tune number of threads or other parameters. However since it is a counter, once it becomes greater than zero, then icinga keeps sending alerts. I want to reset it to zero after resolving the issue (sometimes it is just because VM freeze). Is there a way to reset varnishstat counters without restarting varnish? Thanks. From perbu at varnish-software.com Wed May 18 08:10:58 2016 From: perbu at varnish-software.com (Per Buer) Date: Wed, 18 May 2016 10:10:58 +0200 Subject: reset varnishstat counters? In-Reply-To: References: Message-ID: On Wed, May 18, 2016 at 9:11 AM, Web Teknik wrote: > Hi, > > We are using Varnish 4.0 and it is working great. We are using > varnishstat counters with icinga to monitor Varnish performance. For > example, we monitor MAIN.sess_dropped counter to see if we need to > tune number of threads or other parameters. However since it is a > counter, once it becomes greater than zero, then icinga keeps sending > alerts. I want to reset it to zero after resolving the issue > (sometimes it is just because VM freeze). Is there a way to reset > varnishstat counters without restarting varnish? > Easy question. No, there is no way of doing that without altering Varnish itself. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From webteknik.tcmb at gmail.com Wed May 18 08:42:41 2016 From: webteknik.tcmb at gmail.com (Web Teknik) Date: Wed, 18 May 2016 11:42:41 +0300 Subject: reset varnishstat counters? In-Reply-To: References: Message-ID: It would be nice if Varnish had a way to do it since restarting clears the cache. It seems that we need to find a way to alert based on change rather than value. Thanks for answer. From perbu at varnish-software.com Wed May 18 08:53:00 2016 From: perbu at varnish-software.com (Per Buer) Date: Wed, 18 May 2016 10:53:00 +0200 Subject: reset varnishstat counters? In-Reply-To: References: Message-ID: On Wed, May 18, 2016 at 10:42 AM, Web Teknik wrote: > It would be nice if Varnish had a way to do it since restarting clears > the cache. > It seems that we need to find a way to alert based on change rather than > value. > Thanks for answer. > I disagree. Counters should be counters and I would not expect them to be reset. Look at things like the various linux kernel counters - AFAIK there is no way to reset them. You need to have your monitoring software convert the counters to gauges and then have the system trigger a warning when a certain gauge value passes a threshold. Per. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Wed May 18 11:02:01 2016 From: subharaj.manna at gmail.com (Debraj Manna) Date: Wed, 18 May 2016 16:32:01 +0530 Subject: Viewing varnish cache miss urls along with the backend IP Message-ID: Hi, I am using round-robin director in my vcl. I want to see all the cache miss urls along with the backend IP. Right now doing sudo varnishtop -i BereqURL it is only showing the urls not the backend ip where it is going 54.42 BereqURL /catalog/v1/sizechart/keys 152.29 BereqURL /catalog/v1/products/keys/ 152.17 BereqURL /catalog/v1/bundles/stc/ids/ 145.92 BereqURL /catalog/v1/bundles/stl/ids/ Is there a way I can also view the backend IPs along with the above output? Thanks, -------------- next part -------------- An HTML attachment was scrubbed... URL: From akelly at snafu.de Wed May 18 12:22:15 2016 From: akelly at snafu.de (Andrew Kelly) Date: Wed, 18 May 2016 14:22:15 +0200 Subject: Selective logging Message-ID: <1463574135.4409.3.camel@localhost.localdomain> Hi all, I've just run into an issue where somebody out there has a domain pointing at an IP address on one of my servers where Varnish is listening. It's a fairly new server, so this is probably an old entry that got orphaned or something. No big deal, I'm sure we'll get it cleared up. But in the mean time, it's filling my varnishncsa.log with false entries. Is there any way to selectively NOT log certain traffic? Many thanks in advance, Andy From dridi at varni.sh Wed May 18 14:39:51 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 18 May 2016 16:39:51 +0200 Subject: Viewing varnish cache miss urls along with the backend IP In-Reply-To: References: Message-ID: > Is there a way I can also view the backend IPs along with the above output? I don't think there is, varnishtop is a very basic statistics tool. Dridi From dridi at varni.sh Wed May 18 14:41:44 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 18 May 2016 16:41:44 +0200 Subject: Selective logging In-Reply-To: <1463574135.4409.3.camel@localhost.localdomain> References: <1463574135.4409.3.camel@localhost.localdomain> Message-ID: > But in the mean time, it's filling my varnishncsa.log with > false entries. Is there any way to selectively NOT log > certain traffic? Sure, it's documented in the man pages: man varnishncsa : for the -q and -g options man vsl : for the log records you can use as query criteria man vsl-query : for the query syntax details Dridi From viktor.gunnarson at ericsson.com Wed May 18 14:54:52 2016 From: viktor.gunnarson at ericsson.com (Viktor Gunnarson) Date: Wed, 18 May 2016 16:54:52 +0200 Subject: Selective logging In-Reply-To: <1463574135.4409.3.camel@localhost.localdomain> References: <1463574135.4409.3.camel@localhost.localdomain> Message-ID: <573C823C.2010503@ericsson.com> Hi, You can easily add a VSL query to your varnishncsa to filter what get logged like this: varnishncsa -q 'reqHeader:host !~ "foo.com"' At least on Varnish 4 and 4.1, not sure about Varnish 3... Best regards, Viktor On 18/05/16 14:22, Andrew Kelly wrote: > Hi all, > > I've just run into an issue where somebody out there has > a domain pointing at an IP address on one of my servers > where Varnish is listening. It's a fairly new server, so > this is probably an old entry that got orphaned or something. > No big deal, I'm sure we'll get it cleared up. > > But in the mean time, it's filling my varnishncsa.log with > false entries. Is there any way to selectively NOT log > certain traffic? > > Many thanks in advance, > > Andy > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Wed May 18 18:41:57 2016 From: subharaj.manna at gmail.com (Debraj Manna) Date: Thu, 19 May 2016 00:11:57 +0530 Subject: Viewing varnish cache miss urls along with the backend IP In-Reply-To: References: Message-ID: Is there any other way I can achieve the same? On May 18, 2016 8:10 PM, "Dridi Boukelmoune" wrote: > > Is there a way I can also view the backend IPs along with the above > output? > > I don't think there is, varnishtop is a very basic statistics tool. > > Dridi > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed May 18 18:54:54 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 18 May 2016 13:54:54 -0500 Subject: Viewing varnish cache miss urls along with the backend IP In-Reply-To: References: Message-ID: std.log() ? On May 18, 2016 13:51, "Debraj Manna" wrote: > Is there any other way I can achieve the same? > On May 18, 2016 8:10 PM, "Dridi Boukelmoune" wrote: > >> > Is there a way I can also view the backend IPs along with the above >> output? >> >> I don't think there is, varnishtop is a very basic statistics tool. >> >> Dridi >> > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Thu May 19 07:00:23 2016 From: subharaj.manna at gmail.com (Debraj Manna) Date: Thu, 19 May 2016 12:30:23 +0530 Subject: Viewing varnish cache miss urls along with the backend IP In-Reply-To: <46a24485242ddf1edc5eb9abee23e60c@www.cromagnon.com> References: <46a24485242ddf1edc5eb9abee23e60c@www.cromagnon.com> Message-ID: Not much elegant but using the varnish log I am able to achieve what I am looking for:- varnishlog | grep -i 'BereqURL\|BackendOpen' On Thu, May 19, 2016 at 12:34 AM, Craig Servin wrote: > I send things to varnishncsa log and then push that to an ELK stack so > that we can query on pull things like that out after the fact. > > Craig > > > > > On 2016-05-18 13:41, Debraj Manna wrote: > >> Is there any other way I can achieve the same? >> On May 18, 2016 8:10 PM, "Dridi Boukelmoune" wrote: >> >> Is there a way I can also view the backend IPs along with the >>>> >>> above output? >>> >>> I don't think there is, varnishtop is a very basic statistics tool. >>> >>> Dridi >>> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Thu May 19 08:16:14 2016 From: subharaj.manna at gmail.com (Debraj Manna) Date: Thu, 19 May 2016 13:46:14 +0530 Subject: Making varnish cache only certain requests with multiple backends Message-ID: Hi, We are using varnish for multiple round robin backends. as shown below:- backend d1 { .host = "172.16.84.69"; .port = "9802"; } backend d2 { .host = "172.16.84.68"; .port = "9802"; } backend r1 { .host = "172.16.84.69"; .port = "9802"; } backend r2 { .host = "172.16.84.68"; .port = "9802"; } Our requirement is if the urls is of form /d/r we will send the request to one of the backends given by r.backend() else we will send the request to d.backend(). If the url is of form /d/r but contains a header of Type = p then we will bypass varnish and send the request directly to backend. The vcl looks something like below:- sub vcl_recv { # ... # Don't cache r requests with type = p if(req.http.Type == "p") { req.backend_hint = r.backend(); return (pipe); } if(req.url ~ "^/d/r/") { set req.backend_hint = r.backend(); } else { set req.backend_hint = d.backend(); } Can some one please let me know if the below code is the best way to achieve what I am trying to do? if(req.http.Type == "p") { req.backend_hint = r.backend(); return (pipe); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From akelly at snafu.de Thu May 19 09:47:38 2016 From: akelly at snafu.de (Andrew Kelly) Date: Thu, 19 May 2016 11:47:38 +0200 Subject: Selective logging In-Reply-To: <573C823C.2010503@ericsson.com> References: <1463574135.4409.3.camel@localhost.localdomain> <573C823C.2010503@ericsson.com> Message-ID: <1463651258.4226.8.camel@localhost.localdomain> Viktor, Dridi, Thank you both for your tips, much appreciated. Passing a query filter to varnishncsa from the command line seems to work like a charm. But I can't seem to make it work from the init script. The shell seems to choke on the various quoted bits, no matter how things get escaped. Not a worry, though, I'm happy to use my own scripts to start things. Andy On Mi, 2016-05-18 at 16:54 +0200, Viktor Gunnarson wrote: > Hi, > > You can easily add a VSL query to your varnishncsa to filter what get > logged like this: > > varnishncsa -q 'reqHeader:host !~ "foo.com"' > At least on Varnish 4 and 4.1, not sure about Varnish 3... > > Best regards, > Viktor > > On 18/05/16 14:22, Andrew Kelly wrote: > > > Hi all, > > > > I've just run into an issue where somebody out there has > > a domain pointing at an IP address on one of my servers > > where Varnish is listening. It's a fairly new server, so > > this is probably an old entry that got orphaned or something. > > No big deal, I'm sure we'll get it cleared up. > > > > But in the mean time, it's filling my varnishncsa.log with > > false entries. Is there any way to selectively NOT log > > certain traffic? > > > > Many thanks in advance, > > > > Andy > > > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From dridi at varni.sh Thu May 19 11:04:47 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Thu, 19 May 2016 13:04:47 +0200 Subject: Selective logging In-Reply-To: <1463651258.4226.8.camel@localhost.localdomain> References: <1463574135.4409.3.camel@localhost.localdomain> <573C823C.2010503@ericsson.com> <1463651258.4226.8.camel@localhost.localdomain> Message-ID: > Passing a query filter to varnishncsa from the command > line seems to work like a charm. But I can't seem to > make it work from the init script. The shell seems to > choke on the various quoted bits, no matter how things > get escaped. Oh my, tripping on double shell expansion, what a surprise :) I can have a look at your init script configuration if you want. Dridi From dridi at varni.sh Thu May 19 11:06:59 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Thu, 19 May 2016 13:06:59 +0200 Subject: Viewing varnish cache miss urls along with the backend IP In-Reply-To: References: <46a24485242ddf1edc5eb9abee23e60c@www.cromagnon.com> Message-ID: On Thu, May 19, 2016 at 9:00 AM, Debraj Manna wrote: > Not much elegant but using the varnish log I am able to achieve what I am > looking for:- > > varnishlog | grep -i 'BereqURL\|BackendOpen' Have a look at `man varnishlog`, you can do the same with just `varnishlog -i BereqURL,BackendOpen`. Cheers From dridi at varni.sh Thu May 19 11:11:08 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Thu, 19 May 2016 13:11:08 +0200 Subject: Making varnish cache only certain requests with multiple backends In-Reply-To: References: Message-ID: On Thu, May 19, 2016 at 10:16 AM, Debraj Manna wrote: > Hi, > > We are using varnish for multiple round robin backends. as shown below:- > > backend d1 { > .host = "172.16.84.69"; > .port = "9802"; > } > backend d2 { > .host = "172.16.84.68"; > .port = "9802"; > } > backend r1 { > .host = "172.16.84.69"; > .port = "9802"; > } > backend r2 { > .host = "172.16.84.68"; > .port = "9802"; > } You don't need to duplicate backend definitions, your directors d and r can share two backends b1 and b2. > Can some one please let me know if the below code is the best way to > achieve what I am trying to do? > if(req.http.Type == "p") { > req.backend_hint = r.backend(); > return (pipe); > } It looks OK but if your client sends subsequent HTTP requests, Varnish won't see them after a return(pipe). You may use return(pass) instead, to bypass Varnish for individual requests. Dridi From dridi at varni.sh Thu May 19 11:14:13 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Thu, 19 May 2016 13:14:13 +0200 Subject: Problems with Varnish-LUA integration. In-Reply-To: References: Message-ID: > Also wanted to note that libvmod-lua doesn?t compile with LUA 5.3.2 which is > the latest version of LUA. Are there plans to update the vmod? Hello, This module is not part of the Varnish project, you should contact its author instead. Our community manager opened an issue on Github for your problem 9 days ago: https://github.com/flygoast/libvmod-lua/issues/1 Cheers From subharaj.manna at gmail.com Thu May 19 12:55:50 2016 From: subharaj.manna at gmail.com (Debraj Manna) Date: Thu, 19 May 2016 18:25:50 +0530 Subject: Making varnish cache only certain requests with multiple backends In-Reply-To: References: Message-ID: Can you please explain a bit more what advantage I will have by using return(pass). Are you saying if I use return(pipe) then the client's subsequent request will start getting 404 Error? My intention is if the header req.http.Type == "p" then i want to treat as if varnish is not present for those requests and the request going straight to backend. On Thu, May 19, 2016 at 4:41 PM, Dridi Boukelmoune wrote: > On Thu, May 19, 2016 at 10:16 AM, Debraj Manna > wrote: > > Hi, > > > > We are using varnish for multiple round robin backends. as shown below:- > > > > backend d1 { > > .host = "172.16.84.69"; > > .port = "9802"; > > } > > backend d2 { > > .host = "172.16.84.68"; > > .port = "9802"; > > } > > backend r1 { > > .host = "172.16.84.69"; > > .port = "9802"; > > } > > backend r2 { > > .host = "172.16.84.68"; > > .port = "9802"; > > } > > You don't need to duplicate backend definitions, your directors d and > r can share two backends b1 and b2. > > Can some one please let me know if the below code is the best way to > > achieve what I am trying to do? > > if(req.http.Type == "p") { > > req.backend_hint = r.backend(); > > return (pipe); > > } > > It looks OK but if your client sends subsequent HTTP requests, Varnish > won't see them after a return(pipe). You may use return(pass) instead, > to bypass Varnish for individual requests. > > Dridi > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Thu May 19 13:37:41 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Thu, 19 May 2016 15:37:41 +0200 Subject: Making varnish cache only certain requests with multiple backends In-Reply-To: References: Message-ID: On Thu, May 19, 2016 at 2:55 PM, Debraj Manna wrote: > Can you please explain a bit more what advantage I will have by using > return(pass). Are you saying if I use return(pipe) then the client's > subsequent request will start getting 404 Error? No, I mean that subsequent requests without "Type: p" on the same connection won't be seen by Varnish, they will go directly to the backend. > My intention is if the header req.http.Type == "p" then i want to treat as > if varnish is not present for those requests and the request going straight > to backend. I understand that, and return(pass) will send the request to the backend, but return(pipe) will send the whole connection to the backend. Dridi From pinakee at waltzz.com Fri May 20 14:38:45 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Fri, 20 May 2016 20:08:45 +0530 Subject: Configuring Varnish for Caching based on Cookies Message-ID: <573F2175.1020908@waltzz.com> Hi, I am a beginner with Varnish and planning to deploy it for our ecommerce live platform. I am trying to learn Varnish and VCL and having trouble for our specific scenario: I would like to cache pages when a user is not logged in - and this is determined by a cookie (which is standard). The ecommerce platform is based on python Django - and there are bunch of other cookies which are not relevant for caching but they can't be unset/removed as they are needed by the origin/backend server. Hence, the solution would be to check if the login cookie is present or not. If login cookie not present, cache the page and fetch from cache else pass it to the backend. I tried implementing the above logic in vcl_recv and vcl_backend_response with my limited knowledge, but varnish started serving blank pages. Would really appreciate if you could please guide me with the above. Also, if you could provide some good simple documentations to start with (I am going through the Varnish software book), would be go great help. Thanks, Pinakee From damian.abel at dserwin.me Sun May 22 11:54:19 2016 From: damian.abel at dserwin.me (dserwin) Date: Sun, 22 May 2016 13:54:19 +0200 Subject: VCL not cache but deliver Message-ID: <57419DEB.8090401@dserwin.me> Hi all, I have question related to Varnish VCL. I want to write vcl which under some conditions will not be caching object downloaded from backend but only deliver it to the client. I was trying to use vcl_backend_response subroutine, but returning (abandon) results in returning 503 Service Unavailable to the client. I'm using varnish 4.1.2. If sth is'nt clear enough please ask questions. Thanks for help. -- Damian From pada at posteo.de Sun May 22 22:22:39 2016 From: pada at posteo.de (Daniel Parthey) Date: Mon, 23 May 2016 00:22:39 +0200 Subject: VCL not cache but deliver In-Reply-To: <57419DEB.8090401@dserwin.me> References: <57419DEB.8090401@dserwin.me> Message-ID: <5742312F.8090208@posteo.de> Am 22.05.2016 um 13:54 schrieb dserwin: > I have question related to Varnish VCL. I want to write vcl which under > some conditions will not be caching object downloaded from backend but > only deliver it to the client. I was trying to use vcl_backend_response > subroutine, but returning (abandon) results in returning 503 Service > Unavailable to the client. I'm using varnish 4.1.2. Abandon aborts the backend fetch, see the state diagram: https://www.varnish-cache.org/docs/4.1/reference/states.html You don't want to cancel the download from backend, you want to tell varnish not to cache the object, but to deliver it. In vcl_backend_response you can set beresp.uncacheable = true; in order to prevent caching of the fetched object. See https://www.varnish-cache.org/docs/4.1/reference/vcl.html Example: sub vcl_backend_response { if (put your condition here) { set beresp.uncacheable = true; return (deliver); } } Kind regards Daniel -- https://emailselfdefense.fsf.org https://pgp.mit.edu/pks/lookup?op=get&search=0xB4DD34660B6F0F1B https://pgp.key-server.io/0xB4DD34660B6F0F1B Fingerprint: 366C BC38 B09B 4BFC 38A7 3CBC B4DD 3466 0B6F 0F1B -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From subharaj.manna at gmail.com Mon May 23 05:35:57 2016 From: subharaj.manna at gmail.com (Debraj Manna) Date: Mon, 23 May 2016 11:05:57 +0530 Subject: Making varnish cache only certain requests with multiple backends In-Reply-To: References: Message-ID: Thanks Dridi. One more doubt will the pass urls be counted as miss in the varnish cache miss counter? On May 19, 2016 7:08 PM, "Dridi Boukelmoune" wrote: > On Thu, May 19, 2016 at 2:55 PM, Debraj Manna > wrote: > > Can you please explain a bit more what advantage I will have by using > > return(pass). Are you saying if I use return(pipe) then the client's > > subsequent request will start getting 404 Error? > > No, I mean that subsequent requests without "Type: p" on the same > connection won't be seen by Varnish, they will go directly to the > backend. > > > My intention is if the header req.http.Type == "p" then i want to treat > as > > if varnish is not present for those requests and the request going > straight > > to backend. > > I understand that, and return(pass) will send the request to the > backend, but return(pipe) will send the whole connection to the > backend. > > Dridi > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Mon May 23 09:15:50 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Mon, 23 May 2016 11:15:50 +0200 Subject: Child died on request In-Reply-To: References: Message-ID: On Sat, May 21, 2016 at 2:20 PM, Yiannis Karayiannidis wrote: > Hi Dridi, > any news regarding the bug? > Could you suggest a work around? Hello Yiannis, This bug is fixed in the 4.0 branch and will be part of the next 4.0 release. I don't know when it will be released. A possible workaround to avoid the crash: sub vcl_pipe { if (!std.healthy(bereq.backend)) { return (synth(500)); } } I haven't actually tested the code, but not proceeding with no healthy backend should avoid crashing. Best Regards, Dridi From dridi at varni.sh Mon May 23 09:17:32 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Mon, 23 May 2016 11:17:32 +0200 Subject: Making varnish cache only certain requests with multiple backends In-Reply-To: References: Message-ID: On Mon, May 23, 2016 at 7:35 AM, Debraj Manna wrote: > Thanks Dridi. One more doubt will the pass urls be counted as miss in the > varnish cache miss counter? I suppose it would but in all honesty I have no idea. If I ever need to know I'll just try and see for myself ;) Cheers From pinakee at waltzz.com Mon May 23 09:23:13 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Mon, 23 May 2016 14:53:13 +0530 Subject: Varnish delivering blank page (content-length=0) Message-ID: <8e7a6e6e-b1af-8e28-f9b2-df3bc91ce595@waltzz.com> Hi, I am trying to set Varnish to cache content when login cookie is not present. Hence, I have modified the following subroutines: * vcl_recv: If the login cookie is not present, return hash. * vcl_back_response: If the login cookie is not present, cache the content (beresp.uncacheable set to false Whenever the vcl_recv is returning hash, varnish is generating blank page with content-length set 0. If vcl_recv returns pass, the page is served fine. I am new with Varnish and have spent hours figuring out but not able to. Would really appreciate if someone could help with the above. Thanks, Pinakee Following is the VCL: sub vcl_recv { # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you don't need, # rewriting the request, etc. set req.backend_hint = uwsgi; if (req.http.Cookie !~ "jivaana_user=") { return(hash); } } sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. if (beresp.http.Set-Cookie !~ "jivaana_user=") { if (beresp.ttl <= 0s) { set beresp.ttl = 120s; } set beresp.uncacheable = false; return (deliver); } Varnish log: * << BeReq >> 3 - Begin bereq 2 fetch - Timestamp Start: 1463994442.986360 0.000000 0.000000 - BereqMethod GET - BereqURL /catalogue/ - BereqProtocol HTTP/1.0 - BereqHeader Host: django_test - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader Accept-Encoding: gzip - BereqProtocol HTTP/1.1 - BereqHeader X-Varnish: 3 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 9001 127.0.0.1 38213 - Timestamp Bereq: 1463994442.986883 0.000524 0.000524 - Timestamp Beresp: 1463994443.734972 0.748612 0.748088 - BerespProtocol HTTP/1.1 - BerespStatus 200 - BerespReason OK - BerespHeader Expires: Mon, 23 May 2016 09:07:23 GMT - BerespHeader Vary: Cookie - BerespHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT - BerespHeader Cache-Control: max-age=0 - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 09:07:23 GMT; Max-Age=31449600; Path=/ - BerespHeader Date: Mon, 23 May 2016 09:07:23 GMT - TTL RFC 0 10 -1 1463994444 1463994444 1463994443 1463994443 0 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1463994444 - VCL_return deliver - Storage file s0 - ObjProtocol HTTP/1.1 - ObjStatus 200 - ObjReason OK - ObjHeader Expires: Mon, 23 May 2016 09:07:23 GMT - ObjHeader Vary: Cookie - ObjHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT - ObjHeader Cache-Control: max-age=0 - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 09:07:23 GMT; Max-Age=31449600; Path=/ - ObjHeader Date: Mon, 23 May 2016 09:07:23 GMT - Fetch_Body 0 none - - BackendReuse 21 boot.uwsgi - Timestamp BerespBody: 1463994443.735229 0.748869 0.000257 - Length 0 - BereqAcct 393 0 393 336 0 336 - End * << Request >> 2 - Begin req 1 rxreq - Timestamp Start: 1463994442.986091 0.000000 0.000000 - Timestamp Req: 1463994442.986091 0.000000 0.000000 - ReqStart 127.0.0.1 56147 - ReqMethod GET - ReqURL /catalogue/ - ReqProtocol HTTP/1.0 - ReqHeader Host: django_test - ReqHeader Connection: close - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; pagemap=0,0,0,0,0,0,0; jivaana_country=IN; mp_mixpanel__c=1; mp_mixpanel__c3=2; mp_mixpanel__c4=2; mp_mixpanel__c5=0; _ga=GA1.1.1762020660.1447765749; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20 - ReqHeader If-Modified-Since: Mon, 23 May 2016 09:03:11 GMT - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - ReqUnset Cookie: pagemap=0,0,0,0,0,1,0'; pagemap=0,0,0,0,0,0,0; jivaana_country=IN; mp_mixpanel__c=1; mp_mixpanel__c3=2; mp_mixpanel__c4=2; mp_mixpanel__c5=0; _ga=GA1.1.1762020660.1447765749; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20 - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Encoding: gzip - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 3 fetch - Timestamp Fetch: 1463994443.735287 0.749196 0.749196 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Expires: Mon, 23 May 2016 09:07:23 GMT - RespHeader Vary: Cookie - RespHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT - RespHeader Cache-Control: max-age=0 - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 09:07:23 GMT; Max-Age=31449600; Path=/ - RespHeader Date: Mon, 23 May 2016 09:07:23 GMT - RespHeader X-Varnish: 2 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1463994443.735341 0.749249 0.000054 - RespHeader Accept-Ranges: bytes - RespHeader Content-Length: 0 - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1463994443.735410 0.749319 0.000070 - ReqAcct 929 0 929 476 0 476 - End * << Session >> 1 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 56147 0.0.0.0:9003 127.0.0.1 9003 1463994442.985890 18 - Link req 2 rxreq - SessClose RESP_CLOSE 0.750 - End -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Mon May 23 09:54:33 2016 From: apj at mutt.dk (Andreas Plesner) Date: Mon, 23 May 2016 11:54:33 +0200 Subject: Making varnish cache only certain requests with multiple backends In-Reply-To: References: Message-ID: <20160523095433.GC28877@nerd.dk> On Mon, May 23, 2016 at 11:05:57AM +0530, Debraj Manna wrote: > Thanks Dridi. One more doubt will the pass urls be counted as miss in the > varnish cache miss counter? No, since there's no cache lookup. MAIN.s_pass will be incremented. -- Andreas From damian.abel at dserwin.me Mon May 23 17:46:02 2016 From: damian.abel at dserwin.me (Damian Serwin) Date: Mon, 23 May 2016 19:46:02 +0200 Subject: VCL not cache but deliver In-Reply-To: References: Message-ID: <574341DA.5050209@dserwin.me> Hi, thanks a lot! It is working as expected now. > Am 22.05.2016 um 13:54 schrieb dserwin: >> I have question related to Varnish VCL. I want to write vcl which under >> some conditions will not be caching object downloaded from backend but >> only deliver it to the client. I was trying to use vcl_backend_response >> subroutine, but returning (abandon) results in returning 503 Service >> Unavailable to the client. I'm using varnish 4.1.2. > Abandon aborts the backend fetch, see the state diagram: > https://www.varnish-cache.org/docs/4.1/reference/states.html > > You don't want to cancel the download from backend, you want > to tell varnish not to cache the object, but to deliver it. > > In vcl_backend_response you can > set beresp.uncacheable = true; > in order to prevent caching of the fetched object. > > See https://www.varnish-cache.org/docs/4.1/reference/vcl.html > > Example: > > sub vcl_backend_response { > if (put your condition here) { > set beresp.uncacheable = true; > return (deliver); > } > } > > Kind regards > Daniel From perbu at varnish-software.com Mon May 23 17:51:36 2016 From: perbu at varnish-software.com (Per Buer) Date: Mon, 23 May 2016 19:51:36 +0200 Subject: Configuring Varnish for Caching based on Cookies In-Reply-To: <573F2175.1020908@waltzz.com> References: <573F2175.1020908@waltzz.com> Message-ID: On Fri, May 20, 2016 at 4:38 PM, Pinakee BIswas wrote: > Hi, > > I am a beginner with Varnish and planning to deploy it for our ecommerce > live platform. I am trying to learn Varnish and VCL and having trouble for > our specific scenario: > > I would like to cache pages when a user is not logged in - and this is > determined by a cookie (which is standard). The ecommerce platform is based > on python Django - and there are bunch of other cookies which are not > relevant for caching but they can't be unset/removed as they are needed by > the origin/backend server. > > Hence, the solution would be to check if the login cookie is present or > not. If login cookie not present, cache the page and fetch from cache else > pass it to the backend. > Two tips. One is the cookie VMOD, which helps when dealing with cookies. If you don't wanna use it you can also look at the example here: https://www.varnish-cache.org/docs/4.1/users-guide/increasing-your-hitrate.html#cookies - it will at least point you in the right direction. > I tried implementing the above logic in vcl_recv and vcl_backend_response > with my limited knowledge, but varnish started serving blank pages. > That sounds like a backend bug. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Tue May 24 06:18:18 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Tue, 24 May 2016 11:48:18 +0530 Subject: Configuring Varnish for Caching based on Cookies In-Reply-To: References: <573F2175.1020908@waltzz.com> Message-ID: <615dfb4e-c60b-06ba-3bc1-2e90974e0cdd@waltzz.com> Hi Per Buer, Thanks for your response and for the tips. Would go through them and try to implement. Regarding the blank pages, I have sent a separate mail. I have installed Varnish from source. Since this is the first time with Varnish, we plan to start with simple scenarios for caching and then move on to more sophisticated ones for logged in users etc (also, as we scale up further). Would really appreciate if you could help me resolving the issue so that I could move ahead with configuring and deploying Varnish. Following is the content of the mail with the issue: Hi, I am trying to set Varnish to cache content when login cookie is not present. Hence, I have modified the following subroutines: * vcl_recv: If the login cookie is not present, return hash. * vcl_back_response: If the login cookie is not present, cache the content (beresp.uncacheable set to false Whenever the vcl_recv is returning hash, varnish is generating blank page with content-length set 0. If vcl_recv returns pass, the page is served fine. Following is the VCL: sub vcl_recv { # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you don't need, # rewriting the request, etc. set req.backend_hint = uwsgi; if (req.http.Cookie !~ "jivaana_user=") { return(hash); } } sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. if (beresp.http.Set-Cookie !~ "jivaana_user=") { if (beresp.ttl <= 0s) { set beresp.ttl = 120s; } set beresp.uncacheable = false; return (deliver); } Varnish log: * << BeReq >> 3 - Begin bereq 2 fetch - Timestamp Start: 1463994442.986360 0.000000 0.000000 - BereqMethod GET - BereqURL /catalogue/ - BereqProtocol HTTP/1.0 - BereqHeader Host: django_test - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader Accept-Encoding: gzip - BereqProtocol HTTP/1.1 - BereqHeader X-Varnish: 3 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 9001 127.0.0.1 38213 - Timestamp Bereq: 1463994442.986883 0.000524 0.000524 - Timestamp Beresp: 1463994443.734972 0.748612 0.748088 - BerespProtocol HTTP/1.1 - BerespStatus 200 - BerespReason OK - BerespHeader Expires: Mon, 23 May 2016 09:07:23 GMT - BerespHeader Vary: Cookie - BerespHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT - BerespHeader Cache-Control: max-age=0 - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 09:07:23 GMT; Max-Age=31449600; Path=/ - BerespHeader Date: Mon, 23 May 2016 09:07:23 GMT - TTL RFC 0 10 -1 1463994444 1463994444 1463994443 1463994443 0 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1463994444 - VCL_return deliver - Storage file s0 - ObjProtocol HTTP/1.1 - ObjStatus 200 - ObjReason OK - ObjHeader Expires: Mon, 23 May 2016 09:07:23 GMT - ObjHeader Vary: Cookie - ObjHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT - ObjHeader Cache-Control: max-age=0 - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 09:07:23 GMT; Max-Age=31449600; Path=/ - ObjHeader Date: Mon, 23 May 2016 09:07:23 GMT - Fetch_Body 0 none - - BackendReuse 21 boot.uwsgi - Timestamp BerespBody: 1463994443.735229 0.748869 0.000257 - Length 0 - BereqAcct 393 0 393 336 0 336 - End * << Request >> 2 - Begin req 1 rxreq - Timestamp Start: 1463994442.986091 0.000000 0.000000 - Timestamp Req: 1463994442.986091 0.000000 0.000000 - ReqStart 127.0.0.1 56147 - ReqMethod GET - ReqURL /catalogue/ - ReqProtocol HTTP/1.0 - ReqHeader Host: django_test - ReqHeader Connection: close - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; pagemap=0,0,0,0,0,0,0; jivaana_country=IN; mp_mixpanel__c=1; mp_mixpanel__c3=2; mp_mixpanel__c4=2; mp_mixpanel__c5=0; _ga=GA1.1.1762020660.1447765749; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20 - ReqHeader If-Modified-Since: Mon, 23 May 2016 09:03:11 GMT - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - ReqUnset Cookie: pagemap=0,0,0,0,0,1,0'; pagemap=0,0,0,0,0,0,0; jivaana_country=IN; mp_mixpanel__c=1; mp_mixpanel__c3=2; mp_mixpanel__c4=2; mp_mixpanel__c5=0; _ga=GA1.1.1762020660.1447765749; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20 - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Encoding: gzip - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 3 fetch - Timestamp Fetch: 1463994443.735287 0.749196 0.749196 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Expires: Mon, 23 May 2016 09:07:23 GMT - RespHeader Vary: Cookie - RespHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT - RespHeader Cache-Control: max-age=0 - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 09:07:23 GMT; Max-Age=31449600; Path=/ - RespHeader Date: Mon, 23 May 2016 09:07:23 GMT - RespHeader X-Varnish: 2 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1463994443.735341 0.749249 0.000054 - RespHeader Accept-Ranges: bytes - RespHeader Content-Length: 0 - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1463994443.735410 0.749319 0.000070 - ReqAcct 929 0 929 476 0 476 - End * << Session >> 1 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 56147 0.0.0.0:9003 127.0.0.1 9003 1463994442.985890 18 - Link req 2 rxreq - SessClose RESP_CLOSE 0.750 - End Thanks, Pinakee On 23/05/16 11:21 pm, Per Buer wrote: > > > On Fri, May 20, 2016 at 4:38 PM, Pinakee BIswas > wrote: > > Hi, > > I am a beginner with Varnish and planning to deploy it for our > ecommerce live platform. I am trying to learn Varnish and VCL and > having trouble for our specific scenario: > > I would like to cache pages when a user is not logged in - and > this is determined by a cookie (which is standard). The ecommerce > platform is based on python Django - and there are bunch of other > cookies which are not relevant for caching but they can't be > unset/removed as they are needed by the origin/backend server. > > Hence, the solution would be to check if the login cookie is > present or not. If login cookie not present, cache the page and > fetch from cache else pass it to the backend. > > > Two tips. One is the cookie VMOD, which helps when dealing with > cookies. If you don't wanna use it you can also look at the example > here: > https://www.varnish-cache.org/docs/4.1/users-guide/increasing-your-hitrate.html#cookies > - it will at least point you in the right direction. > > I tried implementing the above logic in vcl_recv and > vcl_backend_response with my limited knowledge, but varnish > started serving blank pages. > > > That sounds like a backend bug. > > -- > *Per Buer* > CTO | Varnish Software AS > Cell: +47 95839117 > We Make Websites Fly! > www.varnish-software.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Tue May 24 08:46:18 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Tue, 24 May 2016 14:16:18 +0530 Subject: Configuring Varnish for Caching based on Cookies In-Reply-To: <615dfb4e-c60b-06ba-3bc1-2e90974e0cdd@waltzz.com> References: <573F2175.1020908@waltzz.com> <615dfb4e-c60b-06ba-3bc1-2e90974e0cdd@waltzz.com> Message-ID: <052563f1-d695-55f7-0fa2-120e76bf5188@waltzz.com> Hi Per Buer, Just figured out that the backend wasn't sending Content-Length and Varnish seemed to use it. I have updated the backend to send Content-Length and now the pages are getting rendered. Will post to the community mailing list if I further stumble upon any query or roadblock. Thanks, Pinakee On 24/05/16 11:48 am, Pinakee BIswas wrote: > > Hi Per Buer, > > Thanks for your response and for the tips. Would go through them and > try to implement. > > Regarding the blank pages, I have sent a separate mail. > > I have installed Varnish from source. Since this is the first time > with Varnish, we plan to start with simple scenarios for caching and > then move on to more sophisticated ones for logged in users etc (also, > as we scale up further). > > Would really appreciate if you could help me resolving the issue so > that I could move ahead with configuring and deploying Varnish. > Following is the content of the mail with the issue: > > Hi, > > I am trying to set Varnish to cache content when login cookie is not > present. Hence, I have modified the following subroutines: > > * vcl_recv: If the login cookie is not present, return hash. > * vcl_back_response: If the login cookie is not present, cache the > content (beresp.uncacheable set to false > > Whenever the vcl_recv is returning hash, varnish is generating blank > page with content-length set 0. If vcl_recv returns pass, the page is > served fine. > > Following is the VCL: > > sub vcl_recv { > # Happens before we check if we have this in cache already. > # > # Typically you clean up the request here, removing cookies you > don't need, > # rewriting the request, etc. > set req.backend_hint = uwsgi; > > if (req.http.Cookie !~ "jivaana_user=") { > return(hash); > } > } > > sub vcl_backend_response { > # Happens after we have read the response headers from the backend. > # > # Here you clean the response headers, removing silly Set-Cookie > headers > # and other mistakes your backend does. > if (beresp.http.Set-Cookie !~ "jivaana_user=") { > if (beresp.ttl <= 0s) { > set beresp.ttl = 120s; > } > set beresp.uncacheable = false; > return (deliver); > } > > Varnish log: > > > * << BeReq >> 3 > - Begin bereq 2 fetch > - Timestamp Start: 1463994442.986360 0.000000 0.000000 > - BereqMethod GET > - BereqURL /catalogue/ > - BereqProtocol HTTP/1.0 > - BereqHeader Host: django_test > - BereqHeader Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > - BereqHeader Upgrade-Insecure-Requests: 1 > - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X > 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 > Safari/537.36 > - BereqHeader Accept-Language: en-US,en;q=0.8 > - BereqHeader X-Forwarded-For: 127.0.0.1 > - BereqHeader Accept-Encoding: gzip > - BereqProtocol HTTP/1.1 > - BereqHeader X-Varnish: 3 > - VCL_call BACKEND_FETCH > - VCL_return fetch > - BackendOpen 21 boot.uwsgi 127.0.0.1 9001 127.0.0.1 38213 > - Timestamp Bereq: 1463994442.986883 0.000524 0.000524 > - Timestamp Beresp: 1463994443.734972 0.748612 0.748088 > - BerespProtocol HTTP/1.1 > - BerespStatus 200 > - BerespReason OK > - BerespHeader Expires: Mon, 23 May 2016 09:07:23 GMT > - BerespHeader Vary: Cookie > - BerespHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT > - BerespHeader Cache-Control: max-age=0 > - BerespHeader X-Frame-Options: SAMEORIGIN > - BerespHeader Content-Type: text/html; charset=utf-8 > - BerespHeader Set-Cookie: > csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 > 09:07:23 GMT; Max-Age=31449600; Path=/ > - BerespHeader Date: Mon, 23 May 2016 09:07:23 GMT > - TTL RFC 0 10 -1 1463994444 1463994444 1463994443 > 1463994443 0 > - VCL_call BACKEND_RESPONSE > - TTL VCL 120 10 0 1463994444 > - VCL_return deliver > - Storage file s0 > - ObjProtocol HTTP/1.1 > - ObjStatus 200 > - ObjReason OK > - ObjHeader Expires: Mon, 23 May 2016 09:07:23 GMT > - ObjHeader Vary: Cookie > - ObjHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT > - ObjHeader Cache-Control: max-age=0 > - ObjHeader X-Frame-Options: SAMEORIGIN > - ObjHeader Content-Type: text/html; charset=utf-8 > - ObjHeader Set-Cookie: > csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 > 09:07:23 GMT; Max-Age=31449600; Path=/ > - ObjHeader Date: Mon, 23 May 2016 09:07:23 GMT > - Fetch_Body 0 none - > - BackendReuse 21 boot.uwsgi > - Timestamp BerespBody: 1463994443.735229 0.748869 0.000257 > - Length 0 > - BereqAcct 393 0 393 336 0 336 > - End > > * << Request >> 2 > - Begin req 1 rxreq > - Timestamp Start: 1463994442.986091 0.000000 0.000000 > - Timestamp Req: 1463994442.986091 0.000000 0.000000 > - ReqStart 127.0.0.1 56147 > - ReqMethod GET > - ReqURL /catalogue/ > - ReqProtocol HTTP/1.0 > - ReqHeader Host: django_test > - ReqHeader Connection: close > - ReqHeader Cache-Control: max-age=0 > - ReqHeader Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > - ReqHeader Upgrade-Insecure-Requests: 1 > - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X > 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 > Safari/537.36 > - ReqHeader Accept-Encoding: gzip, deflate, sdch > - ReqHeader Accept-Language: en-US,en;q=0.8 > - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; > pagemap=0,0,0,0,0,0,0; jivaana_country=IN; mp_mixpanel__c=1; > mp_mixpanel__c3=2; mp_mixpanel__c4=2; mp_mixpanel__c5=0; > _ga=GA1.1.1762020660.1447765749; > mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20 > - ReqHeader If-Modified-Since: Mon, 23 May 2016 09:03:11 GMT > - ReqHeader X-Forwarded-For: 127.0.0.1 > - VCL_call RECV > - ReqUnset Cookie: pagemap=0,0,0,0,0,1,0'; > pagemap=0,0,0,0,0,0,0; jivaana_country=IN; mp_mixpanel__c=1; > mp_mixpanel__c3=2; mp_mixpanel__c4=2; mp_mixpanel__c5=0; > _ga=GA1.1.1762020660.1447765749; > mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20 > - VCL_return hash > - ReqUnset Accept-Encoding: gzip, deflate, sdch > - ReqHeader Accept-Encoding: gzip > - VCL_call HASH > - VCL_return lookup > - VCL_call MISS > - VCL_return fetch > - Link bereq 3 fetch > - Timestamp Fetch: 1463994443.735287 0.749196 0.749196 > - RespProtocol HTTP/1.1 > - RespStatus 200 > - RespReason OK > - RespHeader Expires: Mon, 23 May 2016 09:07:23 GMT > - RespHeader Vary: Cookie > - RespHeader Last-Modified: Mon, 23 May 2016 09:07:23 GMT > - RespHeader Cache-Control: max-age=0 > - RespHeader X-Frame-Options: SAMEORIGIN > - RespHeader Content-Type: text/html; charset=utf-8 > - RespHeader Set-Cookie: > csrftoken=5BUZRL4gdO7uTDtWXnbqQko3CelDmOAg; expires=Mon, 22-May-2017 > 09:07:23 GMT; Max-Age=31449600; Path=/ > - RespHeader Date: Mon, 23 May 2016 09:07:23 GMT > - RespHeader X-Varnish: 2 > - RespHeader Age: 0 > - RespHeader Via: 1.1 varnish-v4 > - VCL_call DELIVER > - VCL_return deliver > - Timestamp Process: 1463994443.735341 0.749249 0.000054 > - RespHeader Accept-Ranges: bytes > - RespHeader Content-Length: 0 > - Debug "RES_MODE 2" > - RespHeader Connection: close > - Timestamp Resp: 1463994443.735410 0.749319 0.000070 > - ReqAcct 929 0 929 476 0 476 > - End > > * << Session >> 1 > - Begin sess 0 HTTP/1 > - SessOpen 127.0.0.1 56147 0.0.0.0:9003 127.0.0.1 9003 > 1463994442.985890 18 > - Link req 2 rxreq > - SessClose RESP_CLOSE 0.750 > - End > > Thanks, > Pinakee > On 23/05/16 11:21 pm, Per Buer wrote: >> >> >> On Fri, May 20, 2016 at 4:38 PM, Pinakee BIswas >> wrote: >> >> Hi, >> >> I am a beginner with Varnish and planning to deploy it for our >> ecommerce live platform. I am trying to learn Varnish and VCL and >> having trouble for our specific scenario: >> >> I would like to cache pages when a user is not logged in - and >> this is determined by a cookie (which is standard). The ecommerce >> platform is based on python Django - and there are bunch of other >> cookies which are not relevant for caching but they can't be >> unset/removed as they are needed by the origin/backend server. >> >> Hence, the solution would be to check if the login cookie is >> present or not. If login cookie not present, cache the page and >> fetch from cache else pass it to the backend. >> >> >> Two tips. One is the cookie VMOD, which helps when dealing with >> cookies. If you don't wanna use it you can also look at the example >> here: >> https://www.varnish-cache.org/docs/4.1/users-guide/increasing-your-hitrate.html#cookies >> - it will at least point you in the right direction. >> >> I tried implementing the above logic in vcl_recv and >> vcl_backend_response with my limited knowledge, but varnish >> started serving blank pages. >> >> >> That sounds like a backend bug. >> >> -- >> *Per Buer* >> CTO | Varnish Software AS >> Cell: +47 95839117 >> We Make Websites Fly! >> www.varnish-software.com >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Tue May 24 08:58:46 2016 From: perbu at varnish-software.com (Per Buer) Date: Tue, 24 May 2016 10:58:46 +0200 Subject: Configuring Varnish for Caching based on Cookies In-Reply-To: <052563f1-d695-55f7-0fa2-120e76bf5188@waltzz.com> References: <573F2175.1020908@waltzz.com> <615dfb4e-c60b-06ba-3bc1-2e90974e0cdd@waltzz.com> <052563f1-d695-55f7-0fa2-120e76bf5188@waltzz.com> Message-ID: On Tue, May 24, 2016 at 10:46 AM, Pinakee BIswas wrote: > I am trying to set Varnish to cache content when login cookie is not > present. Hence, I have modified the following subroutines: > > What you want to do is to modify the request so the default VCL will cache it rather than force Varnish to cache it. If you force Varnish to cache it Varnish will end up caching it with cookies and such. This is potentially dangerous. So I would recommend something like this in vcl_recv: if (req.http.cookie !~ "logged in") { unset req.http.cookie; # strip the cookies - we don't need them } # if there is a cookie here still the default policy will make sure the request isn't cached. Per. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Tue May 24 09:13:13 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Tue, 24 May 2016 14:43:13 +0530 Subject: Configuring Varnish for Caching based on Cookies In-Reply-To: References: <573F2175.1020908@waltzz.com> <615dfb4e-c60b-06ba-3bc1-2e90974e0cdd@waltzz.com> <052563f1-d695-55f7-0fa2-120e76bf5188@waltzz.com> Message-ID: Hi Per Buer, Thanks for the response and suggestion. I agree that caching with Cookies is a dangerous thing as cookies are mostly user specific and caching could leak them. But the challenge here is that we have few cookies which would be needed by the backend. For example, we use a cookie which specifies the country for the client/user. The cookie is used by the backend to figure out the country the user is in. But then I would like to cache the pages for a specific country. So, all the pages for users in US should be cached with a key using the country cookie - was trying this: sub vcl_hash { if (req.http.cookie ~ "user_country=") { set req.http.X-TMP = regsub(req.http.cookie, ".*user_country=([^;]+);.*", "\1"); hash_data(req.http.X-TMP); unset req.http.X-TMP; } } I am not clear if unset strips the cookies so that they don't get cached or they are completely removed from the Cookie header before the request being passed to the backend. If it's former, it would be nice. Any suggestion how to handle the above scenario would be great. Thanks, Pinakee On 24/05/16 2:28 pm, Per Buer wrote: > > > On Tue, May 24, 2016 at 10:46 AM, Pinakee BIswas > wrote: > >> I am trying to set Varnish to cache content when login cookie is >> not present. Hence, I have modified the following subroutines: >> > > What you want to do is to modify the request so the default VCL will > cache it rather than force Varnish to cache it. If you force Varnish > to cache it Varnish will end up caching it with cookies and such. This > is potentially dangerous. > > So I would recommend something like this in vcl_recv: > > if (req.http.cookie !~ "logged in") { > unset req.http.cookie; # strip the cookies - we don't need them > } > > # if there is a cookie here still the default policy will make sure > the request isn't cached. > > > > Per. > > -- > *Per Buer* > CTO | Varnish Software AS > Cell: +47 95839117 > We Make Websites Fly! > www.varnish-software.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Tue May 24 09:20:25 2016 From: perbu at varnish-software.com (Per Buer) Date: Tue, 24 May 2016 11:20:25 +0200 Subject: Configuring Varnish for Caching based on Cookies In-Reply-To: References: <573F2175.1020908@waltzz.com> <615dfb4e-c60b-06ba-3bc1-2e90974e0cdd@waltzz.com> <052563f1-d695-55f7-0fa2-120e76bf5188@waltzz.com> Message-ID: On Tue, May 24, 2016 at 11:13 AM, Pinakee BIswas wrote: > Hi Per Buer, > > Thanks for the response and suggestion. I agree that caching with Cookies > is a dangerous thing as cookies are mostly user specific and caching could > leak them. > > But the challenge here is that we have few cookies which would be needed > by the backend. For example, we use a cookie which specifies the country > for the client/user. The cookie is used by the backend to figure out the > country the user is in. But then I would like to cache the pages for a > specific country. So, all the pages for users in US should be cached with a > key using the country cookie - was trying this: > The preferred way for dealing with issues like countries is to use Vary. You can have Varnish synthesize a X-Client-Country header and then have the backend issue a Vary: X-Client-Country. Then Varnish will keep different pages for different countries apart. In addition a PURGE on a URL will clean out _all_ variants of the country. If you modify the hash this becomes more complicated. As you'll have to invalidate all variants yourself. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Tue May 24 10:47:55 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Tue, 24 May 2016 12:47:55 +0200 Subject: W3C like format logging Message-ID: Hello, I'm new to varnish and I'm setting up for some of the websites I manage. One of the difficulties I'm having is to debug it. I know that there's varnishlog, but it would be really helpful to have something similar to W3C logging for all request served by varnish. In addition to standard fields, there will be one extra field to show if the request was served from cache, from back-end, etc. Is there something like this or a way to have this kind of log? Thanx, Albert -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ Imagination is more important than knowledge *Albert Einstein* The three chief virtues of a programmer are: Laziness, Impatience and Hubris *Larry Wall* Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders *Erwin Rommel* The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. *Randall E. Stross* Measuring programming progress by lines of code is like measuring aircraft building progress by weight. *Bill Gates* -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Tue May 24 10:51:07 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Tue, 24 May 2016 12:51:07 +0200 Subject: VCL debugging Message-ID: Hi everyone, is there a way to log some info from vcl config file? It would help a lot to log some info on a file to check if the configuration is correct or not. Thanx, Albert -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ Imagination is more important than knowledge *Albert Einstein* The three chief virtues of a programmer are: Laziness, Impatience and Hubris *Larry Wall* Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders *Erwin Rommel* The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. *Randall E. Stross* Measuring programming progress by lines of code is like measuring aircraft building progress by weight. *Bill Gates* -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Tue May 24 10:57:03 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Tue, 24 May 2016 12:57:03 +0200 Subject: Serving multiple sites from varnish Message-ID: Hi, I have several sites that I want to cache using the same instance of varnish. I did setup the backend using something like: if (req.http.host == "time.ikub.al") { # Process through time backend set req.backend_hint = timeserver; } if (req.http.host == "m.time.ikub.al") { # Process through time mobile backend set req.backend_hint = timemobileserver; } and hash method looks like: sub vcl_hash { # Called after vcl_recv to create a hash value for the request. This is used as a key # to look up the object in Varnish. hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } # hash cookies for requests that have them if (req.http.Cookie) { hash_data(req.http.Cookie); } } However, I noticed that pages from mobile version where served when on desktop and vice-versa. This happened on pages with the same name, for example "Default.aspx". As I understand the code above the hash should include the host part and this should not happen. Am I missing something, or is there some recommendation on how to handle multiple sites? Thanx, Albert -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ Imagination is more important than knowledge *Albert Einstein* The three chief virtues of a programmer are: Laziness, Impatience and Hubris *Larry Wall* Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders *Erwin Rommel* The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. *Randall E. Stross* Measuring programming progress by lines of code is like measuring aircraft building progress by weight. *Bill Gates* -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Tue May 24 14:08:29 2016 From: perbu at varnish-software.com (Per Buer) Date: Tue, 24 May 2016 16:08:29 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: On Tue, May 24, 2016 at 12:47 PM, Albert Tollku?i wrote: > Hello, > I'm new to varnish and I'm setting up for some of the websites I manage. > > One of the difficulties I'm having is to debug it. I know that there's > varnishlog, but it would be really helpful to have something similar to W3C > logging for all request served by varnish. In addition to standard fields, > there will be one extra field to show if the request was served from cache, > from back-end, etc. > > Is there something like this or a way to have this kind of log? > Yes. Use varnishncsa. See the man page for details. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Tue May 24 14:13:35 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 24 May 2016 16:13:35 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: Hi Albert, I'm not familiar to w3c logging, but it looks to me that you are looking for varnishncsa. The format string can be customized and the '-q' parameter will allow you to filter out unwanted request. Also, you can use the std vmod and std.log() to fill a header with HIT|MISS|PASS and retrieve that in varnishncsa. -- Guillaume Quintard On Tue, May 24, 2016 at 12:47 PM, Albert Tollku?i wrote: > Hello, > I'm new to varnish and I'm setting up for some of the websites I manage. > > One of the difficulties I'm having is to debug it. I know that there's > varnishlog, but it would be really helpful to have something similar to W3C > logging for all request served by varnish. In addition to standard fields, > there will be one extra field to show if the request was served from cache, > from back-end, etc. > > Is there something like this or a way to have this kind of log? > > Thanx, > Albert > > > -- > Web: http://www.tollkuci.com > Follow me on: LinkedIn Google+ > Facebook > Twitter > Career 2.0 > > ------------------------------ > > Imagination is more important than knowledge > *Albert Einstein* > > The three chief virtues of a programmer are: Laziness, Impatience and > Hubris > *Larry Wall* > > Men are basically smart or dumb and lazy or ambitious. The dumb and > ambitious ones are dangerous and I get rid of them. The dumb and lazy ones > I give mundane duties. The smart ambitious ones I put on my staff. The > smart and lazy ones I make my commanders > *Erwin Rommel* > > The best programmers are not marginally better than merely good ones. > They are an order-of-magnitude better, measured by whatever standard: > conceptual creativity, speed, ingenuity of design, or problem-solving > ability. > *Randall E. Stross* > > Measuring programming progress by lines of code is like measuring aircraft > building progress by weight. > *Bill Gates* > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.schumacher at flightaware.com Tue May 24 14:28:40 2016 From: adam.schumacher at flightaware.com (Adam Schumacher) Date: Tue, 24 May 2016 14:28:40 +0000 Subject: W3C like format logging [WARNING: DKIM validation failed] In-Reply-To: References: Message-ID: varnishncsa will output logs in apache/ncsa compatible format. It also supports the -m tag:regex filter that varnishlog does. It also provides additional varnish-specific items including hits/misses. `man varnishnsca` for details. ::Adam On 5/24/16, 5:47 AM, "varnish-misc-bounces+adam.schumacher=flightaware.com at varnish-cache.org on behalf of Albert Tollku?i" wrote: > >Hello, >I'm new to varnish and I'm setting up for some of the websites I manage. > > >One of the difficulties I'm having is to debug it. I know that there's varnishlog, but it would be really helpful to have something similar to W3C logging for all request served by varnish. In addition to standard fields, there will be one extra field to show > if the request was served from cache, from back-end, etc. > > >Is there something like this or a way to have this kind of log? > > >Thanx, >Albert > > > > >-- >Web: http://www.tollkuci.com > >Follow me on: LinkedIn Google+ Facebook Twitter Career > 2.0 >________________________________________ > > >Imagination is more important than knowledge >Albert Einstein > >The three chief virtues of a programmer are: Laziness, Impatience and Hubris >Larry Wall > >Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders >Erwin Rommel > >The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. >Randall E. Stross > > >Measuring programming progress by lines of code is like measuring aircraft building progress by weight. >Bill Gates > > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4694 bytes Desc: not available URL: From guillaume at varnish-software.com Tue May 24 15:31:51 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 24 May 2016 17:31:51 +0200 Subject: VCL debugging In-Reply-To: References: Message-ID: On Tue, May 24, 2016 at 12:51 PM, Albert Tollku?i wrote: > Hi everyone, > is there a way to log some info from vcl config file? It would help a lot > to log some info on a file to check if the configuration is correct or not. > > > What kind of info do you need? "Correct" is a very broad term :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Tue May 24 15:33:27 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Tue, 24 May 2016 17:33:27 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: Thanx Per and Guillaume. From what I'm reading varnishncsa is exactly what I need. I'll try adding the extra header for HIT|MISS|PASS and hopefully it will work. Albert On Tue, May 24, 2016 at 4:13 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > Hi Albert, > > I'm not familiar to w3c logging, but it looks to me that you are looking > for varnishncsa. The format string can be customized and the '-q' parameter > will allow you to filter out unwanted request. > > Also, you can use the std vmod and std.log() to fill a header with > HIT|MISS|PASS and retrieve that in varnishncsa. > > -- > Guillaume Quintard > > On Tue, May 24, 2016 at 12:47 PM, Albert Tollku?i < > albert.tollkuci at gmail.com> wrote: > >> Hello, >> I'm new to varnish and I'm setting up for some of the websites I manage. >> >> One of the difficulties I'm having is to debug it. I know that there's >> varnishlog, but it would be really helpful to have something similar to W3C >> logging for all request served by varnish. In addition to standard fields, >> there will be one extra field to show if the request was served from cache, >> from back-end, etc. >> >> Is there something like this or a way to have this kind of log? >> >> Thanx, >> Albert >> >> >> -- >> Web: http://www.tollkuci.com >> Follow me on: LinkedIn >> Google+ Facebook >> Twitter >> Career 2.0 >> >> ------------------------------ >> >> Imagination is more important than knowledge >> *Albert Einstein* >> >> The three chief virtues of a programmer are: Laziness, Impatience and >> Hubris >> *Larry Wall* >> >> Men are basically smart or dumb and lazy or ambitious. The dumb and >> ambitious ones are dangerous and I get rid of them. The dumb and lazy ones >> I give mundane duties. The smart ambitious ones I put on my staff. The >> smart and lazy ones I make my commanders >> *Erwin Rommel* >> >> The best programmers are not marginally better than merely good ones. >> They are an order-of-magnitude better, measured by whatever standard: >> conceptual creativity, speed, ingenuity of design, or problem-solving >> ability. >> *Randall E. Stross* >> >> Measuring programming progress by lines of code is like measuring >> aircraft building progress by weight. >> *Bill Gates* >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ Imagination is more important than knowledge *Albert Einstein* The three chief virtues of a programmer are: Laziness, Impatience and Hubris *Larry Wall* Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders *Erwin Rommel* The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. *Randall E. Stross* Measuring programming progress by lines of code is like measuring aircraft building progress by weight. *Bill Gates* -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Tue May 24 15:35:06 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Tue, 24 May 2016 17:35:06 +0200 Subject: VCL debugging In-Reply-To: References: Message-ID: For example, if a cookie is present and I'm testing in vecl_recv some condition, I would like to have the option to log it somewhere, so I know that the condition is met. Albert On Tue, May 24, 2016 at 5:31 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > > > On Tue, May 24, 2016 at 12:51 PM, Albert Tollku?i < > albert.tollkuci at gmail.com> wrote: > >> Hi everyone, >> is there a way to log some info from vcl config file? It would help a lot >> to log some info on a file to check if the configuration is correct or not. >> >> >> What kind of info do you need? > > "Correct" is a very broad term :-) > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ Imagination is more important than knowledge *Albert Einstein* The three chief virtues of a programmer are: Laziness, Impatience and Hubris *Larry Wall* Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders *Erwin Rommel* The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. *Randall E. Stross* Measuring programming progress by lines of code is like measuring aircraft building progress by weight. *Bill Gates* -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Tue May 24 15:53:56 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 24 May 2016 17:53:56 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: My bad, you don't need std.log at all (sorry, I was distracted) It boils down to: sub vcl_hit { set req.http.x-state = "HIT"; } sub vcl_miss { set req.http.x-state = "MISS"; } sub vcl_pass { set req.http.x-state = "PASS"; } And getting that x-state header in varnishncsa. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Tue May 24 15:57:14 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 24 May 2016 17:57:14 +0200 Subject: VCL debugging In-Reply-To: References: Message-ID: On May 24, 2016 17:35, "Albert Tollku?i" wrote: > > For example, if a cookie is present and I'm testing in vecl_recv some condition, I would like to have the option to log it somewhere, so I know that the condition is met. > Then, here, std.log is an option. Another option is writing a vtc to test this out (to get you started). -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Tue May 24 16:24:46 2016 From: perbu at varnish-software.com (Per Buer) Date: Tue, 24 May 2016 18:24:46 +0200 Subject: VCL debugging In-Reply-To: References: Message-ID: On Tue, May 24, 2016 at 12:51 PM, Albert Tollku?i wrote: > Hi everyone, > is there a way to log some info from vcl config file? It would help a lot > to log some info on a file to check if the configuration is correct or not. > import std; (..) std.log("X-Foo is " + req.http.x-foo); -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Tue May 24 19:53:12 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 24 May 2016 21:53:12 +0200 Subject: Serving multiple sites from varnish In-Reply-To: References: Message-ID: It should indeed work, IIRC, the hash_data calls are visible in varnishlog, you can start with that. On a side note, please consider trimming down your signature when you post on public mailing-list. As a rule of thumb, your signature shouldn't be longer than your actual message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ross at trademe.co.nz Tue May 24 20:21:08 2016 From: ross at trademe.co.nz (Ross Brown) Date: Tue, 24 May 2016 20:21:08 +0000 Subject: VCL debugging In-Reply-To: References: Message-ID: Have you looked at Varnishlog? https://www.varnish-cache.org/docs/trunk/reference/varnishlog.html To add custom events, you can use the std.log function, which is part of vmod_std. https://www.varnish-cache.org/docs/trunk/reference/vmod_std.generated.html?#func-log import std; sub vcl_recv { std.log (?Requested URL : ? + req.url); ? From: varnish-misc-bounces+ross=trademe.co.nz at varnish-cache.org [mailto:varnish-misc-bounces+ross=trademe.co.nz at varnish-cache.org] On Behalf Of Albert Tollku?i Sent: Tuesday, 24 May 2016 10:51 PM To: varnish-misc at varnish-cache.org Subject: VCL debugging Hi everyone, is there a way to log some info from vcl config file? It would help a lot to log some info on a file to check if the configuration is correct or not. Thanx, Albert -- Web: http://www.tollkuci.com Follow me on: [http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2flinkedin_icon.png.jpgx] LinkedIn [http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2fgoogleplus-icon.png.jpgx] Google+ [http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2fFacebook_Icon.png.jpgx] Facebook [http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2fTwitterIcon.png.jpgx] Twitter [http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2fcareers-icon.png.jpgx] Career 2.0 ________________________________ Imagination is more important than knowledge Albert Einstein The three chief virtues of a programmer are: Laziness, Impatience and Hubris Larry Wall Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders Erwin Rommel The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. Randall E. Stross Measuring programming progress by lines of code is like measuring aircraft building progress by weight. Bill Gates -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Wed May 25 09:02:23 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Wed, 25 May 2016 14:32:23 +0530 Subject: CSRF token and caching Message-ID: <46c47618-a37b-788e-34d3-24755601f666@waltzz.com> Hi, Our backend uses CSRF tokens for form posts. For pages with forms, if Varnish caching is enabled for such pages, form post is failing due to CSRF error. Is there a way to handle this? I have been reading that using ESI is a solution. Would really appreciate if someone could help with the above. Thanks, Pinakee PS: Currently, we are using template caching by Nginx. The pages with form posts surprisingly works fine with Nginx template caching (but isn't working with Varnish). Trying to investigate why. From dridi at varni.sh Wed May 25 10:23:39 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 25 May 2016 12:23:39 +0200 Subject: CSRF token and caching In-Reply-To: <46c47618-a37b-788e-34d3-24755601f666@waltzz.com> References: <46c47618-a37b-788e-34d3-24755601f666@waltzz.com> Message-ID: On Wed, May 25, 2016 at 11:02 AM, Pinakee BIswas wrote: > Hi, > > Our backend uses CSRF tokens for form posts. For pages with forms, if > Varnish caching is enabled for such pages, form post is failing due to CSRF > error. Is there a way to handle this? I have been reading that using ESI is > a solution. > > Would really appreciate if someone could help with the above. IIUC a client may GET a page that contains tokens for a later POST, and such pages should not be cached by Varnish. The solution would then be to have your backend add Cache-Control:private when responses target specific clients. Dridi From pinakee at waltzz.com Wed May 25 10:38:12 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Wed, 25 May 2016 16:08:12 +0530 Subject: CSRF token and caching In-Reply-To: References: <46c47618-a37b-788e-34d3-24755601f666@waltzz.com> Message-ID: <7b37a9ea-fe58-935f-6f93-d934c27449c4@waltzz.com> Hi Dridi, Thanks for your response. For example, ours is an ecommerce site and this is a Product details page. This page contains the details about a product, product images (which is common for all users/visitors) and a form to add the product to cart. It would be great to cache the all the portions of the page that contains the product details (except the one with the form to add to cart). This page is a heavily visited page and hence, caching would help in improving the throughput. Thanks, Pinakee On 25/05/16 3:53 pm, Dridi Boukelmoune wrote: > On Wed, May 25, 2016 at 11:02 AM, Pinakee BIswas wrote: >> Hi, >> >> Our backend uses CSRF tokens for form posts. For pages with forms, if >> Varnish caching is enabled for such pages, form post is failing due to CSRF >> error. Is there a way to handle this? I have been reading that using ESI is >> a solution. >> >> Would really appreciate if someone could help with the above. > IIUC a client may GET a page that contains tokens for a later POST, and > such pages should not be cached by Varnish. > > The solution would then be to have your backend add > Cache-Control:private when responses target specific clients. > > Dridi From dridi at varni.sh Wed May 25 10:52:00 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 25 May 2016 12:52:00 +0200 Subject: CSRF token and caching In-Reply-To: <7b37a9ea-fe58-935f-6f93-d934c27449c4@waltzz.com> References: <46c47618-a37b-788e-34d3-24755601f666@waltzz.com> <7b37a9ea-fe58-935f-6f93-d934c27449c4@waltzz.com> Message-ID: On Wed, May 25, 2016 at 12:38 PM, Pinakee BIswas wrote: > Hi Dridi, > > Thanks for your response. > > For example, ours is an ecommerce site and this is a Product details page. > This page contains the details about a product, product images (which is > common for all users/visitors) and a form to add the product to cart. It > would be great to cache the all the portions of the page that contains the > product details (except the one with the form to add to cart). This page is > a heavily visited page and hence, caching would help in improving the > throughput. If you want to partially cache the page, then the Cache-Control solution is definitely counter-productive. I can't help you there but I can at least mention things that come to mind: - put the form in an ESI fragment - put the form in an iframe - retrieve the form using javascript Cheers From pinakee at waltzz.com Wed May 25 13:13:09 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Wed, 25 May 2016 18:43:09 +0530 Subject: CSRF token and caching In-Reply-To: References: <46c47618-a37b-788e-34d3-24755601f666@waltzz.com> <7b37a9ea-fe58-935f-6f93-d934c27449c4@waltzz.com> Message-ID: Hi Dridi, Thanks for your response. I have implemented ESI and put the form in ESI framework. Seems to work fine. Thanks, Pinakee On 25/05/16 4:22 pm, Dridi Boukelmoune wrote: > On Wed, May 25, 2016 at 12:38 PM, Pinakee BIswas wrote: >> Hi Dridi, >> >> Thanks for your response. >> >> For example, ours is an ecommerce site and this is a Product details page. >> This page contains the details about a product, product images (which is >> common for all users/visitors) and a form to add the product to cart. It >> would be great to cache the all the portions of the page that contains the >> product details (except the one with the form to add to cart). This page is >> a heavily visited page and hence, caching would help in improving the >> throughput. > If you want to partially cache the page, then the Cache-Control > solution is definitely counter-productive. > > I can't help you there but I can at least mention things that come to mind: > - put the form in an ESI fragment > - put the form in an iframe > - retrieve the form using javascript > > Cheers From albert.tollkuci at gmail.com Wed May 25 15:35:43 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Wed, 25 May 2016 17:35:43 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: Thank you Guillaume, this is OK but I'm having trouble with custom format in varnishncsa. I have it installed on Ubuntu 15.10. To setup a custom format string I have edited /etc/systemd/system/varnishncsa.service to the following: [Unit] Description=Varnish HTTP accelerator log daemon After=varnish.service [Service] User=varnishlog ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F "%h %l %u" ExecReload=/bin/kill -HUP $MAINPID [Install] WantedBy=multi-user.target However if I try to use "-F" I get an error like Job for varnishncsa.service failed because a configured resource limit was exceeded. See "systemctl status varnishncsa.service" and "journalctl -xe" for details. How can I use a cusom format string? Thanx, Albert On Tue, May 24, 2016 at 5:53 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > My bad, you don't need std.log at all (sorry, I was distracted) > > It boils down to: > > sub vcl_hit { > set req.http.x-state = "HIT"; > } > sub vcl_miss { > set req.http.x-state = "MISS"; > } > sub vcl_pass { > set req.http.x-state = "PASS"; > } > > And getting that x-state header in varnishncsa. > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Wed May 25 15:37:06 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Wed, 25 May 2016 17:37:06 +0200 Subject: W3C like format logging [WARNING: DKIM validation failed] In-Reply-To: References: Message-ID: Thank you Adam, I'm playing around with varnishncsa and it's what I need. Albert On Tue, May 24, 2016 at 4:28 PM, Adam Schumacher < adam.schumacher at flightaware.com> wrote: > varnishncsa will output logs in apache/ncsa compatible format. It also > supports the -m tag:regex filter that varnishlog does. It also provides > additional varnish-specific items including hits/misses. `man varnishnsca` > for details. > > ::Adam > > > > > > > > > On 5/24/16, 5:47 AM, "varnish-misc-bounces+adam.schumacher= > flightaware.com at varnish-cache.org on behalf of Albert Tollku?i" > on behalf of albert.tollkuci at gmail.com> wrote: > > > > >Hello, > >I'm new to varnish and I'm setting up for some of the websites I manage. > > > > > >One of the difficulties I'm having is to debug it. I know that there's > varnishlog, but it would be really helpful to have something similar to W3C > logging for all request served by varnish. In addition to standard fields, > there will be one extra field to show > > if the request was served from cache, from back-end, etc. > > > > > >Is there something like this or a way to have this kind of log? > > > > > >Thanx, > >Albert > > > > > > > > > >- > > > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Wed May 25 15:38:33 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Wed, 25 May 2016 17:38:33 +0200 Subject: Serving multiple sites from varnish In-Reply-To: References: Message-ID: Thank you Guillaume...probably the issue comes from the back-end...will look into more details. Also, sorry for the signature...fixed it now. Albert On Tue, May 24, 2016 at 9:53 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > It should indeed work, IIRC, the hash_data calls are visible in > varnishlog, you can start with that. > > On a side note, please consider trimming down your signature when you post > on public mailing-list. As a rule of thumb, your signature shouldn't be > longer than your actual message. > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Wed May 25 15:39:50 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Wed, 25 May 2016 17:39:50 +0200 Subject: VCL debugging In-Reply-To: References: Message-ID: Thank you Ross, ? std.log() is the way. Albert On Tue, May 24, 2016 at 10:21 PM, Ross Brown wrote: > Have you looked at Varnishlog? > > > > https://www.varnish-cache.org/docs/trunk/reference/varnishlog.html > > > > To add custom events, you can use the std.log function, which is part of > vmod_std. > > > > > https://www.varnish-cache.org/docs/trunk/reference/vmod_std.generated.html?#func-log > > > > import std; > > > > sub vcl_recv { > > > ?? > std.log (?Requested URL : ? + req.url); > > ? > > > > > > > > *From:* varnish-misc-bounces+ross=trademe.co.nz at varnish-cache.org [mailto: > varnish-misc-bounces+ross=trademe.co.nz at varnish-cache.org] *On Behalf Of *Albert > Tollku?i > *Sent:* Tuesday, 24 May 2016 10:51 PM > *To:* varnish-misc at varnish-cache.org > *Subject:* VCL debugging > > > > Hi everyone, > > is there a way to log some info from vcl config file? It would help a lot > to log some info on a file to check if the configuration is correct or not. > > > > Thanx, > > Albert > > > > -- > > Web: http://www.tollkuci.com > > Follow me on: [image: > http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2flinkedin_icon.png.jpgx] > LinkedIn [image: > http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2fgoogleplus-icon.png.jpgx] > Google+ [image: > http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2fFacebook_Icon.png.jpgx] > Facebook [image: > http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2fTwitterIcon.png.jpgx] > Twitter [image: > http://www.tollkuci.com/blog/IMAGES%2f2014%2f02%2fcareers-icon.png.jpgx]Career > 2.0 > ------------------------------ > > > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed May 25 15:40:42 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 25 May 2016 17:40:42 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: On Wed, May 25, 2016 at 5:35 PM, Albert Tollku?i wrote: > > Job for varnishncsa.service failed because a configured resource limit was > exceeded. See "systemctl status varnishncsa.service" and "journalctl -xe" > for details. > > Actually, showing us the details would help here, we are in the dark without them. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed May 25 15:42:51 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 25 May 2016 17:42:51 +0200 Subject: Serving multiple sites from varnish In-Reply-To: References: Message-ID: On Wed, May 25, 2016 at 5:38 PM, Albert Tollku?i wrote: > Thank you Guillaume...probably the issue comes from the back-end...will > look into more details. > If you don't find the issue, please post the relevant vcl portion on pastebin, as well as a varnishlog of two requests: one for the desktop, one for the mobile. And post the three links here. > Also, sorry for the signature...fixed it now. > > No problem. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Wed May 25 15:44:52 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Wed, 25 May 2016 17:44:52 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: Output from journalctl -xe is: May 25 15:33:29 CACHE-SRV systemd[1]: Started Varnish HTTP accelerator log daemon. -- Subject: Unit varnishncsa.service has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit varnishncsa.service has finished starting up. -- -- The start-up result is done. May 25 15:33:29 CACHE-SRV polkitd(authority=local)[799]: Unregistered Authentication Agent for unix-process:2383:95786 (system bus name :1.28, object path /org/freedesktop/ On Wed, May 25, 2016 at 5:40 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > > > On Wed, May 25, 2016 at 5:35 PM, Albert Tollku?i < > albert.tollkuci at gmail.com> wrote: > >> >> Job for varnishncsa.service failed because a configured resource limit >> was exceeded. See "systemctl status varnishncsa.service" and "journalctl >> -xe" for details. >> >> > Actually, showing us the details would help here, we are in the dark > without them. > > -- > Guillaume Quintard > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Wed May 25 16:41:55 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Wed, 25 May 2016 18:41:55 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: Tried even the solution posted here https://blog.onetechnical.com/2014/05/02/hacking-up-varnish-logging-on-ubuntu-12-04/ by modifying /etc/init.d/varnishncsa but still log format stays the same... On Wed, May 25, 2016 at 5:44 PM, Albert Tollku?i wrote: > Output from journalctl -xe is: > > May 25 15:33:29 CACHE-SRV systemd[1]: Started Varnish HTTP accelerator log > daemon. > -- Subject: Unit varnishncsa.service has finished start-up > -- Defined-By: systemd > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel > -- > -- Unit varnishncsa.service has finished starting up. > -- > -- The start-up result is done. > May 25 15:33:29 CACHE-SRV polkitd(authority=local)[799]: Unregistered > Authentication Agent for unix-process:2383:95786 (system bus name :1.28, > object path /org/freedesktop/ > > > > > On Wed, May 25, 2016 at 5:40 PM, Guillaume Quintard < > guillaume at varnish-software.com> wrote: > >> >> >> On Wed, May 25, 2016 at 5:35 PM, Albert Tollku?i < >> albert.tollkuci at gmail.com> wrote: >> >>> >>> Job for varnishncsa.service failed because a configured resource limit >>> was exceeded. See "systemctl status varnishncsa.service" and "journalctl >>> -xe" for details. >>> >>> >> Actually, showing us the details would help here, we are in the dark >> without them. >> >> -- >> Guillaume Quintard >> > > > > -- > Web: http://www.tollkuci.com > Follow me on: LinkedIn Google+ > Facebook > Twitter > Career 2.0 > > ------------------------------ > > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Thu May 26 12:54:48 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Thu, 26 May 2016 18:24:48 +0530 Subject: Strange Issue with cache and logged in users Message-ID: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> Hi, I am facing a strange problem with Varnish cache. Ours is an ecommerce site wherein users can visit with or without logging in. I have written the VCL to unset cookies when user not logged in. The scenario is user logs in and then logs out. When the user logs out, Varnish is still delivering logged in pages to the user which is strange as the requests for logged in users go all the way to the backend and the pages are not supposed to get cached. I have been looking into this since quite some time and still not able to figure out the reason. Would appreciate any help. Thanks, Pinakee PFA the output from varnishlog. Following is the VCL code: sub vcl_recv { # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you don't need, # rewriting the request, etc. set req.backend_hint = uwsgi; if (req.method == "GET") { if ((req.http.Cookie !~ "sessionid=") && (req.http.Cookie !~ "loggedin_user=") && (req.url !~ "^/esi")) { unset req.http.cookie; # strip the cookies - we don't need them } } } sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. if (bereq.method == "GET") { set beresp.do_esi = true; if ((beresp.http.Set-Cookie !~ "jivaana_user=") && (beresp.http.Set-Cookie !~ "sessionid=") && (bereq.http.Cookie !~ "sessionid=") && (bereq.http.Cookie !~ "jivaana_user=") && (bereq.url !~ "^/product/addtobasket")) { #unset beresp.http.Set-Cookie; set beresp.uncacheable = false; return(deliver); } } } -------------- next part -------------- * << Request >> 8 - Begin req 7 rxreq - Timestamp Start: 1464266702.728722 0.000000 0.000000 - Timestamp Req: 1464266702.728722 0.000000 0.000000 - ReqStart 127.0.0.1 42902 - ReqMethod GET - ReqURL /clothing/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/juttis/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - ReqUnset Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Encoding: gzip - VCL_call HASH - VCL_return lookup - Hit 6 - VCL_call HIT - VCL_return deliver - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 334932 - RespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 12:44:46 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; expires=Thu, 25-May-2017 12:44:46 GMT; Max-Age=31449600; Path=/ - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespHeader X-Varnish: 8 6 - RespHeader Age: 16 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464266702.728845 0.000123 0.000123 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464266702.729090 0.000368 0.000245 - ReqAcct 1179 0 1179 624 334932 335556 - End * << Session >> 7 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 42902 0.0.0.0:9003 127.0.0.1 9003 1464266702.728659 19 - Link req 8 rxreq - SessClose RESP_CLOSE 0.001 - End * << BeReq >> 32774 - Begin bereq 32773 pass - Timestamp Start: 1464266810.235816 0.000000 0.000000 - BereqMethod GET - BereqURL /clothing/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Cache-Control: max-age=0 - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/clothing/ - BereqHeader Accept-Encoding: gzip, deflate, sdch - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader X-Varnish: 32774 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57250 - Timestamp Bereq: 1464266810.235999 0.000184 0.000184 - Timestamp Beresp: 1464266810.324350 0.088535 0.088351 - BerespProtocol HTTP/1.0 - BerespStatus 200 - BerespReason OK - BerespHeader Content-Length: 334932 - BerespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 12:46:50 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; expires=Thu, 25-May-2017 12:46:50 GMT; Max-Age=31449600; Path=/ - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - BerespHeader Set-Cookie: sessionid=b8sozfjdp41xudva0yxhqdca880dcqsg; expires=Sat, 02-May-2116 12:46:50 GMT; httponly; Max-Age=3153600000; Path=/ - TTL RFC 300 10 -1 1464266810 1464266810 1464266810 1464266986 300 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1464266810 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.0 - ObjStatus 200 - ObjReason OK - ObjHeader Content-Length: 334932 - ObjHeader Expires: Thu, 26 May 2016 12:49:46 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 12:46:50 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; expires=Thu, 25-May-2017 12:46:50 GMT; Max-Age=31449600; Path=/ - ObjHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - ObjHeader Set-Cookie: sessionid=b8sozfjdp41xudva0yxhqdca880dcqsg; expires=Sat, 02-May-2116 12:46:50 GMT; httponly; Max-Age=3153600000; Path=/ - Fetch_Body 3 length stream - BackendClose 21 boot.uwsgi - Timestamp BerespBody: 1464266810.326406 0.090590 0.002055 - Length 334932 - BereqAcct 1385 0 1385 671 334932 335603 - End * << Request >> 32773 - Begin req 32772 rxreq - Timestamp Start: 1464266810.235653 0.000000 0.000000 - Timestamp Req: 1464266810.235653 0.000000 0.000000 - ReqStart 127.0.0.1 42905 - ReqMethod GET - ReqURL /clothing/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/clothing/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 32774 pass - Timestamp Fetch: 1464266810.324477 0.088825 0.088825 - RespProtocol HTTP/1.0 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 334932 - RespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 12:46:50 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; expires=Thu, 25-May-2017 12:46:50 GMT; Max-Age=31449600; Path=/ - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespHeader Set-Cookie: sessionid=b8sozfjdp41xudva0yxhqdca880dcqsg; expires=Sat, 02-May-2116 12:46:50 GMT; httponly; Max-Age=3153600000; Path=/ - RespProtocol HTTP/1.1 - RespHeader X-Varnish: 32773 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464266810.324504 0.088851 0.000026 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464266810.330895 0.095242 0.006391 - ReqAcct 1358 0 1358 759 334932 335691 - End * << Session >> 32772 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 42905 0.0.0.0:9003 127.0.0.1 9003 1464266810.235566 18 - Link req 32773 rxreq - SessClose RESP_CLOSE 0.096 - End * << BeReq >> 11 - Begin bereq 10 pass - Timestamp Start: 1464266840.814078 0.000000 0.000000 - BereqMethod GET - BereqURL /clothing/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Cache-Control: max-age=0 - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/clothing/ - BereqHeader Accept-Encoding: gzip, deflate, sdch - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader X-Varnish: 11 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57255 - Timestamp Bereq: 1464266840.819873 0.005795 0.005795 - Timestamp Beresp: 1464266840.850814 0.036736 0.030941 - BerespProtocol HTTP/1.0 - BerespStatus 200 - BerespReason OK - BerespHeader Content-Length: 334932 - BerespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 12:47:20 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - TTL RFC 300 10 -1 1464266841 1464266841 1464266840 1464266986 300 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1464266841 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.0 - ObjStatus 200 - ObjReason OK - ObjHeader Content-Length: 334932 - ObjHeader Expires: Thu, 26 May 2016 12:49:46 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 12:47:20 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - Fetch_Body 3 length stream - BackendClose 21 boot.uwsgi - Timestamp BerespBody: 1464266840.852163 0.038085 0.001349 - Length 334932 - BereqAcct 1426 0 1426 415 334932 335347 - End * << Request >> 10 - Begin req 9 rxreq - Timestamp Start: 1464266840.813811 0.000000 0.000000 - Timestamp Req: 1464266840.813811 0.000000 0.000000 - ReqStart 127.0.0.1 42910 - ReqMethod GET - ReqURL /clothing/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/clothing/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 11 pass - Timestamp Fetch: 1464266840.851103 0.037292 0.037292 - RespProtocol HTTP/1.0 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 334932 - RespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 12:47:20 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespProtocol HTTP/1.1 - RespHeader X-Varnish: 10 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464266840.851142 0.037330 0.000039 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464266840.852340 0.038528 0.001198 - ReqAcct 1402 0 1402 500 334932 335432 - End * << Session >> 9 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 42910 0.0.0.0:9003 127.0.0.1 9003 1464266840.813701 18 - Link req 10 rxreq - SessClose RESP_CLOSE 0.039 - End * << Request >> 32776 - Begin req 32775 rxreq - Timestamp Start: 1464266845.838164 0.000000 0.000000 - Timestamp Req: 1464266845.838164 0.000000 0.000000 - ReqStart 127.0.0.1 42912 - ReqMethod GET - ReqURL /juttis/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/clothing/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 32777 pass - Timestamp Fetch: 1464266846.721065 0.882900 0.882900 - RespProtocol HTTP/1.0 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 320382 - RespHeader Expires: Thu, 26 May 2016 12:52:26 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 12:47:26 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; expires=Thu, 25-May-2017 12:47:26 GMT; Max-Age=31449600; Path=/ - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespProtocol HTTP/1.1 - RespHeader X-Varnish: 32776 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464266846.721142 0.882978 0.000077 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464266846.727199 0.889035 0.006057 - ReqAcct 1374 0 1374 625 320382 321007 - End * << Session >> 32775 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 42912 0.0.0.0:9003 127.0.0.1 9003 1464266845.838071 18 - Link req 32776 rxreq - SessClose RESP_CLOSE 0.889 - End * << BeReq >> 32777 - Begin bereq 32776 pass - Timestamp Start: 1464266845.838299 0.000000 0.000000 - BereqMethod GET - BereqURL /juttis/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/clothing/ - BereqHeader Accept-Encoding: gzip, deflate, sdch - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader X-Varnish: 32777 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57257 - Timestamp Bereq: 1464266845.838568 0.000269 0.000269 - Timestamp Beresp: 1464266846.720820 0.882521 0.882252 - BerespProtocol HTTP/1.0 - BerespStatus 200 - BerespReason OK - BerespHeader Content-Length: 320382 - BerespHeader Expires: Thu, 26 May 2016 12:52:26 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 12:47:26 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; expires=Thu, 25-May-2017 12:47:26 GMT; Max-Age=31449600; Path=/ - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - TTL RFC 300 10 -1 1464266847 1464266847 1464266846 1464267146 300 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1464266847 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.0 - ObjStatus 200 - ObjReason OK - ObjHeader Content-Length: 320382 - ObjHeader Expires: Thu, 26 May 2016 12:52:26 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 12:47:26 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; expires=Thu, 25-May-2017 12:47:26 GMT; Max-Age=31449600; Path=/ - ObjHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - Fetch_Body 3 length stream - BackendClose 21 boot.uwsgi - Timestamp BerespBody: 1464266846.727478 0.889179 0.006658 - Length 320382 - BereqAcct 1401 0 1401 537 320382 320919 - End * << BeReq >> 14 - Begin bereq 13 pass - Timestamp Start: 1464266850.003886 0.000000 0.000000 - BereqMethod GET - BereqURL /clothing/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/clothing/ - BereqHeader Accept-Encoding: gzip, deflate, sdch - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader X-Varnish: 14 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57260 - Timestamp Bereq: 1464266850.004331 0.000445 0.000445 - Timestamp Beresp: 1464266850.042777 0.038891 0.038445 - BerespProtocol HTTP/1.0 - BerespStatus 200 - BerespReason OK - BerespHeader Content-Length: 334932 - BerespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 12:47:30 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - TTL RFC 300 10 -1 1464266850 1464266850 1464266850 1464266986 300 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1464266850 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.0 - ObjStatus 200 - ObjReason OK - ObjHeader Content-Length: 334932 - ObjHeader Expires: Thu, 26 May 2016 12:49:46 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 12:47:30 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - Fetch_Body 3 length stream - BackendClose 21 boot.uwsgi - Timestamp BerespBody: 1464266850.043962 0.040076 0.001185 - Length 334932 - BereqAcct 1400 0 1400 415 334932 335347 - End * << Request >> 13 - Begin req 12 rxreq - Timestamp Start: 1464266850.003736 0.000000 0.000000 - Timestamp Req: 1464266850.003736 0.000000 0.000000 - ReqStart 127.0.0.1 42915 - ReqMethod GET - ReqURL /clothing/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/clothing/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 14 pass - Timestamp Fetch: 1464266850.043010 0.039274 0.039274 - RespProtocol HTTP/1.0 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 334932 - RespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 12:47:30 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespProtocol HTTP/1.1 - RespHeader X-Varnish: 13 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464266850.043042 0.039306 0.000032 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464266850.044136 0.040400 0.001094 - ReqAcct 1376 0 1376 500 334932 335432 - End * << Session >> 12 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 42915 0.0.0.0:9003 127.0.0.1 9003 1464266850.003624 19 - Link req 13 rxreq - SessClose RESP_CLOSE 0.041 - End * << BeReq >> 17 - Begin bereq 16 pass - Timestamp Start: 1464266852.268802 0.000000 0.000000 - BereqMethod GET - BereqURL /clothing/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Cache-Control: max-age=0 - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/clothing/ - BereqHeader Accept-Encoding: gzip, deflate, sdch - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - BereqHeader If-Modified-Since: Thu, 26 May 2016 12:44:46 GMT - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader X-Varnish: 17 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57262 - Timestamp Bereq: 1464266852.269130 0.000327 0.000327 - Timestamp Beresp: 1464266852.286691 0.017889 0.017561 - BerespProtocol HTTP/1.0 - BerespStatus 304 - BerespReason OK - BerespHeader Content-Length: 0 - BerespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 12:47:32 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - TTL RFC 300 10 -1 1464266852 1464266852 1464266852 1464266986 300 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1464266852 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.0 - ObjStatus 304 - ObjReason OK - ObjHeader Content-Length: 0 - ObjHeader Expires: Thu, 26 May 2016 12:49:46 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 12:47:32 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - Fetch_Body 0 none - - BackendClose 21 boot.uwsgi - Timestamp BerespBody: 1464266852.286892 0.018089 0.000201 - Length 0 - BereqAcct 1476 0 1476 410 0 410 - End * << Request >> 16 - Begin req 15 rxreq - Timestamp Start: 1464266852.268643 0.000000 0.000000 - Timestamp Req: 1464266852.268643 0.000000 0.000000 - ReqStart 127.0.0.1 42917 - ReqMethod GET - ReqURL /clothing/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/clothing/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - ReqHeader If-Modified-Since: Thu, 26 May 2016 12:44:46 GMT - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 17 pass - Timestamp Fetch: 1464266852.286993 0.018349 0.018349 - RespProtocol HTTP/1.0 - RespStatus 304 - RespReason OK - RespHeader Content-Length: 0 - RespHeader Expires: Thu, 26 May 2016 12:49:46 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 12:44:46 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 12:47:32 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespProtocol HTTP/1.1 - RespHeader X-Varnish: 16 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464266852.287160 0.018517 0.000167 - RespUnset Content-Length: 0 - Debug "RES_MODE 0" - RespHeader Connection: close - Timestamp Resp: 1464266852.287240 0.018597 0.000080 - ReqAcct 1452 0 1452 454 0 454 - End * << Session >> 15 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 42917 0.0.0.0:9003 127.0.0.1 9003 1464266852.268507 18 - Link req 16 rxreq - SessClose RESP_CLOSE 0.019 - End * << BeReq >> 65539 - Begin bereq 65538 pass - Timestamp Start: 1464266855.401311 0.000000 0.000000 - BereqMethod GET - BereqURL /juttis/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/clothing/ - BereqHeader Accept-Encoding: gzip, deflate, sdch - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader X-Varnish: 65539 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57264 - Timestamp Bereq: 1464266855.401576 0.000265 0.000265 - Timestamp Beresp: 1464266855.439149 0.037838 0.037573 - BerespProtocol HTTP/1.0 - BerespStatus 200 - BerespReason OK - BerespHeader Content-Length: 320382 - BerespHeader Expires: Thu, 26 May 2016 12:52:26 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 12:47:35 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - TTL RFC 300 10 -1 1464266855 1464266855 1464266855 1464267146 300 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1464266855 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.0 - ObjStatus 200 - ObjReason OK - ObjHeader Content-Length: 320382 - ObjHeader Expires: Thu, 26 May 2016 12:52:26 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 12:47:35 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - Fetch_Body 3 length stream - BackendClose 21 boot.uwsgi - Timestamp BerespBody: 1464266855.439849 0.038538 0.000700 - Length 320382 - BereqAcct 1401 0 1401 415 320382 320797 - End * << Request >> 65538 - Begin req 65537 rxreq - Timestamp Start: 1464266855.401159 0.000000 0.000000 - Timestamp Req: 1464266855.401159 0.000000 0.000000 - ReqStart 127.0.0.1 42919 - ReqMethod GET - ReqURL /juttis/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/clothing/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-151 - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 65539 pass - Timestamp Fetch: 1464266855.439933 0.038774 0.038774 - RespProtocol HTTP/1.0 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 320382 - RespHeader Expires: Thu, 26 May 2016 12:52:26 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 12:47:35 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespProtocol HTTP/1.1 - RespHeader X-Varnish: 65538 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464266855.439978 0.038819 0.000045 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464266855.440157 0.038998 0.000179 - ReqAcct 1374 0 1374 503 320382 320885 - End * << Session >> 65537 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 42919 0.0.0.0:9003 127.0.0.1 9003 1464266855.401067 19 - Link req 65538 rxreq - SessClose RESP_CLOSE 0.039 - End * << BeReq >> 65542 - Begin bereq 65541 fetch - Timestamp Start: 1464266862.312386 0.000000 0.000000 - BereqMethod GET - BereqURL /juttis/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/home - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader Accept-Encoding: gzip - BereqProtocol HTTP/1.1 - BereqHeader X-Varnish: 65542 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57270 - Timestamp Bereq: 1464266862.312759 0.000373 0.000373 - Timestamp Beresp: 1464266862.330167 0.017781 0.017408 - BerespProtocol HTTP/1.1 - BerespStatus 200 - BerespReason OK - BerespHeader Content-Length: 320382 - BerespHeader Expires: Thu, 26 May 2016 12:52:26 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 12:47:42 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - TTL RFC 300 10 -1 1464266862 1464266862 1464266862 1464267146 300 - VCL_call BACKEND_RESPONSE - VCL_return deliver - Storage file s0 - ObjProtocol HTTP/1.1 - ObjStatus 200 - ObjReason OK - ObjHeader Content-Length: 320382 - ObjHeader Expires: Thu, 26 May 2016 12:52:26 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 12:47:42 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - Fetch_Body 3 length stream - BackendReuse 21 boot.uwsgi - Timestamp BerespBody: 1464266862.332980 0.020595 0.002813 - Length 320382 - BereqAcct 458 0 458 415 320382 320797 - End * << Request >> 65541 - Begin req 65540 rxreq - Timestamp Start: 1464266862.312107 0.000000 0.000000 - Timestamp Req: 1464266862.312107 0.000000 0.000000 - ReqStart 127.0.0.1 42925 - ReqMethod GET - ReqURL /juttis/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/home - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; _gat=1; csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; _ga=GA1.2.886266615.1446454138; mp_774636c1ed2371eaf99455f718710 - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - ReqUnset Cookie: pagemap=0,0,0,0,0,1,0'; _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; _gat=1; csrftoken=WPe3VfvSNhL1bwf6bcgopNaGzK02Y9Bp; _ga=GA1.2.886266615.1446454138; mp_774636c1ed2371eaf99455f718710 - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Encoding: gzip - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 65542 fetch - Timestamp Fetch: 1464266862.330719 0.018612 0.018612 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 320382 - RespHeader Expires: Thu, 26 May 2016 12:52:26 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 12:47:26 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 12:47:42 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespHeader X-Varnish: 65541 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464266862.330795 0.018688 0.000076 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464266862.334010 0.021903 0.003215 - ReqAcct 1148 0 1148 503 320382 320885 - End * << Session >> 65540 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 42925 0.0.0.0:9003 127.0.0.1 9003 1464266862.311702 18 - Link req 65541 rxreq - SessClose RESP_CLOSE 0.022 - End From pinakee at waltzz.com Thu May 26 14:07:02 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Thu, 26 May 2016 19:37:02 +0530 Subject: Strange Issue with cache and logged in users In-Reply-To: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> References: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> Message-ID: <373a8b4a-1fac-d8ee-17c1-2ccc8dd9ae48@waltzz.com> Further adding the observations with the following scenario: 1. Visit URL A-not-loggedin 2. Visit URL B-not-loggedin 3. User logs in 4. Visit URL A after logging in - varnish is serving the page - A-not-loggedin 5. Visit URL B after logging in - varnish is serving the page B-not-loggedin This means Varnish is serving the pages just visited (till cache expiry) irrespective of the fact that the page is being fetched from backend and that there is a pass in vcl_recv. I am unable to understand the above mentioned behavior - the VCL logic I used is simple as mentioned in the mail below. Regards, Pinakee On 26/05/16 6:24 pm, Pinakee BIswas wrote: > Hi, > > I am facing a strange problem with Varnish cache. Ours is an ecommerce > site wherein users can visit with or without logging in. I have > written the VCL to unset cookies when user not logged in. > > The scenario is user logs in and then logs out. When the user logs > out, Varnish is still delivering logged in pages to the user which is > strange as the requests for logged in users go all the way to the > backend and the pages are not supposed to get cached. I have been > looking into this since quite some time and still not able to figure > out the reason. > > Would appreciate any help. > > Thanks, > > Pinakee > > PFA the output from varnishlog. > > Following is the VCL code: > > sub vcl_recv { > # Happens before we check if we have this in cache already. > # > # Typically you clean up the request here, removing cookies you > don't need, > # rewriting the request, etc. > set req.backend_hint = uwsgi; > > if (req.method == "GET") { > if ((req.http.Cookie !~ "sessionid=") && > (req.http.Cookie !~ "loggedin_user=") && > (req.url !~ "^/esi")) { > unset req.http.cookie; # strip the cookies - we don't > need them > } > } > } > > sub vcl_backend_response { > # Happens after we have read the response headers from the backend. > # > # Here you clean the response headers, removing silly Set-Cookie > headers > # and other mistakes your backend does. > if (bereq.method == "GET") { > set beresp.do_esi = true; > if ((beresp.http.Set-Cookie !~ "jivaana_user=") && > (beresp.http.Set-Cookie !~ "sessionid=") && > (bereq.http.Cookie !~ "sessionid=") && > (bereq.http.Cookie !~ "jivaana_user=") && > (bereq.url !~ "^/product/addtobasket")) { > #unset beresp.http.Set-Cookie; > set beresp.uncacheable = false; > return(deliver); > } > } > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Thu May 26 14:28:27 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 26 May 2016 16:28:27 +0200 Subject: Strange Issue with cache and logged in users In-Reply-To: <373a8b4a-1fac-d8ee-17c1-2ccc8dd9ae48@waltzz.com> References: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> <373a8b4a-1fac-d8ee-17c1-2ccc8dd9ae48@waltzz.com> Message-ID: Are you able to isolate the faulty request in varnishlog? The one that should be passed and isn't? -- Guillaume Quintard On Thu, May 26, 2016 at 4:07 PM, Pinakee BIswas wrote: > Further adding the observations with the following scenario: > > 1. Visit URL A-not-loggedin > 2. Visit URL B-not-loggedin > 3. User logs in > 4. Visit URL A after logging in - varnish is serving the page - > A-not-loggedin > 5. Visit URL B after logging in - varnish is serving the page > B-not-loggedin > > This means Varnish is serving the pages just visited (till cache expiry) > irrespective of the fact that the page is being fetched from backend and > that there is a pass in vcl_recv. > > I am unable to understand the above mentioned behavior - the VCL logic I > used is simple as mentioned in the mail below. > > Regards, > > Pinakee > > On 26/05/16 6:24 pm, Pinakee BIswas wrote: > > Hi, > > I am facing a strange problem with Varnish cache. Ours is an ecommerce > site wherein users can visit with or without logging in. I have written the > VCL to unset cookies when user not logged in. > > The scenario is user logs in and then logs out. When the user logs out, > Varnish is still delivering logged in pages to the user which is strange as > the requests for logged in users go all the way to the backend and the > pages are not supposed to get cached. I have been looking into this since > quite some time and still not able to figure out the reason. > > Would appreciate any help. > > Thanks, > > Pinakee > > PFA the output from varnishlog. > > Following is the VCL code: > > sub vcl_recv { > # Happens before we check if we have this in cache already. > # > # Typically you clean up the request here, removing cookies you don't > need, > # rewriting the request, etc. > set req.backend_hint = uwsgi; > > if (req.method == "GET") { > if ((req.http.Cookie !~ "sessionid=") && > (req.http.Cookie !~ "loggedin_user=") && > (req.url !~ "^/esi")) { > unset req.http.cookie; # strip the cookies - we don't need > them > } > } > } > > sub vcl_backend_response { > # Happens after we have read the response headers from the backend. > # > # Here you clean the response headers, removing silly Set-Cookie > headers > # and other mistakes your backend does. > if (bereq.method == "GET") { > set beresp.do_esi = true; > if ((beresp.http.Set-Cookie !~ "jivaana_user=") && > (beresp.http.Set-Cookie !~ "sessionid=") && > (bereq.http.Cookie !~ "sessionid=") && > (bereq.http.Cookie !~ "jivaana_user=") && > (bereq.url !~ "^/product/addtobasket")) { > #unset beresp.http.Set-Cookie; > set beresp.uncacheable = false; > return(deliver); > } > } > } > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Thu May 26 14:40:19 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Thu, 26 May 2016 20:10:19 +0530 Subject: Strange Issue with cache and logged in users In-Reply-To: References: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> <373a8b4a-1fac-d8ee-17c1-2ccc8dd9ae48@waltzz.com> Message-ID: <48b993e0-827c-4bfb-e5b7-b1ef7ce6dd08@waltzz.com> Hi Guillaume, Thanks for your response. I did some more testing and it seems the pages are somehow getting cached irrespective of whatever logic I have put in the vcl_backend_response: sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. if (bereq.method == "GET") { set beresp.do_esi = true; if ((beresp.http.Set-Cookie !~ "loggedin_user=") && (beresp.http.Set-Cookie !~ "sessionid=") && (bereq.http.Cookie !~ "sessionid=") && (bereq.http.Cookie !~ "loggedin_user=") && (bereq.url !~ "^/esi")) { #unset beresp.http.Set-Cookie; set beresp.uncacheable = false; return(deliver); } } } Regarding the faulty request, Following are two requests for the same URL - /heels/ - one when the user was logged in and another when the user had logged out. The second one (when the user was logged out) is the faulty one as the user is still seeing the data when he was logged in. *_For the logged in User:_* * << BeReq >> 65542 - Begin bereq 65541 pass - Timestamp Start: 1464273298.494530 0.000000 0.000000 - BereqMethod GET - BereqURL /heels/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Cache-Control: max-age=0 - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/juttis/ - BereqHeader Accept-Encoding: gzip, deflate, sdch - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader Cookie: _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-1518b8ba6c428a%22%2C%22%24i - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader X-Varnish: 65542 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57783 - Timestamp Bereq: 1464273298.494747 0.000217 0.000217 - Timestamp Beresp: 1464273298.507940 0.013410 0.013193 - BerespProtocol HTTP/1.0 - BerespStatus 200 - BerespReason OK - BerespHeader Content-Length: 257701 - BerespHeader Expires: Thu, 26 May 2016 14:39:55 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 14:34:55 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 14:34:58 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - TTL RFC 300 10 -1 1464273299 1464273299 1464273298 1464273595 300 - VCL_call BACKEND_RESPONSE - TTL VCL 120 10 0 1464273299 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.0 - ObjStatus 200 - ObjReason OK - ObjHeader Content-Length: 257701 - ObjHeader Expires: Thu, 26 May 2016 14:39:55 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 14:34:55 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 14:34:58 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - Fetch_Body 3 length stream - BackendClose 21 boot.uwsgi - Timestamp BerespBody: 1464273298.512453 0.017923 0.004513 - Length 257701 - BereqAcct 1396 0 1396 415 257701 258116 - End * << Request >> 65541 - Begin req 65540 rxreq - Timestamp Start: 1464273298.494400 0.000000 0.000000 - Timestamp Req: 1464273298.494400 0.000000 0.000000 - ReqStart 127.0.0.1 43438 - ReqMethod GET - ReqURL /heels/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/juttis/ - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-1518b8ba6c428a%22%2C%22%24i - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 65542 pass - Timestamp Fetch: 1464273298.508162 0.013762 0.013762 - RespProtocol HTTP/1.0 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 257701 - RespHeader Expires: Thu, 26 May 2016 14:39:55 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 14:34:55 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 14:34:58 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - RespProtocol HTTP/1.1 - RespHeader X-Varnish: 65541 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464273298.508196 0.013796 0.000034 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464273298.512479 0.018080 0.004284 - ReqAcct 1369 0 1369 503 257701 258204 - End * << Session >> 65540 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 43438 0.0.0.0:9003 127.0.0.1 9003 1464273298.494343 20 - Link req 65541 rxreq - SessClose RESP_CLOSE 0.018 - End _*For the logged out user:*_ * << BeReq >> 32788 - Begin bereq 32787 fetch - Timestamp Start: 1464273406.124375 0.000000 0.000000 - BereqMethod GET - BereqURL /heels/ - BereqProtocol HTTP/1.0 - BereqHeader X-CLIENT-COUNTRY: IN - BereqHeader Host: varnish_staging - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - BereqHeader Referer: http://test.jivaana.in/home - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader X-Forwarded-For: 127.0.0.1 - BereqHeader Accept-Encoding: gzip - BereqProtocol HTTP/1.1 - BereqHeader X-Varnish: 32788 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 21 boot.uwsgi 127.0.0.1 7090 127.0.0.1 57794 - Timestamp Bereq: 1464273406.124744 0.000369 0.000369 - Timestamp Beresp: 1464273406.137721 0.013346 0.012977 - BerespProtocol HTTP/1.1 - BerespStatus 200 - BerespReason OK - BerespHeader Content-Length: 257701 - BerespHeader Expires: Thu, 26 May 2016 14:39:55 GMT - BerespHeader Vary: X-CLIENT-COUNTRY, Cookie - BerespHeader X-CLIENT-COUNTRY: IN - BerespHeader Last-Modified: Thu, 26 May 2016 14:34:55 GMT - BerespHeader Cache-Control: max-age=300 - BerespHeader Date: Thu, 26 May 2016 14:36:46 GMT - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - TTL RFC 300 10 -1 1464273406 1464273406 1464273406 1464273595 300 - VCL_call BACKEND_RESPONSE - BerespUnset Set-Cookie: jivaana_num_basket_items=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/ - VCL_Log Caching the url : **********************/heels/ - VCL_return deliver - Storage file s0 - ObjProtocol HTTP/1.1 - ObjStatus 200 - ObjReason OK - ObjHeader Content-Length: 257701 - ObjHeader Expires: Thu, 26 May 2016 14:39:55 GMT - ObjHeader Vary: X-CLIENT-COUNTRY, Cookie - ObjHeader X-CLIENT-COUNTRY: IN - ObjHeader Last-Modified: Thu, 26 May 2016 14:34:55 GMT - ObjHeader Cache-Control: max-age=300 - ObjHeader Date: Thu, 26 May 2016 14:36:46 GMT - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Content-Type: text/html; charset=utf-8 - Fetch_Body 3 length stream - BackendReuse 21 boot.uwsgi - Timestamp BerespBody: 1464273406.139210 0.014835 0.001489 - Length 257701 - BereqAcct 457 0 457 415 257701 258116 - End * << Request >> 32787 - Begin req 32786 rxreq - Timestamp Start: 1464273406.124217 0.000000 0.000000 - Timestamp Req: 1464273406.124217 0.000000 0.000000 - ReqStart 127.0.0.1 43449 - ReqMethod GET - ReqURL /heels/ - ReqProtocol HTTP/1.0 - ReqHeader X-CLIENT-COUNTRY: IN - ReqHeader Host: varnish_staging - ReqHeader Connection: close - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 - ReqHeader Referer: http://test.jivaana.in/home - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; _gat=1; csrftoken=PFm9lTAeFcQLIHY2UsUjkuO73R1GVaxn; _ga=GA1.2.886266615.1446454138; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22disti - ReqHeader X-Forwarded-For: 127.0.0.1 - VCL_call RECV - ReqUnset Cookie: _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; _gat=1; csrftoken=PFm9lTAeFcQLIHY2UsUjkuO73R1GVaxn; _ga=GA1.2.886266615.1446454138; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22disti - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Encoding: gzip - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 32788 fetch - Timestamp Fetch: 1464273406.138165 0.013947 0.013947 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Content-Length: 257701 - RespHeader Expires: Thu, 26 May 2016 14:39:55 GMT - RespHeader Vary: X-CLIENT-COUNTRY, Cookie - RespHeader X-CLIENT-COUNTRY: IN - RespHeader Last-Modified: Thu, 26 May 2016 14:34:55 GMT - RespHeader Cache-Control: max-age=300 - RespHeader Date: Thu, 26 May 2016 14:36:46 GMT - RespHeader X-Frame-Options: SAMEORIGIN - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader X-Varnish: 32787 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1464273406.138211 0.013993 0.000046 - RespHeader Accept-Ranges: bytes - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1464273406.139451 0.015234 0.001240 - ReqAcct 1119 0 1119 405 257701 258106 - End * << Session >> 32786 - Begin sess 0 HTTP/1 - SessOpen 127.0.0.1 43449 0.0.0.0:9003 127.0.0.1 9003 1464273406.124096 16 - Link req 32787 rxreq - SessClose RESP_CLOSE 0.016 - End Thanks, Pinakee On 26/05/16 7:58 pm, Guillaume Quintard wrote: > Are you able to isolate the faulty request in varnishlog? The one that > should be passed and isn't? > > -- > Guillaume Quintard > > On Thu, May 26, 2016 at 4:07 PM, Pinakee BIswas > wrote: > > Further adding the observations with the following scenario: > > 1. Visit URL A-not-loggedin > 2. Visit URL B-not-loggedin > 3. User logs in > 4. Visit URL A after logging in - varnish is serving the page - > A-not-loggedin > 5. Visit URL B after logging in - varnish is serving the page > B-not-loggedin > > This means Varnish is serving the pages just visited (till cache > expiry) irrespective of the fact that the page is being fetched > from backend and that there is a pass in vcl_recv. > > I am unable to understand the above mentioned behavior - the VCL > logic I used is simple as mentioned in the mail below. > > Regards, > > Pinakee > > > On 26/05/16 6:24 pm, Pinakee BIswas wrote: >> Hi, >> >> I am facing a strange problem with Varnish cache. Ours is an >> ecommerce site wherein users can visit with or without logging >> in. I have written the VCL to unset cookies when user not logged in. >> >> The scenario is user logs in and then logs out. When the user >> logs out, Varnish is still delivering logged in pages to the user >> which is strange as the requests for logged in users go all the >> way to the backend and the pages are not supposed to get cached. >> I have been looking into this since quite some time and still not >> able to figure out the reason. >> >> Would appreciate any help. >> >> Thanks, >> >> Pinakee >> >> PFA the output from varnishlog. >> >> Following is the VCL code: >> >> sub vcl_recv { >> # Happens before we check if we have this in cache already. >> # >> # Typically you clean up the request here, removing cookies >> you don't need, >> # rewriting the request, etc. >> set req.backend_hint = uwsgi; >> >> if (req.method == "GET") { >> if ((req.http.Cookie !~ "sessionid=") && >> (req.http.Cookie !~ "loggedin_user=") && >> (req.url !~ "^/esi")) { >> unset req.http.cookie; # strip the cookies - we >> don't need them >> } >> } >> } >> >> sub vcl_backend_response { >> # Happens after we have read the response headers from the >> backend. >> # >> # Here you clean the response headers, removing silly >> Set-Cookie headers >> # and other mistakes your backend does. >> if (bereq.method == "GET") { >> set beresp.do_esi = true; >> if ((beresp.http.Set-Cookie !~ "jivaana_user=") && >> (beresp.http.Set-Cookie !~ "sessionid=") && >> (bereq.http.Cookie !~ "sessionid=") && >> (bereq.http.Cookie !~ "jivaana_user=") && >> (bereq.url !~ "^/product/addtobasket")) { >> #unset beresp.http.Set-Cookie; >> set beresp.uncacheable = false; >> return(deliver); >> } >> } >> } >> > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Thu May 26 14:40:31 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Thu, 26 May 2016 16:40:31 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: Still no solution for this...If i manually run /usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F "%h|%u|%{%Y-%m-%d}t" I get the correct log format. However when I put this on /etc/systemd/system/varnishncsa.service, such as: [Unit] Description=Varnish HTTP accelerator log daemon After=varnish.service [Service] User=varnishlog ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F "%h|%u|%{%Y-%m-%d}t" ExecReload=/bin/kill -HUP $MAINPID [Install] WantedBy=multi-user.target I get an error May 26 14:39:57 CACHE-SRV systemd[1]: Failed to start Varnish HTTP accelerator log daemon. -- Subject: Unit varnishncsa.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit varnishncsa.service has failed. -- -- The result is failed. May 26 14:39:57 CACHE-SRV systemd[1]: varnishncsa.service: Failed with result 'resources'. May 26 14:39:57 CACHE-SRV polkitd(authority=local)[799]: Unregistered Authentication Agent for unix-process:15422:8414612 (system bus name :1.67, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) Any help with this is greatly appreciated. Albert On Wed, May 25, 2016 at 6:41 PM, Albert Tollku?i wrote: > Tried even the solution posted here > https://blog.onetechnical.com/2014/05/02/hacking-up-varnish-logging-on-ubuntu-12-04/ > by modifying /etc/init.d/varnishncsa but still log format stays the same... > > On Wed, May 25, 2016 at 5:44 PM, Albert Tollku?i < > albert.tollkuci at gmail.com> wrote: > >> Output from journalctl -xe is: >> >> May 25 15:33:29 CACHE-SRV systemd[1]: Started Varnish HTTP accelerator >> log daemon. >> -- Subject: Unit varnishncsa.service has finished start-up >> -- Defined-By: systemd >> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel >> -- >> -- Unit varnishncsa.service has finished starting up. >> -- >> -- The start-up result is done. >> May 25 15:33:29 CACHE-SRV polkitd(authority=local)[799]: Unregistered >> Authentication Agent for unix-process:2383:95786 (system bus name :1.28, >> object path /org/freedesktop/ >> >> >> >> >> On Wed, May 25, 2016 at 5:40 PM, Guillaume Quintard < >> guillaume at varnish-software.com> wrote: >> >>> >>> >>> On Wed, May 25, 2016 at 5:35 PM, Albert Tollku?i < >>> albert.tollkuci at gmail.com> wrote: >>> >>>> >>>> Job for varnishncsa.service failed because a configured resource limit >>>> was exceeded. See "systemctl status varnishncsa.service" and "journalctl >>>> -xe" for details. >>>> >>>> >>> Actually, showing us the details would help here, we are in the dark >>> without them. >>> >>> -- >>> Guillaume Quintard >>> >> >> >> >> -- >> Web: http://www.tollkuci.com >> Follow me on: LinkedIn >> Google+ Facebook >> Twitter >> Career 2.0 >> >> ------------------------------ >> >> > > > -- > Web: http://www.tollkuci.com > Follow me on: LinkedIn Google+ > Facebook > Twitter > Career 2.0 > > ------------------------------ > > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ Imagination is more important than knowledge *Albert Einstein* The three chief virtues of a programmer are: Laziness, Impatience and Hubris *Larry Wall* Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders *Erwin Rommel* The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. *Randall E. Stross* Measuring programming progress by lines of code is like measuring aircraft building progress by weight. *Bill Gates* -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Thu May 26 14:53:25 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 26 May 2016 16:53:25 +0200 Subject: Strange Issue with cache and logged in users In-Reply-To: <48b993e0-827c-4bfb-e5b7-b1ef7ce6dd08@waltzz.com> References: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> <373a8b4a-1fac-d8ee-17c1-2ccc8dd9ae48@waltzz.com> <48b993e0-827c-4bfb-e5b7-b1ef7ce6dd08@waltzz.com> Message-ID: On Thu, May 26, 2016 at 4:40 PM, Pinakee BIswas wrote: > Regarding the faulty request, Following are two requests for the same URL > - /heels/ - one when the user was logged in and another when the user had > logged out. The second one (when the user was logged out) is the faulty one > as the user is still seeing the data when he was logged in. > > - BereqMethod GET > - BereqURL /heels/ > - BereqProtocol HTTP/1.0 > - BereqHeader X-CLIENT-COUNTRY: IN > - BereqHeader Host: varnish_staging > - BereqHeader Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > - BereqHeader Upgrade-Insecure-Requests: 1 > - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X > 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 > Safari/537.36 > - BereqHeader Referer: http://test.jivaana.in/home > - BereqHeader Accept-Language: en-US,en;q=0.8 > - BereqHeader X-Forwarded-For: 127.0.0.1 > - BereqHeader Accept-Encoding: gzip > - BereqProtocol HTTP/1.1 > - BereqHeader X-Varnish: 32788 > Are you sure your backend isn't caching stuff or something like that? Looking at bereq 32788, varnish is fetch a fresh object, and I see no header that would identify the user. Have you tried without varnish? -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Thu May 26 14:55:42 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 26 May 2016 16:55:42 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: On Thu, May 26, 2016 at 4:40 PM, Albert Tollku?i wrote: > Still no solution for this...If i manually run > > /usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F > "%h|%u|%{%Y-%m-%d}t" > > I get the correct log format. However when I put this > on /etc/systemd/system/varnishncsa.service, such as: > > Then yours seems to be a systemd problem. Do the package-provided script work? Is SELinux running on your system? If yes, try disabling it just to test. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Thu May 26 15:05:23 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Thu, 26 May 2016 17:05:23 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: SELinux is not running...it looks like a systemd problem, but I have the default installation for Ubuntu, not sure why it's messed up. On Thu, May 26, 2016 at 4:55 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > On Thu, May 26, 2016 at 4:40 PM, Albert Tollku?i < > albert.tollkuci at gmail.com> wrote: > >> Still no solution for this...If i manually run >> >> /usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F >> "%h|%u|%{%Y-%m-%d}t" >> >> I get the correct log format. However when I put this >> on /etc/systemd/system/varnishncsa.service, such as: >> >> > Then yours seems to be a systemd problem. Do the package-provided script > work? Is SELinux running on your system? If yes, try disabling it just to > test. > > > -- > Guillaume Quintard > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Thu May 26 15:29:18 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Thu, 26 May 2016 20:59:18 +0530 Subject: Strange Issue with cache and logged in users In-Reply-To: References: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> <373a8b4a-1fac-d8ee-17c1-2ccc8dd9ae48@waltzz.com> <48b993e0-827c-4bfb-e5b7-b1ef7ce6dd08@waltzz.com> Message-ID: <5c2abbaa-5b06-fea7-e79e-8c687a4afec0@waltzz.com> Hi Guillaume, Our application has been live since last few months and has been working fine. We have been using nginx caching in our production. We have been planning to replace nginx caching with Varnish and use nginx purely as a proxy. For some requests, we plan to bypass Varnish as they need not be cached. Also, we are serving static content using nginx. Right now the Varnish is on our test environment (will test thoroughly before deploying) where we do not use caching as developers want to see immediate results of the changes they make. To answer your query, it works fine without varnish. I have tested removing varnish and then adding it. I also, was surprised that it wasn't working properly as the logs say otherwise. Will investigate further. I hope the VCL logic/code is fine. One more thing is the cookie in the request log isn't showing all the cookies: - ReqHeader Cookie: _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-1518b8ba6c428a%22%2C%22%24i There are couple of more cookies like 'loggedin_user' and 'sessionid' which are not shown in the log. Whereas the vcl_recv works fine as per the code written. Also, the network flow in inspect element (of the browser) shows the cookies present in the Request header. Thanks, Pinakee On 26/05/16 8:23 pm, Guillaume Quintard wrote: > On Thu, May 26, 2016 at 4:40 PM, Pinakee BIswas > wrote: > > Regarding the faulty request, Following are two requests for the > same URL - /heels/ - one when the user was logged in and another > when the user had logged out. The second one (when the user was > logged out) is the faulty one as the user is still seeing the data > when he was logged in. > > - BereqMethod GET > - BereqURL /heels/ > - BereqProtocol HTTP/1.0 > - BereqHeader X-CLIENT-COUNTRY: IN > - BereqHeader Host: varnish_staging > - BereqHeader Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > - BereqHeader Upgrade-Insecure-Requests: 1 > - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac > OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) > Chrome/50.0.2661.102 Safari/537.36 > - BereqHeader Referer: http://test.jivaana.in/home > - BereqHeader Accept-Language: en-US,en;q=0.8 > - BereqHeader X-Forwarded-For: 127.0.0.1 > - BereqHeader Accept-Encoding: gzip > - BereqProtocol HTTP/1.1 > - BereqHeader X-Varnish: 32788 > > > Are you sure your backend isn't caching stuff or something like that? > Looking at bereq 32788, varnish is fetch a fresh object, and I see no > header that would identify the user. Have you tried without varnish? > > -- > Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Thu May 26 15:41:41 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 26 May 2016 17:41:41 +0200 Subject: Strange Issue with cache and logged in users In-Reply-To: <5c2abbaa-5b06-fea7-e79e-8c687a4afec0@waltzz.com> References: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> <373a8b4a-1fac-d8ee-17c1-2ccc8dd9ae48@waltzz.com> <48b993e0-827c-4bfb-e5b7-b1ef7ce6dd08@waltzz.com> <5c2abbaa-5b06-fea7-e79e-8c687a4afec0@waltzz.com> Message-ID: On Thu, May 26, 2016 at 5:29 PM, Pinakee BIswas wrote: > We have been planning to replace nginx caching with Varnish and use nginx > purely as a proxy. For some requests, we plan to bypass Varnish as they > need not be cached. Also, we are serving static content using nginx. > Once this is sorted, I'd rather use varnish as a proxy, but, that's just me :-) > To answer your query, it works fine without varnish. I have tested > removing varnish and then adding it. I also, was surprised that it wasn't > working properly as the logs say otherwise. Will investigate further. > Please do, the logs aren't showing anything weird. Note one thing: varnish will automatically add the X-Varnish header and put the VXID of the request (in case of a MIS/PASS) and the of the bereq (for a HIT), so when you get a wrong object, you can look a the varnishlog (using "-d" to look at past records), possibly filtering it with "-q" and check what was the cached request. One more thing, for the unlogged user, it seems your backend is replying with a set-cookie header. I don't think it's necessary. One more thing is the cookie in the request log isn't showing all the > cookies: > > - ReqHeader Cookie: > _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; > jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; > mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-1518b8ba6c428a%22%2C%22%24i > That's expected, varnish will truncate long log records, you can change vsl_reclen to see longer lines in the log. It doesn't change what actually goes on the wire. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Thu May 26 16:57:03 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Thu, 26 May 2016 22:27:03 +0530 Subject: Strange Issue with cache and logged in users In-Reply-To: References: <308a61e8-58ce-50b7-0ef9-3c4479180c41@waltzz.com> <373a8b4a-1fac-d8ee-17c1-2ccc8dd9ae48@waltzz.com> <48b993e0-827c-4bfb-e5b7-b1ef7ce6dd08@waltzz.com> <5c2abbaa-5b06-fea7-e79e-8c687a4afec0@waltzz.com> Message-ID: Thanks a lot for your help. Would certainly follow your instructions while debugging the issue. Once this is sorted, will look into Varnish as a proxy as well :) - But not sure of it's benchmark as compared to Nginx and also for delivering static content. Nginx has a very small footprint and performs well. Thanks, Pinakee On 26/05/16 9:11 pm, Guillaume Quintard wrote: > > One more thing, for the unlogged user, it seems your backend is replying From perbu at varnish-software.com Fri May 27 06:50:57 2016 From: perbu at varnish-software.com (Per Buer) Date: Fri, 27 May 2016 08:50:57 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: On Thu, May 26, 2016 at 5:05 PM, Albert Tollku?i wrote: > SELinux is not running...it looks like a systemd problem, but I have the > default installation for Ubuntu, not sure why it's messed up. > You messed it up when you stuck the pipes in there. Remove or figure out how to quote them and it should work fine. There is a manpage called systemd.service or something. It should tell you the rules. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.tollkuci at gmail.com Fri May 27 08:43:50 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Fri, 27 May 2016 10:43:50 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: Thank you Per. I've tried various combinations, such ash: ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F "%h" ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F '%h' ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F %h In all cases, the error is: May 27 08:37:28 CACHE-SRV systemd[1]: varnishncsa.service: Failed to run 'start' task: Operation not supported May 27 08:37:28 CACHE-SRV systemd[1]: Failed to start Varnish HTTP accelerator log daemon. -- Subject: Unit varnishncsa.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit varnishncsa.service has failed. -- -- The result is failed. May 27 08:37:28 CACHE-SRV systemd[1]: varnishncsa.service: Failed with result 'resources'. May 27 08:37:28 CACHE-SRV polkitd(authority=local)[799]: Unregistered Authentication Agent for unix-process:18570:14879687 (system bus name :1.130, object path /org/freedes For completion, here's the full /etc/init.d/varnishncsa file: --------------------------------------- #! /bin/sh ### BEGIN INIT INFO # Provides: varnishncsa # Required-Start: $local_fs $remote_fs $network # Required-Stop: $local_fs $remote_fs $network # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start HTTP accelerator log daemon # Description: This script provides logging for varnish ### END INIT INFO # Source function library . /lib/lsb/init-functions NAME=varnishncsa DESC="HTTP accelerator log deamon" PATH=/sbin:/bin:/usr/sbin:/usr/bin DAEMON=/usr/bin/$NAME PIDFILE=/run/$NAME/$NAME.pid LOGFILE=/var/log/varnish/varnishncsa.log USER=varnishlog DAEMON_OPTS="-a -w ${LOGFILE} -D -P ${PIDFILE}" # Include defaults if available if [ -f /etc/default/$NAME ] ; then . /etc/default/$NAME fi # If unset, or set to "0" or "no", exit if [ -z "${VARNISHNCSA_ENABLED}" ] || \ [ "${VARNISHNCSA_ENABLED}" = "0" ] || \ [ "${VARNISHNCSA_ENABLED}" = "no" ]; then exit 0; fi test -x $DAEMON || exit 0 fi test -x $DAEMON || exit 0 start_varnishncsa() { output=$(/bin/tempfile -s.varnish) log_daemon_msg "Starting $DESC" "$NAME" create_pid_directory if start-stop-daemon --start --quiet --pidfile ${PIDFILE} \ --chuid $USER --exec ${DAEMON} -- ${DAEMON_OPTS} \ > ${output} 2>&1; then log_end_msg 0 else log_end_msg 1 cat $output exit 1 fi rm $output } stop_varnishncsa(){ log_daemon_msg "Stopping $DESC" "$NAME" if start-stop-daemon --stop --quiet --pidfile $PIDFILE \ --retry 10 --exec $DAEMON; then log_end_msg 0 else log_end_msg 1 fi } reload_varnishncsa(){ log_daemon_msg "Reloading $DESC" "$NAME" if kill -HUP $(cat $PIDFILE) >/dev/null 2>&1; then log_end_msg 0 else log_end_msg 1 exit 1 fi } status_varnishncsa(){ status_of_proc -p "${PIDFILE}" "${DAEMON}" "${NAME}" exit $? } create_pid_directory() { install -o $USER -g $USER -d $(dirname $PIDFILE) } case "$1" in start) start_varnishncsa ;; stop) stop_varnishncsa ;; reload) reload_varnishncsa ;; status) status_varnishncsa ;; restart|force-reload) $0 stop $0 start ;; *) log_success_msg "Usage: $0 {start|stop|restart|force-reload|reload}" exit 1 ;; esac -------------------------------------------- Even tried to change DAEMON_OPTS directly in /etc/init.d/varnishncsa to: DAEMON_OPTS="-a -w ${LOGFILE} -D -P ${PIDFILE} -F '%h'" or DAEMON_OPTS="-a -w ${LOGFILE} -D -P ${PIDFILE} -F \"%h\"" but the log file format would not change. Albert On Fri, May 27, 2016 at 8:50 AM, Per Buer wrote: > > > On Thu, May 26, 2016 at 5:05 PM, Albert Tollku?i < > albert.tollkuci at gmail.com> wrote: > >> SELinux is not running...it looks like a systemd problem, but I have the >> default installation for Ubuntu, not sure why it's messed up. >> > > You messed it up when you stuck the pipes in there. Remove or figure out > how to quote them and it should work fine. > > There is a manpage called systemd.service or something. It should tell you > the rules. > > -- > *Per Buer* > CTO | Varnish Software AS > Cell: +47 95839117 > We Make Websites Fly! > www.varnish-software.com > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ Imagination is more important than knowledge *Albert Einstein* The three chief virtues of a programmer are: Laziness, Impatience and Hubris *Larry Wall* Men are basically smart or dumb and lazy or ambitious. The dumb and ambitious ones are dangerous and I get rid of them. The dumb and lazy ones I give mundane duties. The smart ambitious ones I put on my staff. The smart and lazy ones I make my commanders *Erwin Rommel* The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability. *Randall E. Stross* Measuring programming progress by lines of code is like measuring aircraft building progress by weight. *Bill Gates* -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Fri May 27 14:15:18 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Fri, 27 May 2016 16:15:18 +0200 Subject: Varnish delivering blank page (content-length=0) In-Reply-To: <8e7a6e6e-b1af-8e28-f9b2-df3bc91ce595@waltzz.com> References: <8e7a6e6e-b1af-8e28-f9b2-df3bc91ce595@waltzz.com> Message-ID: Looks like you stumbled on https://github.com/varnishcache/varnish-cache/issues/1954 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mr.jarry at gmail.com Fri May 27 15:53:45 2016 From: mr.jarry at gmail.com (Jarry) Date: Fri, 27 May 2016 17:53:45 +0200 Subject: Proper Pound/Varnish set-up? Message-ID: <7a740a0f-918f-5977-01d0-8a20cb3c8ae5@gmail.com> Hi, I'm quite new to varnish and http(s) caching, but I'd like to set up caching for my apache webserver (running a few web-sites with Drupal). I searched google for some info and found basically two possibilities: _____________________________________ A): Varnish listening on the port 80/http (caching Apache, running on 8080), and Pound listening only on the port 443/https (after https- termination forwarding traffic to Varnish). -> Varnish(public_IP:80) -> Apache(lo:8080) ^ | -> Pound(public_IP:443) _____________________________________ B): Pound listening on both 80/http and 443/https ports. Both http (directly) and https (after termination) traffic is forwarded to Varnish, and further to Apache. -> Pound(public_IP:80,443) -> Varnish(lo:8008) -> Apache(lo:8080) _____________________________________ I tested both of these configurations, both of them seem to work (sort of, there are certain specific problems with both set-ups). But I still can not decide: which set-up is better, and why? Jarry -- _______________________________________________________________ This mailbox accepts e-mails only from selected mailing-lists! Everything else is considered to be spam and therefore deleted. From pinakee at waltzz.com Sat May 28 08:04:05 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Sat, 28 May 2016 13:34:05 +0530 Subject: Varnish delivering blank page (content-length=0) In-Reply-To: References: <8e7a6e6e-b1af-8e28-f9b2-df3bc91ce595@waltzz.com> Message-ID: Hi Guillaume, Thanks for the reference. The issue you mentioned was what I was facing. But I solved it by configuring backend to send Content-Length. Thanks, Pinakee On 27/05/16 7:45 pm, Guillaume Quintard wrote: > Looks like you stumbled on > https://github.com/varnishcache/varnish-cache/issues/1954 -------------- next part -------------- An HTML attachment was scrubbed... URL: From yianniska at gmail.com Sat May 28 20:53:24 2016 From: yianniska at gmail.com (Yiannis Karayiannidis) Date: Sat, 28 May 2016 23:53:24 +0300 Subject: W3C like format logging In-Reply-To: References: Message-ID: Hi all, you could check the following http://unix.stackexchange.com/questions/204653/systemds-parameter-expansion-is-interfering-with-my-services-own-command-line hope it helps 2016-05-27 11:43 GMT+03:00 Albert Tollku?i : > Thank you Per. I've tried various combinations, such ash: > > ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F > "%h" > > ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F > '%h' > > ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F %h > > In all cases, the error is: > > May 27 08:37:28 CACHE-SRV systemd[1]: varnishncsa.service: Failed to run > 'start' task: Operation not supported > May 27 08:37:28 CACHE-SRV systemd[1]: Failed to start Varnish HTTP > accelerator log daemon. > -- Subject: Unit varnishncsa.service has failed > -- Defined-By: systemd > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel > -- > -- Unit varnishncsa.service has failed. > -- > -- The result is failed. > May 27 08:37:28 CACHE-SRV systemd[1]: varnishncsa.service: Failed with > result 'resources'. > May 27 08:37:28 CACHE-SRV polkitd(authority=local)[799]: Unregistered > Authentication Agent for unix-process:18570:14879687 (system bus name > :1.130, object path /org/freedes > > > For completion, here's the full /etc/init.d/varnishncsa file: > > --------------------------------------- > > #! /bin/sh > > ### BEGIN INIT INFO > # Provides: varnishncsa > # Required-Start: $local_fs $remote_fs $network > # Required-Stop: $local_fs $remote_fs $network > # Default-Start: 2 3 4 5 > # Default-Stop: 0 1 6 > # Short-Description: Start HTTP accelerator log daemon > # Description: This script provides logging for varnish > ### END INIT INFO > > # Source function library > . /lib/lsb/init-functions > > NAME=varnishncsa > DESC="HTTP accelerator log deamon" > PATH=/sbin:/bin:/usr/sbin:/usr/bin > DAEMON=/usr/bin/$NAME > PIDFILE=/run/$NAME/$NAME.pid > LOGFILE=/var/log/varnish/varnishncsa.log > USER=varnishlog > DAEMON_OPTS="-a -w ${LOGFILE} -D -P ${PIDFILE}" > > # Include defaults if available > if [ -f /etc/default/$NAME ] ; then > . /etc/default/$NAME > fi > > # If unset, or set to "0" or "no", exit > if [ -z "${VARNISHNCSA_ENABLED}" ] || \ > [ "${VARNISHNCSA_ENABLED}" = "0" ] || \ > [ "${VARNISHNCSA_ENABLED}" = "no" ]; then > exit 0; > fi > > test -x $DAEMON || exit 0 > > > fi > > test -x $DAEMON || exit 0 > > start_varnishncsa() { > output=$(/bin/tempfile -s.varnish) > log_daemon_msg "Starting $DESC" "$NAME" > create_pid_directory > if start-stop-daemon --start --quiet --pidfile ${PIDFILE} \ > --chuid $USER --exec ${DAEMON} -- ${DAEMON_OPTS} \ > > ${output} 2>&1; then > log_end_msg 0 > else > log_end_msg 1 > cat $output > exit 1 > fi > rm $output > } > > stop_varnishncsa(){ > log_daemon_msg "Stopping $DESC" "$NAME" > if start-stop-daemon --stop --quiet --pidfile $PIDFILE \ > --retry 10 --exec $DAEMON; then > log_end_msg 0 > else > log_end_msg 1 > fi > } > > > reload_varnishncsa(){ > log_daemon_msg "Reloading $DESC" "$NAME" > if kill -HUP $(cat $PIDFILE) >/dev/null 2>&1; then > log_end_msg 0 > else > log_end_msg 1 > exit 1 > fi > } > > status_varnishncsa(){ > status_of_proc -p "${PIDFILE}" "${DAEMON}" "${NAME}" > exit $? > } > > create_pid_directory() { > install -o $USER -g $USER -d $(dirname $PIDFILE) > } > > > case "$1" in > start) > start_varnishncsa > ;; > stop) > stop_varnishncsa > ;; > reload) > reload_varnishncsa > ;; > status) > status_varnishncsa > ;; > restart|force-reload) > $0 stop > $0 start > ;; > *) > log_success_msg "Usage: $0 > {start|stop|restart|force-reload|reload}" > exit 1 > ;; > esac > > -------------------------------------------- > > Even tried to change DAEMON_OPTS directly in /etc/init.d/varnishncsa to: > > DAEMON_OPTS="-a -w ${LOGFILE} -D -P ${PIDFILE} -F '%h'" > > or > > DAEMON_OPTS="-a -w ${LOGFILE} -D -P ${PIDFILE} -F \"%h\"" > > but the log file format would not change. > > Albert > > > On Fri, May 27, 2016 at 8:50 AM, Per Buer > wrote: > >> >> >> On Thu, May 26, 2016 at 5:05 PM, Albert Tollku?i < >> albert.tollkuci at gmail.com> wrote: >> >>> SELinux is not running...it looks like a systemd problem, but I have the >>> default installation for Ubuntu, not sure why it's messed up. >>> >> >> You messed it up when you stuck the pipes in there. Remove or figure out >> how to quote them and it should work fine. >> >> There is a manpage called systemd.service or something. It should tell >> you the rules. >> >> -- >> *Per Buer* >> CTO | Varnish Software AS >> Cell: +47 95839117 >> We Make Websites Fly! >> www.varnish-software.com >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > > -- > Web: http://www.tollkuci.com > Follow me on: LinkedIn Google+ > Facebook > Twitter > Career 2.0 > > ------------------------------ > > Imagination is more important than knowledge > *Albert Einstein* > > The three chief virtues of a programmer are: Laziness, Impatience and > Hubris > *Larry Wall* > > Men are basically smart or dumb and lazy or ambitious. The dumb and > ambitious ones are dangerous and I get rid of them. The dumb and lazy ones > I give mundane duties. The smart ambitious ones I put on my staff. The > smart and lazy ones I make my commanders > *Erwin Rommel* > > The best programmers are not marginally better than merely good ones. > They are an order-of-magnitude better, measured by whatever standard: > conceptual creativity, speed, ingenuity of design, or problem-solving > ability. > *Randall E. Stross* > > Measuring programming progress by lines of code is like measuring aircraft > building progress by weight. > *Bill Gates* > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pada at posteo.de Sat May 28 22:44:42 2016 From: pada at posteo.de (Daniel Parthey) Date: Sun, 29 May 2016 00:44:42 +0200 Subject: W3C like format logging In-Reply-To: References: Message-ID: <3868D496-3291-4879-BD82-B1F155596038@posteo.de> So what about ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F "%%h %%l %%u" From albert.tollkuci at gmail.com Mon May 30 09:00:31 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Mon, 30 May 2016 11:00:31 +0200 Subject: W3C like format logging In-Reply-To: <3868D496-3291-4879-BD82-B1F155596038@posteo.de> References: <3868D496-3291-4879-BD82-B1F155596038@posteo.de> Message-ID: That was it Daniel. Appreciated. On Sun, May 29, 2016 at 12:44 AM, Daniel Parthey wrote: > So what about > > ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F > "%%h %%l %%u" > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at massivescale.net Mon May 30 11:31:35 2016 From: info at massivescale.net (Andrzej Godziuk) Date: Mon, 30 May 2016 13:31:35 +0200 Subject: Proper Pound/Varnish set-up? In-Reply-To: <7a740a0f-918f-5977-01d0-8a20cb3c8ae5@gmail.com> References: <7a740a0f-918f-5977-01d0-8a20cb3c8ae5@gmail.com> Message-ID: <20160530133135.69f59556@gdr-desktop.gdr.name> Hi, I've always used A), with the assumption that the less data is being copied, the better for performance. I also find it easier to debug. The only advanage of B) I can see is that it may be easier to make a mistake in A) scenario - such as accepting X-Forwarded-For from external IPs. Andrzej Godziuk From yianniska at gmail.com Mon May 30 12:04:40 2016 From: yianniska at gmail.com (Yiannis Karayiannidis) Date: Mon, 30 May 2016 15:04:40 +0300 Subject: One minute delay on return (pipe) varnish-4.1.2 revision 0d7404e Message-ID: Hi all, one minute delay on return (pipe) and a POST-Request #1806 Current version varnish-4.1.2 revision 0d7404e still has this bug. Any idea when this fix is going to rearch Redhat EL7 ? Thanks very much. Yiannis p.s I've used ---------------- sub vcl_pipe { set bereq.http.connection = "close"; return (pipe); } -------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkarsten at varnish-software.com Mon May 30 14:46:06 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Mon, 30 May 2016 16:46:06 +0200 Subject: One minute delay on return (pipe) varnish-4.1.2 revision 0d7404e In-Reply-To: References: Message-ID: <20160530144605.GA15814@immer.varnish-software.com> On Mon, May 30, 2016 at 03:04:40PM +0300, Yiannis Karayiannidis wrote: > Hi all, > one minute delay on return (pipe) and a POST-Request #1806 > Current version varnish-4.1.2 revision 0d7404e still has this bug. > Any idea when this fix is going to rearch Redhat EL7 ? We've started the release work for 4.1.3, but there are some outstanding backporting from master that needs to be done before we can roll the release. Running from the 4.1 branch on git should be fairly safe. -- Lasse Karstensen Varnish Software AS From pinakee at waltzz.com Tue May 31 12:51:09 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Tue, 31 May 2016 18:21:09 +0530 Subject: Cache Invalidation Message-ID: Hi, I have deployed varnish on our production server and everything seems to work fine. Right now the TTL is getting set (by backend Cache-Control) and hence, the cache expiry. The next goal would be to have Varnish infinitely cache the pages with cache invalidation. Based on the following article: https://www.smashingmagazine.com/2014/04/cache-invalidation-strategies-with-varnish-cache/ it seems all the mechanisms of cache invalidation are manual (using Purge, Ban etc.). Is there an automatic way of doing it from the backend? The manual process would be cumbersome as we have separate team for data entry/operations and they would have to notify the system person etc. etc. Would appreciate if anyone could guide me to an optimal approach for the above. Thanks, Pinakee PS: How to set the maximum value for TTL? From dridi at varni.sh Tue May 31 13:34:03 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 31 May 2016 15:34:03 +0200 Subject: Cache Invalidation In-Reply-To: References: Message-ID: > Right now the TTL is getting set (by backend Cache-Control) and hence, the > cache expiry. The next goal would be to have Varnish infinitely cache the > pages with cache invalidation. That's a very sane goal :) > it seems all the mechanisms of cache invalidation are manual (using Purge, > Ban etc.). Is there an automatic way of doing it from the backend? The Unfortunately, Varnish only speaks HTTP to the backend. There's no way to automatically do that. > manual process would be cumbersome as we have separate team for data > entry/operations and they would have to notify the system person etc. etc. A different strategy could be to have a "short" TTL, a virtually infinite grace period (you can also set it via Cache-Control) and make use of Last-Modified/Etag headers. This way Varnish will frequently ask your backend (short TTL) but will probably serve most requests from cache (high grace) and hammer your backend more often. The last point can be cheap if you implement IMS properly in your backend. This way you don't need to involve more teams, the people responsible for the backend's Cache-Control can now handle grace and conditional requests. Cheers From guillaume at varnish-software.com Tue May 31 13:50:30 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 31 May 2016 15:50:30 +0200 Subject: Cache Invalidation In-Reply-To: References: Message-ID: FWIW: a lot of CMS can be configured to warn varnish that content has changed ( http://devdocs.magento.com/guides/v2.0/config-guide/varnish/config-varnish-magento.html ) In this case, the backend will issue HTTP requests to purge content, and of course, your VCL must be ready to treat them. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdh132 at psu.edu Tue May 31 20:16:29 2016 From: jdh132 at psu.edu (Jason Heffner) Date: Tue, 31 May 2016 16:16:29 -0400 Subject: Proper Pound/Varnish set-up? In-Reply-To: <7a740a0f-918f-5977-01d0-8a20cb3c8ae5@gmail.com> References: <7a740a0f-918f-5977-01d0-8a20cb3c8ae5@gmail.com> Message-ID: Hi Jarry, I personally prefer B myself, even with Pound. I used to use Pound but have since switched to nginx as my front end to handle ssl, ipv6, custom headers, virtual hosts, etc. I ran into issues with Pound at one point and have been very pleased with nginx and it gives me more flexibility. Jason > On May 27, 2016, at 11:53 AM, Jarry wrote: > > Hi, > > I'm quite new to varnish and http(s) caching, but I'd like to set up > caching for my apache webserver (running a few web-sites with Drupal). > I searched google for some info and found basically two possibilities: > _____________________________________ > > A): Varnish listening on the port 80/http (caching Apache, running on > 8080), and Pound listening only on the port 443/https (after https- > termination forwarding traffic to Varnish). > > -> Varnish(public_IP:80) -> Apache(lo:8080) > ^ > | > -> Pound(public_IP:443) > _____________________________________ > > B): Pound listening on both 80/http and 443/https ports. Both http > (directly) and https (after termination) traffic is forwarded to > Varnish, and further to Apache. > > -> Pound(public_IP:80,443) -> Varnish(lo:8008) -> Apache(lo:8080) > _____________________________________ > > I tested both of these configurations, both of them seem to work > (sort of, there are certain specific problems with both set-ups). > But I still can not decide: which set-up is better, and why? > > Jarry > > -- > _______________________________________________________________ > This mailbox accepts e-mails only from selected mailing-lists! > Everything else is considered to be spam and therefore deleted. > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc