From jbq at caraldi.com Fri Jun 3 13:37:42 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Fri, 3 Jun 2011 15:37:42 +0200 Subject: Varnish 3.0 NCSA log format missing object age Message-ID: Hi there, I'm trying to upgrade my production host to Varnish 3.0 beta 1, and came across a missing format option in varnishncsa: the object age doesn't seem to be implemented. Also, an unrecognized format option like the afore-mentioned %{Age}i will cause varnishncsa to stop at the first log line processed with the following error in varnishncsa's stderr: Unknown format character: { And last but not least, the following formats do not appear to be documented: %{Varnish:time_firstbyte}x, %{Varnish:hitmiss}x, %{Varnish:handling}x. These are certainly very interesting facts to monitor, but I'd like to have a formal definition for them. Thanks in advance, and keep up the good job. -- Jean-Baptiste Quenot From martin at varnish-software.com Mon Jun 6 13:59:08 2011 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Mon, 6 Jun 2011 15:59:08 +0200 Subject: [PATCH] Proposed patch to fix bug #876 Message-ID: See attached patch for a proposal to fix bug #876. Any comments appreciated. >From commit message: Register an atexit handler that clears the master pid from the shmlog at exit, and make the 'owned by other running varnishd' message a warning. Fixes: #876 -Martin -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Register-an-atexit-handler-that-clears-the-master-pi.patch Type: application/octet-stream Size: 1716 bytes Desc: not available URL: From martin at varnish-software.com Thu Jun 9 11:53:41 2011 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 9 Jun 2011 13:53:41 +0200 Subject: Patch review for adding locking on the SHMFILE to prevent multiple varnish instances using the same -n argument Message-ID: Please find the patch here: https://gist.github.com/1016579 Any comments appreciated. -Martin -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at varnish-software.com Tue Jun 14 11:43:09 2011 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 14 Jun 2011 13:43:09 +0200 Subject: Patch review for adding locking on the SHMFILE to prevent multiple varnish instances using the same -n argument In-Reply-To: References: Message-ID: Updated patch at https://gist.github.com/1024738 Changelog: Moved the testing of file locks into a function in libvarnish. -Martin On Thu, Jun 9, 2011 at 13:53, Martin Blix Grydeland < martin at varnish-software.com> wrote: > Please find the patch here: https://gist.github.com/1016579 > > Any comments appreciated. > > -Martin > > -- > Martin Blix Grydeland > Varnish Software AS > > -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Tue Jun 14 11:47:36 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 14 Jun 2011 11:47:36 +0000 Subject: Patch review for adding locking on the SHMFILE to prevent multiple varnish instances using the same -n argument In-Reply-To: Your message of "Tue, 14 Jun 2011 13:43:09 +0200." Message-ID: <60494.1308052056@critter.freebsd.dk> Looks good. commit when tested cross-platform. In message , Martin Blix Grydeland writes: >--===============8734530158992454258== >Content-Type: multipart/alternative; boundary=bcaec5431bb03d763a04a5aa8aef > >--bcaec5431bb03d763a04a5aa8aef >Content-Type: text/plain; charset=ISO-8859-1 > >Updated patch at https://gist.github.com/1024738 > >Changelog: Moved the testing of file locks into a function in libvarnish. > >-Martin > >On Thu, Jun 9, 2011 at 13:53, Martin Blix Grydeland < >martin at varnish-software.com> wrote: > >> Please find the patch here: https://gist.github.com/1016579 >> >> Any comments appreciated. >> >> -Martin >> >> -- >> Martin Blix Grydeland >> Varnish Software AS >> >> > > >-- >Martin Blix Grydeland >Varnish Software AS > >--bcaec5431bb03d763a04a5aa8aef >Content-Type: text/html; charset=ISO-8859-1 >Content-Transfer-Encoding: quoted-printable > >Updated patch at=A0charset=3Dutf-8">https://gist.g= >ithub.com/1024738

Changelog: Moved the testing of fi= >le locks into a function in libvarnish.
>

-Martin

On Thu, Jun 9= >, 2011 at 13:53, Martin Blix Grydeland <lto:martin at varnish-software.com">martin at varnish-software.com>= > wrote:
>
x #ccc solid;padding-left:1ex;">Please find the patch here:=A0tps://gist.github.com/1016579" target=3D"_blank">https://gist.github.com/10= >16579
>
Any comments appreciated.
>

-Martin
=A0
--
Martin Blix Gryd= >eland
Varnish Software AS

>
>



--
Martin Blix Grydelandr>Varnish Software AS

>
> >--bcaec5431bb03d763a04a5aa8aef-- > > >--===============8734530158992454258== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >varnish-dev mailing list >varnish-dev at varnish-cache.org >http://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev >--===============8734530158992454258==-- > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From magnus at hagander.net Fri Jun 17 11:57:48 2011 From: magnus at hagander.net (Magnus Hagander) Date: Fri, 17 Jun 2011 13:57:48 +0200 Subject: Wrong version of server in RPM? Message-ID: Just downloaded varnish-3.0.0-1.el5.x86_64.rpm from the varnish repo at http://repo.varnish-cache.org/redhat/varnish-3.0/el5/x86_64/. This varnishd in that RPM reports that it's 3.0.0-beta2 revision a6d07c0. That can't be right... Is it actually the wrong version, or the right version with the wrong version stamp? Oh, and related to that - is there a way to see the version of a running varnishd using varnishadm (or the telnet interface...) To verify if a server has been reloaded or not... -- ?Magnus Hagander ?Me: http://www.hagander.net/ ?Work: http://www.redpill-linpro.com/ From yangxu4000 at sina.com Mon Jun 20 08:39:56 2011 From: yangxu4000 at sina.com (yangxu4000 at sina.com) Date: Mon, 20 Jun 2011 16:39:56 +0800 Subject: puzzle about unset req.http.cookie Message-ID: <20110620083956.E2434364424@mail3-111.sinamail.sina.com.cn>  Hello everyone, I do not know where to ask my question  so I send it here. thanks everyone. I hava questions about  req.http.cookie:I see some configuration on google like this? if?req.url ~ ".(js|css|jpg|png|gif|swf|jpeg|ico)$?{unset req.http.cookie?} it says if images have cookie?It can not be cached. but I do not set this in my configuration, I still saw CACHE-HIT (resp.http.X-Cache = "CACHE-HIT") in http response. anyone could explain  this? thank you very much! -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrea.campi at zephirworks.com Mon Jun 20 08:47:52 2011 From: andrea.campi at zephirworks.com (Andrea Campi) Date: Mon, 20 Jun 2011 10:47:52 +0200 Subject: puzzle about unset req.http.cookie In-Reply-To: <20110620083956.E2434364424@mail3-111.sinamail.sina.com.cn> References: <20110620083956.E2434364424@mail3-111.sinamail.sina.com.cn> Message-ID: 2011/6/20 : > ?Hello everyone, > I do not know where to ask my question ?so I send it here. thanks everyone. > I hava questions about ?req.http.cookie: > I see some configuration on google like this? > if?req.url ~ ".(js|css|jpg|png|gif|swf|jpeg|ico)$?{ > unset req.http.cookie? > } > it says if images have cookie?It can not be cached. > but I do not set this in my configuration, I still saw?CACHE-HIT > (resp.http.X-Cache = "CACHE-HIT") in http response. > anyone could explain ?this? You probably have something else in your VCL that allows caching. Send the whole VCL, we can't just guess. Andrea From yangxu4000 at sina.com Mon Jun 20 08:56:57 2011 From: yangxu4000 at sina.com (yangxu4000 at sina.com) Date: Mon, 20 Jun 2011 16:56:57 +0800 Subject: =?GBK?B?u9i4tKO6UmU6IHB1enpsZSBhYm91dCB1bnNldCByZXEuaHR0cC5jb29raWU=?= Message-ID: <20110620085657.BD9B0364420@mail3-111.sinamail.sina.com.cn> Hi,this is my configuration, thanks~backend www {.host = "xx.xx.xx.xx";.port = "80";.connect_timeout = 60s;.first_byte_timeout = 10s;.between_bytes_timeout = 5s;}sub vcl_recv {         if ( req.http.host ~ "^xxx.com") {        set req.backend = www;        }        else        {        error 404 "not allowed hostname";        }         if (req.request == "GET" && req.url ~ "\.(js|css|jpg|png|gif|swf|jpeg|ico)$") {        unset req.http.cookie;        }         if (req.request != "GET" && req.request != "HEAD") {        return (pass);        }        if (req.http.x-forwarded-for) {        set req.http.X-Forwarded-For = req.http.X-Forwarded-For ", " client.ip;        }        else {        set req.http.X-Forwarded-For = client.ip;        }     if (req.request == "GET" && req.url !~ "/images/captcha\.gif" && (req.url ~ ".(js|css|jpg|png|gif|swf|jpeg|ico)$"  || req.url ~ "/images/.*" || req.url ~ "/files/.*"  || req.url ~ "/javascripts/.*" || req.url ~ "/stylesheets/.*"  ||  req.url ~ "^/gwt/.*" ) ) {        lookup;     }    else    {    pass;    }}sub vcl_pipe {return (pipe);}sub vcl_pass {return (pass);} sub vcl_hit{   set obj.ttl=7d;}sub vcl_deliver {                remove resp.http.Via;                remove resp.http.X-Varnish;                remove resp.http.server;        if (obj.hits > 0) {                set resp.http.X-Cache = "MIC-HIT";        } else {                set resp.http.X-Cache = "MIC-MISS";        }    return (deliver);}...----- ???? ----- ????Andrea Campi ????yangxu4000 at sina.com ???Re: puzzle about unset req.http.cookie ???2011-6-20 16:47:52 2011/6/20 : >  Hello everyone, > I do not know where to ask my question  so I send it here. thanks everyone. > I hava questions about  req.http.cookie: > I see some configuration on google like this? > if?req.url ~ ".(js|css|jpg|png|gif|swf|jpeg|ico)$?{ > unset req.http.cookie? > } > it says if images have cookie?It can not be cached. > but I do not set this in my configuration, I still saw CACHE-HIT > (resp.http.X-Cache = "CACHE-HIT") in http response. > anyone could explain  this? You probably have something else in your VCL that allows caching. Send the whole VCL, we can't just guess. Andrea -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattias at nucleus.be Mon Jun 20 08:58:06 2011 From: mattias at nucleus.be (Mattias Geniar) Date: Mon, 20 Jun 2011 10:58:06 +0200 Subject: puzzle about unset req.http.cookie In-Reply-To: <20110620083956.E2434364424@mail3-111.sinamail.sina.com.cn> References: <20110620083956.E2434364424@mail3-111.sinamail.sina.com.cn> Message-ID: <18834F5BEC10824891FB8B22AC821A5A01734D3D@nucleus-srv01.Nucleus.local> > if?req.url ? ".(js|css|jpg|png|gif|swf|jpeg|ico)$?{ > unset req.http.cookie? > } > > it says if images have cookie?It can not be cached. Actually, that's not what it says. You're saying if the requested file ends in .js, .css, .jpg, ... or any of those file types, you will unset any cookie that is being sent along and force it to be cached. If you want to say "if the image has a cookie, don't cache it" you can do a check if a cookie exists on your request and just pass it on to your backend, something like this: if (req.http.cookie) { return (pass); } Regards, Mattias Geniar From yangxu4000 at sina.com Mon Jun 20 09:08:57 2011 From: yangxu4000 at sina.com (yangxu4000 at sina.com) Date: Mon, 20 Jun 2011 17:08:57 +0800 Subject: =?GBK?B?u9i4tKO6UkU6IHB1enpsZSBhYm91dCB1bnNldCByZXEuaHR0cC5jb29raWU=?= Message-ID: <20110620090857.AB8CB364420@mail3-111.sinamail.sina.com.cn> thanks for replying,I want to cache images and js, But I saw someone says "If Images have cookie, it can not be cached,so you must unset cookie first",so I want to know is there any   relations between images have cookie or not? thank you~ ----- ???? ----- ????Mattias Geniar ????, ???RE: puzzle about unset req.http.cookie ???2011-6-20 16:58:06 > if?req.url ? ".(js|css|jpg|png|gif|swf|jpeg|ico)$?{ > unset req.http.cookie? > } > > it says if images have cookie?It can not be cached. Actually, that's not what it says. You're saying if the requested file ends in .js, .css, .jpg, ... or any of those file types, you will unset any cookie that is being sent along and force it to be cached. If you want to say "if the image has a cookie, don't cache it" you can do a check if a cookie exists on your request and just pass it on to your backend, something like this: if (req.http.cookie) { return (pass); } Regards, Mattias Geniar -------------- next part -------------- An HTML attachment was scrubbed... URL: From yangxu4000 at sina.com Mon Jun 20 09:13:11 2011 From: yangxu4000 at sina.com (yangxu4000 at sina.com) Date: Mon, 20 Jun 2011 17:13:11 +0800 Subject: =?GBK?B?u9i4tKO6UkU6IHB1enpsZSBhYm91dCB1bnNldCByZXEuaHR0cC5jb29raWU=?= Message-ID: <20110620091312.3316736441E@mail3-111.sinamail.sina.com.cn> thanks for replying.I want to cache images and js, But I saw someone says "If Images have cookie, it can not be cached,so you must unset cookie first",so I want to know is there any relations if images have cookie or not?thank you~----- ???? ----- ????Mattias Geniar ????, ???RE: puzzle about unset req.http.cookie ???2011-6-20 16:58:06 > if?req.url ? ".(js|css|jpg|png|gif|swf|jpeg|ico)$?{ > unset req.http.cookie? > } > > it says if images have cookie?It can not be cached. Actually, that's not what it says. You're saying if the requested file ends in .js, .css, .jpg, ... or any of those file types, you will unset any cookie that is being sent along and force it to be cached. If you want to say "if the image has a cookie, don't cache it" you can do a check if a cookie exists on your request and just pass it on to your backend, something like this: if (req.http.cookie) { return (pass); } Regards, Mattias Geniar -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrea.campi at zephirworks.com Mon Jun 20 09:24:08 2011 From: andrea.campi at zephirworks.com (Andrea Campi) Date: Mon, 20 Jun 2011 11:24:08 +0200 Subject: =?UTF-8?B?UmU6IOWbnuWkje+8mlJlOiBwdXp6bGUgYWJvdXQgdW5zZXQgcmVxLmh0dHAuY29va2ll?= In-Reply-To: <20110620085657.BD9B0364420@mail3-111.sinamail.sina.com.cn> References: <20110620085657.BD9B0364420@mail3-111.sinamail.sina.com.cn> Message-ID: 2011/6/20 : > Hi, > this is my configuration, thanks~ > backend www { > .host = "xx.xx.xx.xx"; > .port = "80"; > .connect_timeout = 60s; > .first_byte_timeout = 10s; > .between_bytes_timeout = 5s; > } > sub vcl_recv { > ? ? ? ? if ( req.http.host ~ "^xxx.com") { > ? ? ? ? set req.backend = www; > ? ? ? ? } > ? ? ? ? else > ? ? ? ? { > ? ? ? ? error 404 "not allowed hostname"; > ? ? ? ? } > ? ? ? ? if (req.request == "GET" && req.url ~ > "\.(js|css|jpg|png|gif|swf|jpeg|ico)$") { > ? ? ? ? unset req.http.cookie; > ? ? ? ? } So there you go, you are in fact stripping cookies > > ? ? if (req.request == "GET" && req.url !~ "/images/captcha\.gif" && > (req.url ~ ".(js|css|jpg|png|gif|swf|jpeg|ico)$" ?|| req.url ~ "/images/.*" > || req.url ~ "/files/.*" ?|| req.url ~ "/javascripts/.*" || req.url ~ > "/stylesheets/.*" ?|| ?req.url ~ "^/gwt/.*" ) ) { > ? ? ? ? lookup; > ? ? ?} Depending on which request you're testing with, you may be hitting this, or falling through the end of the proc and hitting the default logic (which will do the lookup anyway). So to summarize, you are stripping cookies, and you are caching images. It seems you are already doing what you wanted to do? From martin at varnish-software.com Fri Jun 24 13:22:01 2011 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Fri, 24 Jun 2011 15:22:01 +0200 Subject: Streaming details Message-ID: Hi Poul-Henning, I've detailed some aspects of how I imagine the call paths and actions will be when we get the next part of streaming in place, using pseudo-something :) I've tried to detail out some of the locking strategies to get a picture about what will be locked when and see if we are creating too heavy contention on anything. You were working on the moving of the waiting lists to the busyobj right? How is that progressing? I'm going to start hacking along these lines beginning next week. Should I create the branch in git for this? -Martin Call flow for streaming fetch&delivery (pass is not detailed in this flow): Note: The busyobj from HSH_Prealloc has refcnt==1 1. cnt_lookup() * Creates the object marked as busy. The object will have a busyobj already attached to it (attached when it is created in HSH_Prealloc) * Next step is STP_MISS 2. cnt_miss() * Nothing special * Next step is STP_FETCH 3. cnt_fetch() * Nothing special * Next step is STP_FETCHBODY 4. cnt_fetchbody() * If sp->wrk->do_stream, skips calling FetchBody() * Next step is STP_PREPRESP 5. cnt_prepresp() * If sp->wrk->do_stream, sets next step to STP_STREAMBODY 6. cnt_streambody() * This function has the stream_ctx on the stack * Does not need to grab a busyobj->refcnt as it is initialized to 1, consequently does not need to lock objhdr here * Calls RES_StreamStart * Then calls FetchBody() * FetchBody() (in both the gzip and nop vfp's) will do callbacks to RES_StreamPoll if sp->wrk->do_stream is true * RES_StreamPoll will on the first call to it (looking at it's stream_ctx) know that on the first call to it set the busyobj->can_stream to true. This will allow subsequent requests coming in to start streaming the content directly (not go on waiting list / not receive grace candidate). It should also wake (all or just some) sessions from the waiting list. These will then compete for tokens. (Waking all here might create some thundering herd? Wake as many as we have tokens for? And let any thread releasing a token try to wake an amount equal to the number of tokens currently available?) * RES_StreamPoll should (looking at it's stream_ctx) know that it is the backend receiving thread as well and should: * Lock the busyobj->lock * Update busyobj counters with new end of data * Signal the cond to wake one sleeping thread. This thread will in turn trigger another until all the tokens are in use. * Unlock busobj->lock * Note: RES_StreamPoll will free data as it progresses when in pass mode. * Calls HSH_Unbusy() * While holding objhdr->lock: * Grab busyobj->lock * Sets busyobj->complete * Clears the busy flag so later calls to this object will not go by the streaming bits, and there will eventually be no more refs to the busyobj. * Decrements busyobj->refcnt, last one to leave turns off the light and frees it. * Signals the cond. * Release busyobj->lock * Calls RES_StreamEnd() * Next step is STP_DONE 7. Nothing special from this point on Call flow for streaming delivery only (hit on streaming object): 1. cnt_lookup() * Does a HSH_Lookup and gets an oc that is busy * Next step is STP_HIT 2. cnt_hit() * nothing special * Next step is STP_PREPRESP 3. cnt_prepresp() * (Here some work on different length algorithms is done) * Also calls RES_BuildHttp * Next step is STP_DELIVER 3. cnt_deliver() * Will be split into seperate paths for streaming and non-streaming execution * Has the stream_ctx on the stack for this delivery * Test if object is busy (OC_F_BUSY). If not normal implementation is executed * Lock objhdr * Test again on (OC_F_BUSY && objhdr->objcore->busyobj->can_stream), if not unlock objhdr and run normal implementation. Normal here means go on the waiting list as before, waiting for the first parts of the body to appear. * Increase busyobj->refcnt * Unlock objhdr * Will call RES_StreamStart to set up the client receive state * Grab busyobj->lock * Loop until all has been delivered (delivered up to busyobj->total_len bytes or busyobj->complete is true) * Try to get a token (busyobj->token--). If unsuccessful, wait on busyobj->cond (releasing lock) and continue loop * If other tokens available, signal busyobj->cond to give the next in line a chance to grab a token * Loop around while your stream_ctx->end_of_data < busyobj->end_of_data * Update your stream_ctx end of data from the busyobj * Release busyobj->lock * Deliver what you can (call RES_StreamPoll) * Grab busyobj->lock * Release token (busyobj->token++) * Wait on busyobj->cond (releasing lock) * Call RES_StreamEnd * Deref busyobj * Grab objhead->lock * busyobj->refcnt-- * If zero * Free the busyobj * Release objhead->lock * Next step is STP_DONE Lifetime of busyobj: * busyobj is created in HSH_Prealloc and attached to objcore * busyobj is free'd by the current calling thread (and the pointer in objcore set to NULL) when it's refcnt reaches zero * busyobj is free'd whenever the objcore is free'd is the pointer is not NULL struct busyobj { unsigned magic; #define BUSYOBJ_MAGIC something int complete; int can_stream; int refcnt; int total_len; /* If known */ int current_len; int tokens; pthread_cond_t cond; pthread_lock_t lock; }; -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Tue Jun 28 12:47:05 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Tue, 28 Jun 2011 14:47:05 +0200 Subject: Wrong version of server in RPM? In-Reply-To: (Magnus Hagander's message of "Fri, 17 Jun 2011 13:57:48 +0200") References: Message-ID: <87hb7at96e.fsf@qurzaw.varnish-software.com> ]] Magnus Hagander | Just downloaded varnish-3.0.0-1.el5.x86_64.rpm from the varnish repo | at http://repo.varnish-cache.org/redhat/varnish-3.0/el5/x86_64/. | | This varnishd in that RPM reports that it's 3.0.0-beta2 revision a6d07c0. | | That can't be right... Is it actually the wrong version, or the right | version with the wrong version stamp? It was actually the wrong version. I've built 32 bit versions too now and uploaded a -2 RPM to the repo now. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From martin at varnish-software.com Tue Jun 28 13:56:51 2011 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 28 Jun 2011 15:56:51 +0200 Subject: Review for patch to reintroduce logging of TTL changes done in VCL Message-ID: Attached patch reintroduces the logging of TTL changes done in VCL. Any comments appreciated. - Martin -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ttl-logging.patch Type: application/octet-stream Size: 1859 bytes Desc: not available URL: From harm.verhagen at gmail.com Wed Jun 29 10:03:12 2011 From: harm.verhagen at gmail.com (Harm Verhagen) Date: Wed, 29 Jun 2011 12:03:12 +0200 Subject: varnishtop feature request: show _expensive_ urls Message-ID: Hi, I was looking into varnishtop to find performance bottlenecks on a website. Thinking about it I think a useful feature is currently missing from this tool. (while it _has_ all the available raw data). I'd like to discuss it here, to get your opinion. Currently varnishtop shows the top urls purely based on frequency. Depending on your usecase that might be usefull, but might also NOT give you the bottlenecks you are looking for. I'd like to use varnish for the following (example) scenario: Consider an website site with many dynamic urls. (dynamic means varnish does not cache them, they are always passed through to the backend) Of those there are two urls that are behaving "badly" X and Y (that is they are slow: lets say 400ms) We don't know that _these_ to are behaving badly. X: is called very low frequently -> so this is NOT a perf bottleneck. Y: is called often -> THIS IS THE BOTTLENECK -> this is the url that brings down the site. Z: is called often -> this one is fast -> not a bottleneck. I'd like varnishtop to show Y, (and not X-like urls( urls that are not called often), and not Z-like urls (urls that are fast). For this to work varnishtop should (optionally ofcourse) order the url not by request/rate, but rather by "total time spend" on this url. Something like the product of freq and avg time on an url. in this case expensive url != often called backend url expensive url != slow backend url expensive url = backend url with high total time spend (freq x duration) (So the slow ones, that are called often) What do you guys think ? This would instantly give the problem areas of a website, no ? Regards, Harm Verhagen -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Thu Jun 30 08:53:28 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 30 Jun 2011 08:53:28 +0000 Subject: Review for patch to reintroduce logging of TTL changes done in VCL In-Reply-To: Your message of "Tue, 28 Jun 2011 15:56:51 +0200." Message-ID: <90668.1309424008@critter.freebsd.dk> In message , Martin Blix Grydeland writes: Looks good, commit it. (Nit: you split one line at the top of the patch that does not need to be split over two lines) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From drwilco at drwilco.net Wed Jun 29 13:16:28 2011 From: drwilco at drwilco.net (Rogier R. Mulhuijzen) Date: Wed, 29 Jun 2011 15:16:28 +0200 (CEST) Subject: varnishtop feature request: show _expensive_ urls In-Reply-To: References: Message-ID: <20110629143027.E55234@ishtar.drwilco.net> > Currently varnishtop shows the top urls purely based on frequency. Depending > on your usecase that might be usefull, but might also NOT give you the > bottlenecks you are looking for. Agreed. > I'd like to use varnish for the following (example) scenario: ---- SNIP ---- > expensive url = backend url with high total time spend (freq x > duration) (So the slow ones, that are called often) > > What do you guys think ? > This would instantly give the problem areas of a website, no ? I think this would work quite nicely. I would just change the "freq x duration" definition to "cumulative duration". It would mean adding the 5th (or 6th) field from ReqEnd to the counter instead of 1. The hard part is probably (guessing here, have yet to look at v-top source) getting the ReqEnd values to add to the individual counters. Throw it on the shopping list, I'd say. Cheers, Doc