From gslin at gslin.org  Thu Nov  8 20:22:35 2007
From: gslin at gslin.org (Gea-Suan Lin)
Date: Fri, 9 Nov 2007 04:22:35 +0800
Subject: Large file cache problem
Message-ID: <20071108202235.GA62640@gslin.org>

I found that Varnish will download a file from backend, and then send
it to client.  If the file is large, it might take some time to download.
And squid will "stream" it to client.

Is there any configuration I can do with ?

-- 
* Gea-Suan Lin  (public key: Using https://keyserver.pgp.com/ to search)
* If you cannot convince them, confuse them.           -- Harry S Truman


From phk at phk.freebsd.dk  Thu Nov  8 20:45:52 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 08 Nov 2007 20:45:52 +0000
Subject: Large file cache problem 
In-Reply-To: Your message of "Fri, 09 Nov 2007 04:22:35 +0800."
	<20071108202235.GA62640@gslin.org> 
Message-ID: <52803.1194554752@critter.freebsd.dk>

In message <20071108202235.GA62640 at gslin.org>, Gea-Suan Lin writes:
>I found that Varnish will download a file from backend, and then send
>it to client.  If the file is large, it might take some time to download.
>And squid will "stream" it to client.
>
>Is there any configuration I can do with ?

Not yet.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From dave at cheney.net  Tue Nov 13 13:15:46 2007
From: dave at cheney.net (Dave Cheney)
Date: Wed, 14 Nov 2007 00:15:46 +1100
Subject: Recommended branch for multiple backend support
In-Reply-To: <24039.1189617252@critter.freebsd.dk>
References: <24039.1189617252@critter.freebsd.dk>
Message-ID: <5CCDA18F-108B-4D9E-B4DA-CBB4846F6D71@cheney.net>

Hi,

	We currently use 1.1.1 but I'd like to try a newer version that  
includes the Date: fix and also possibly the multiple backend support.  
Which branch (or trunk) would you recommend I use?

Cheers

Dave



From jfbustarret at wat.tv  Wed Nov 14 09:57:48 2007
From: jfbustarret at wat.tv (BUSTARRET, Jean-francois)
Date: Wed, 14 Nov 2007 10:57:48 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
Message-ID: <53C652A09719C54DA24741D0157CB2695FF960@TFPRDEXS1.tf1.groupetf1.fr>

"If the no-cache directive does not specify a field-name, then a cache MUST NOT use the response to satisfy a subsequent request without successful revalidation with the origin server. This allows an origin server to prevent caching even by caches that have been configured to return stale responses to client requests."
 
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.1
 
varnish currently only uses max-age to check the cacheability of objects. This can be done with some vcl code, but IMHO vcl is not the right place.
 
Why not fully support RFC2616 caching ?
 
Jean-Fran?ois Bustarret
WAT - Responsable technique
http://www.wat.tv
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20071114/34257fd8/attachment.html>

From des at linpro.no  Mon Nov 19 15:19:40 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Mon, 19 Nov 2007 16:19:40 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <53C652A09719C54DA24741D0157CB2695FF960@TFPRDEXS1.tf1.groupetf1.fr>
	(Jean-francois BUSTARRET's message of "Wed, 14 Nov 2007 10:57:48
	+0100")
References: <53C652A09719C54DA24741D0157CB2695FF960@TFPRDEXS1.tf1.groupetf1.fr>
Message-ID: <ujrd4u62rg3.fsf@false.linpro.no>

"BUSTARRET, Jean-francois" <jfbustarret at wat.tv> writes:
> varnish currently only uses max-age to check the cacheability of
> objects. This can be done with some vcl code, but IMHO vcl is not
> the right place.

Why not?  Actually, our intention is to move as much policy as
possible (including the interpretation of the Cache-Control header)
into VCL as soon as VCL is able to handle it.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From phk at phk.freebsd.dk  Mon Nov 19 17:20:31 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 19 Nov 2007 17:20:31 +0000
Subject: Support Cache-Control no-cache/private as per RFC2616 ? 
In-Reply-To: Your message of "Mon, 19 Nov 2007 17:50:13 +0100."
	<53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr> 
Message-ID: <4333.1195492831@critter.freebsd.dk>

In message <53C652A09719C54DA24741D0157CB2695FF997 at TFPRDEXS1.tf1.groupetf1.fr>,
 "BUSTARRET, Jean-francois" writes:
>
>Yet http://varnish.projects.linpro.no/wiki/FAQ says "Varnish was written =
>from the ground up to be a high performance caching reverse proxy." =
>Varnish is a cache, and should follow HTTP/1.1 RFCs.

You're welcome to your opinion :-)

We belive we have good arguments for the choices we have made, and
as development continues, it is not inconceiveable that Varnish
might some time in the future grow a "act like an RFC2616 cache",
should somebody code the necessary changes or sponsor somebody
else to do so.

In the mean time, we try to get the maximum bang out of Varnish and
think we have made the correct call on this point.

One thing we have heard severalt times is that content providers
want to be able to use Cache-Control for client instructions and
not penalize their Varnish performance with its settings.

There is a draft floating around which defines a "Surrogate-Control"
along the lines of "Cache-Control" but it seems to have little
backing and even less use.

Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From jfbustarret at wat.tv  Mon Nov 19 16:50:13 2007
From: jfbustarret at wat.tv (BUSTARRET, Jean-francois)
Date: Mon, 19 Nov 2007 17:50:13 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ? 
In-Reply-To: <3936.1195489214@critter.freebsd.dk>
References: Your message of "Fri, 16 Nov 2007 12:23:18
	+0100."<53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr>
	<3936.1195489214@critter.freebsd.dk>
Message-ID: <53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr>


Yet http://varnish.projects.linpro.no/wiki/FAQ says "Varnish was written from the ground up to be a high performance caching reverse proxy." Varnish is a cache, and should follow HTTP/1.1 RFCs.

http://varnish.projects.linpro.no/wiki/HTTPFeatures also references these features :
355	MUST NOT	14.9.1	cache responses with cache-control: private
356	MUST NOT	14.9.1	use responses with cache-control: no-cache to satisfy other requests without successful revalidation

I don't see a problem of having to have the Cache-control logic written in VCL, as long as :
- the default configuration uses it (or the vcl code is in the man pages, not in google). 
- it is as fast as C code (vcl is compiled, so it's ok)
- I don't have to use regexps. For now, I have to use two of them : (obj.http.Pragma ~ "no-cache" || obj.http.Cache-Control ~ "no-cache" - I am not using Cache-control: private), but two strpos-like would do the job, with less overhead.

Jean-Fran?ois

> -----Message d'origine-----
> De : varnish-misc-bounces at projects.linpro.no 
> [mailto:varnish-misc-bounces at projects.linpro.no] De la part 
> de Poul-Henning Kamp
> Envoy? : lundi 19 novembre 2007 17:20
> ? : BUSTARRET, Jean-francois
> Cc : varnish-misc at projects.linpro.no
> Objet : Re: Support Cache-Control no-cache/private as per RFC2616 ? 
> 
> In message 
> <53C652A09719C54DA24741D0157CB2695FF981 at TFPRDEXS1.tf1.groupetf1.fr>,
>  "BUSTARRET, Jean-francois" writes:
> 
> >"If the no-cache directive does not specify a field-name, 
> then a cache 
> >MUST NOT [...]
> 
> Varnish is not a cache in the RFC2616 sense.  It more of an 
> extension of the web-server, which is probably best thought 
> of as "a webserver that picks up its contents with HTTP".
> 
> Therefore, a lot of what RFC2616 has to say about caches do 
> not apply to Varnish.
> 
> The key distinction is that a RFC2616 cache is not under the 
> content providers control, so it must follow whatever 
> intructions he gives in the HTTP headers, whereas Varnish is 
> under his control and he therefore has other means of 
> instruction (notably VCL).
> 
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe    
> Never attribute to malice what can adequately be explained by 
> incompetence.
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
> 


From des at linpro.no  Tue Nov 20 10:23:51 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Tue, 20 Nov 2007 11:23:51 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <4333.1195492831@critter.freebsd.dk> (Poul-Henning Kamp's message
	of "Mon, 19 Nov 2007 17:20:31 +0000")
References: <4333.1195492831@critter.freebsd.dk>
Message-ID: <ujrlk8t9pvs.fsf@false.linpro.no>

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> There is a draft floating around which defines a "Surrogate-Control"
> along the lines of "Cache-Control" but it seems to have little
> backing and even less use.

It has just as much backing and use as ESI; in fact, it is a
prerequisite for ESI.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Tue Nov 20 10:53:18 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Tue, 20 Nov 2007 11:53:18 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr>
	(Jean-francois BUSTARRET's message of "Mon, 19 Nov 2007 17:50:13
	+0100")
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr>
	<3936.1195489214@critter.freebsd.dk>
	<53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr>
Message-ID: <ujrhcjh9oip.fsf@false.linpro.no>

"BUSTARRET, Jean-francois" <jfbustarret at wat.tv> writes:
> Yet http://varnish.projects.linpro.no/wiki/FAQ says "Varnish was
> written from the ground up to be a high performance caching reverse
> proxy." Varnish is a cache, and should follow HTTP/1.1 RFCs.

Excuse me, but who are you to tell us what Varnish is or is not?  Do
you realize how arrogant that is?

That aside, you are trying to fit Varnish into an RFC2616 pigeonhole,
but there is no pigeonhole that fits - RFC2616 did not anticipate
anything like Varnish.  There is a draft W3 specification, the Edge
Architecture Specification, which attempts to fill that hole, but it
is not widely known, so I'm not sure it would help much to write that
Varnish is an HTTP surrogate rather than an HTTP accelerator (I try to
avoid the term "reverse proxy").

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From rudd-o at rudd-o.com  Tue Nov 20 11:30:50 2007
From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O))
Date: Tue, 20 Nov 2007 06:30:50 -0500
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <ujrhcjh9oip.fsf@false.linpro.no>
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr>
	<53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr>
	<ujrhcjh9oip.fsf@false.linpro.no>
Message-ID: <200711200630.56746.rudd-o@rudd-o.com>

Do you really need to snap at contributors, Dag-Erling?  I happen to agree 
with him in the sense that pulling Varnish (VCL or *not*) in the direction of 
a complete standards-compliant configuration-free smart Web accelerator is a 
very good idea.

The grandfather poster may be a bit misguided as to what standards Varnish 
would need to comply to, but your comment would have easily been more 
constructive if you just had limited yourself to the Edge mention.

And, of course, you are already publicly aware of my position regarding the 
matter.

An idea: We need a matrix (OK, a nice table) or a decision tree of what 
actions Varnish should take by default, given a set of requests and content 
of varied freshness degrees.  Once that work is done, moving VCL (and the 
default VCL config) in the direction that will allow us to actually produce 
that decision tree, should be much easier and clearer.

El Mar 20 Nov 2007, Dag-Erling Sm?rgrav escribi?:
> "BUSTARRET, Jean-francois" <jfbustarret at wat.tv> writes:
> > Yet http://varnish.projects.linpro.no/wiki/FAQ says "Varnish was
> > written from the ground up to be a high performance caching reverse
> > proxy." Varnish is a cache, and should follow HTTP/1.1 RFCs.
>
> Excuse me, but who are you to tell us what Varnish is or is not?  Do
> you realize how arrogant that is?
>
> That aside, you are trying to fit Varnish into an RFC2616 pigeonhole,
> but there is no pigeonhole that fits - RFC2616 did not anticipate
> anything like Varnish.  There is a draft W3 specification, the Edge
> Architecture Specification, which attempts to fill that hole, but it
> is not widely known, so I'm not sure it would help much to write that
> Varnish is an HTTP surrogate rather than an HTTP accelerator (I try to
> avoid the term "reverse proxy").
>
> DES



-- 

	Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
	Rudd-O.com - http://rudd-o.com/
	GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/

Now playing, courtesy of Amarok: Voodoo & Serano - Blood is pumping
So this is it.  We're going to die.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20071120/c1f459be/attachment.bin>

From jfbustarret at wat.tv  Tue Nov 20 13:49:16 2007
From: jfbustarret at wat.tv (BUSTARRET, Jean-francois)
Date: Tue, 20 Nov 2007 14:49:16 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <200711200630.56746.rudd-o@rudd-o.com>
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr><53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr><ujrhcjh9oip.fsf@false.linpro.no>
	<200711200630.56746.rudd-o@rudd-o.com>
Message-ID: <53C652A09719C54DA24741D0157CB2695FF99F@TFPRDEXS1.tf1.groupetf1.fr>


I won't reply to the first part of DES message...

I understand the point, and, as I said before, I really like the VCL approach. The real problem is that, for a new varnish user, it is difficult to understand what varnish really is. A new user would read the FAQ and think that varnish is a reverse proxy, but, as said, it isn't (out of the box, I agree that a reverse proxy is only a few lines of VCL code away).

IMHO, this is mainly a documentation problem. Why not :
- remove the term "reverse proxy" from the FAQ and replace it by "HTTP Accelerator",
- describe exactly what/when varnish caches by default,
- describe how to build a RFC2616 reverse proxy, and bundle a sample vcl with varnish.

As Poul-Henning told, someone should contribute/sponsor this. Having more time than money (and feeling not 100% welcome here), I can write some documentation if you agree.

Jean-Fran?ois

> -----Message d'origine-----
> De : varnish-dev-bounces at projects.linpro.no 
> [mailto:varnish-dev-bounces at projects.linpro.no] De la part de 
> Manuel Amador (Rudd-O)
> Envoy? : mardi 20 novembre 2007 12:31
> ? : varnish-dev at projects.linpro.no
> Objet : Re: Support Cache-Control no-cache/private as per RFC2616 ?
> 
> Do you really need to snap at contributors, Dag-Erling?  I 
> happen to agree with him in the sense that pulling Varnish 
> (VCL or *not*) in the direction of a complete 
> standards-compliant configuration-free smart Web accelerator 
> is a very good idea.
> 
> The grandfather poster may be a bit misguided as to what 
> standards Varnish would need to comply to, but your comment 
> would have easily been more constructive if you just had 
> limited yourself to the Edge mention.
> 
> And, of course, you are already publicly aware of my position 
> regarding the matter.
> 
> An idea: We need a matrix (OK, a nice table) or a decision 
> tree of what actions Varnish should take by default, given a 
> set of requests and content of varied freshness degrees.  
> Once that work is done, moving VCL (and the default VCL 
> config) in the direction that will allow us to actually 
> produce that decision tree, should be much easier and clearer.
> 
> El Mar 20 Nov 2007, Dag-Erling Sm?rgrav escribi?:
> > "BUSTARRET, Jean-francois" <jfbustarret at wat.tv> writes:
> > > Yet http://varnish.projects.linpro.no/wiki/FAQ says "Varnish was 
> > > written from the ground up to be a high performance 
> caching reverse 
> > > proxy." Varnish is a cache, and should follow HTTP/1.1 RFCs.
> >
> > Excuse me, but who are you to tell us what Varnish is or is 
> not?  Do 
> > you realize how arrogant that is?
> >
> > That aside, you are trying to fit Varnish into an RFC2616 
> pigeonhole, 
> > but there is no pigeonhole that fits - RFC2616 did not anticipate 
> > anything like Varnish.  There is a draft W3 specification, the Edge 
> > Architecture Specification, which attempts to fill that 
> hole, but it 
> > is not widely known, so I'm not sure it would help much to 
> write that 
> > Varnish is an HTTP surrogate rather than an HTTP 
> accelerator (I try to 
> > avoid the term "reverse proxy").
> >
> > DES
> 
> 
> 
> -- 
> 
> 	Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
> 	Rudd-O.com - http://rudd-o.com/
> 	GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/
> 
> Now playing, courtesy of Amarok: Voodoo & Serano - Blood is 
> pumping So this is it.  We're going to die.
> 


From des at linpro.no  Tue Nov 20 17:12:10 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Tue, 20 Nov 2007 18:12:10 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <200711200630.56746.rudd-o@rudd-o.com> (Manuel Amador's message
	of "Tue, 20 Nov 2007 06:30:50 -0500")
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr>
	<53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr>
	<ujrhcjh9oip.fsf@false.linpro.no>
	<200711200630.56746.rudd-o@rudd-o.com>
Message-ID: <ujrve7w7set.fsf@false.linpro.no>

"Manuel Amador (Rudd-O)" <rudd-o at rudd-o.com> writes:
> Do you really need to snap at contributors, Dag-Erling?

Which contributors?  All I see in this discussion are people trying to
tell me what to do with my (and my employer's) time.  It is one thing
to ask questions about something you do not understand, and quite
another to summarily decide that what you do not understand is wrong,
and must be changed, at somebody else's expense.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Tue Nov 20 17:19:16 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Tue, 20 Nov 2007 18:19:16 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <53C652A09719C54DA24741D0157CB2695FF99F@TFPRDEXS1.tf1.groupetf1.fr>
	(Jean-francois BUSTARRET's message of "Tue, 20 Nov 2007 14:49:16
	+0100")
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr>
	<53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr>
	<ujrhcjh9oip.fsf@false.linpro.no>
	<200711200630.56746.rudd-o@rudd-o.com>
	<53C652A09719C54DA24741D0157CB2695FF99F@TFPRDEXS1.tf1.groupetf1.fr>
Message-ID: <ujrr6ik7s2z.fsf@false.linpro.no>

"BUSTARRET, Jean-francois" <jfbustarret at wat.tv> writes:
> IMHO, this is mainly a documentation problem. Why not :
> - remove the term "reverse proxy" from the FAQ and replace it by "HTTP Accelerator",
> - describe exactly what/when varnish caches by default,
> - describe how to build a RFC2616 reverse proxy, and bundle a sample vcl with varnish.
>
> As Poul-Henning told, someone should contribute/sponsor this. Having
> more time than money (and feeling not 100% welcome here), I can
> write some documentation if you agree.

Documentation is always welcome.  The wiki needs a lot of work, and
both Poul-Henning and I have far too much to do (both with Varnish and
other projects) to be able to spend much time on it.

BTW, could I suggest that you subscribe to varnish-misc or varnish-dev
(or both) so I don't have to manually approve everything you send to
the list?

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From jfbustarret at wat.tv  Tue Nov 20 17:30:51 2007
From: jfbustarret at wat.tv (BUSTARRET, Jean-francois)
Date: Tue, 20 Nov 2007 18:30:51 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <ujrr6ik7s2z.fsf@false.linpro.no>
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr><53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr><ujrhcjh9oip.fsf@false.linpro.no><200711200630.56746.rudd-o@rudd-o.com><53C652A09719C54DA24741D0157CB2695FF99F@TFPRDEXS1.tf1.groupetf1.fr>
	<ujrr6ik7s2z.fsf@false.linpro.no>
Message-ID: <53C652A09719C54DA24741D0157CB2695FF9A4@TFPRDEXS1.tf1.groupetf1.fr>

> -----Message d'origine-----
> De : Dag-Erling Sm?rgrav [mailto:des at linpro.no] 
> Documentation is always welcome.  The wiki needs a lot of 
> work, and both Poul-Henning and I have far too much to do 
> (both with Varnish and other projects) to be able to spend 
> much time on it.

I'll send something next week on the list

> BTW, could I suggest that you subscribe to varnish-misc or 
> varnish-dev (or both) so I don't have to manually approve 
> everything you send to the list?

Sorry for that. I'll never get used to what Exchange does to emails... This should be ok.

Jean-Fran?ois


From jallspaw at yahoo.com  Tue Nov 20 17:24:28 2007
From: jallspaw at yahoo.com (john allspaw)
Date: Tue, 20 Nov 2007 09:24:28 -0800 (PST)
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
Message-ID: <963327.72802.qm@web31802.mail.mud.yahoo.com>

Calling someone arrogant for assuming that Varnish would follow RFC2616 sounds over reactive to me. 

-john

p.s. I follow this list because I expect Varnish to be better than squid for my purposes (very large working set HTTP acceleration, both product and community wise. 

----- Original Message ----
From: Dag-Erling Sm?rgrav <des at linpro.no>
To: Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
Cc: varnish-dev at projects.linpro.no
Sent: Tuesday, November 20, 2007 9:12:10 AM
Subject: Re: Support Cache-Control no-cache/private as per RFC2616 ?


"Manuel Amador (Rudd-O)" <rudd-o at rudd-o.com> writes:
> Do you really need to snap at contributors, Dag-Erling?

Which contributors?  All I see in this discussion are people trying to
tell me what to do with my (and my employer's) time.  It is one thing
to ask questions about something you do not understand, and quite
another to summarily decide that what you do not understand is wrong,
and must be changed, at somebody else's expense.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no
_______________________________________________
varnish-dev mailing list
varnish-dev at projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-dev





      ____________________________________________________________________________________
Be a better pen pal. 
Text or chat with friends inside Yahoo! Mail. See how.  http://overview.mail.yahoo.com/


From des at linpro.no  Tue Nov 20 17:42:20 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Tue, 20 Nov 2007 18:42:20 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <53C652A09719C54DA24741D0157CB2695FF9A4@TFPRDEXS1.tf1.groupetf1.fr>
	(Jean-francois BUSTARRET's message of "Tue, 20 Nov 2007 18:30:51
	+0100")
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr>
	<53C652A09719C54DA24741D0157CB2695FF997@TFPRDEXS1.tf1.groupetf1.fr>
	<ujrhcjh9oip.fsf@false.linpro.no>
	<200711200630.56746.rudd-o@rudd-o.com>
	<53C652A09719C54DA24741D0157CB2695FF99F@TFPRDEXS1.tf1.groupetf1.fr>
	<ujrr6ik7s2z.fsf@false.linpro.no>
	<53C652A09719C54DA24741D0157CB2695FF9A4@TFPRDEXS1.tf1.groupetf1.fr>
Message-ID: <ujrir3w7r0j.fsf@false.linpro.no>

"BUSTARRET, Jean-francois" <jfbustarret at wat.tv> writes:
> Dag-Erling Sm?rgrav <des at linpro.no> writes:
> > BTW, could I suggest that you subscribe to varnish-misc or
> > varnish-dev (or both) so I don't have to manually approve
> > everything you send to the list?
> Sorry for that. I'll never get used to what Exchange does to
> emails... This should be ok.

No, you are posting from a different address than that with which you
subscribed.  The easiest fix is probably to move your subscription
over to the address you are posting from.  Alternatively, you can
subscribe with both addresses and disable one of them so you don't get
everything twice; you will then be able to post to the lists using
either address.  Let me know which you prefer, and I'll take care of
it for you.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From rudd-o at rudd-o.com  Tue Nov 20 17:41:23 2007
From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O))
Date: Tue, 20 Nov 2007 12:41:23 -0500
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <53C652A09719C54DA24741D0157CB2695FF99F@TFPRDEXS1.tf1.groupetf1.fr>
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr>
	<200711200630.56746.rudd-o@rudd-o.com>
	<53C652A09719C54DA24741D0157CB2695FF99F@TFPRDEXS1.tf1.groupetf1.fr>
Message-ID: <200711201241.28401.rudd-o@rudd-o.com>

If you build a description/table of how an RFC2616 cache (I understand that to 
be an HTTP accelerator, but google's kind of slow to come up in this slow 
machine) should act, I promise I will build at least a flowchart.

And your ideas are great!

El Mar 20 Nov 2007, BUSTARRET, Jean-francois escribi?:
> I won't reply to the first part of DES message...
>
> I understand the point, and, as I said before, I really like the VCL
> approach. The real problem is that, for a new varnish user, it is difficult
> to understand what varnish really is. A new user would read the FAQ and
> think that varnish is a reverse proxy, but, as said, it isn't (out of the
> box, I agree that a reverse proxy is only a few lines of VCL code away).
>
> IMHO, this is mainly a documentation problem. Why not :
> - remove the term "reverse proxy" from the FAQ and replace it by "HTTP
> Accelerator", - describe exactly what/when varnish caches by default,
> - describe how to build a RFC2616 reverse proxy, and bundle a sample vcl
> with varnish.
>
> As Poul-Henning told, someone should contribute/sponsor this. Having more
> time than money (and feeling not 100% welcome here), I can write some
> documentation if you agree.
>
> Jean-Fran?ois
>
> > -----Message d'origine-----
> > De : varnish-dev-bounces at projects.linpro.no
> > [mailto:varnish-dev-bounces at projects.linpro.no] De la part de
> > Manuel Amador (Rudd-O)
> > Envoy? : mardi 20 novembre 2007 12:31
> > ? : varnish-dev at projects.linpro.no
> > Objet : Re: Support Cache-Control no-cache/private as per RFC2616 ?
> >
> > Do you really need to snap at contributors, Dag-Erling?  I
> > happen to agree with him in the sense that pulling Varnish
> > (VCL or *not*) in the direction of a complete
> > standards-compliant configuration-free smart Web accelerator
> > is a very good idea.
> >
> > The grandfather poster may be a bit misguided as to what
> > standards Varnish would need to comply to, but your comment
> > would have easily been more constructive if you just had
> > limited yourself to the Edge mention.
> >
> > And, of course, you are already publicly aware of my position
> > regarding the matter.
> >
> > An idea: We need a matrix (OK, a nice table) or a decision
> > tree of what actions Varnish should take by default, given a
> > set of requests and content of varied freshness degrees.
> > Once that work is done, moving VCL (and the default VCL
> > config) in the direction that will allow us to actually
> > produce that decision tree, should be much easier and clearer.
> >
> > El Mar 20 Nov 2007, Dag-Erling Sm?rgrav escribi?:
> > > "BUSTARRET, Jean-francois" <jfbustarret at wat.tv> writes:
> > > > Yet http://varnish.projects.linpro.no/wiki/FAQ says "Varnish was
> > > > written from the ground up to be a high performance
> >
> > caching reverse
> >
> > > > proxy." Varnish is a cache, and should follow HTTP/1.1 RFCs.
> > >
> > > Excuse me, but who are you to tell us what Varnish is or is
> >
> > not?  Do
> >
> > > you realize how arrogant that is?
> > >
> > > That aside, you are trying to fit Varnish into an RFC2616
> >
> > pigeonhole,
> >
> > > but there is no pigeonhole that fits - RFC2616 did not anticipate
> > > anything like Varnish.  There is a draft W3 specification, the Edge
> > > Architecture Specification, which attempts to fill that
> >
> > hole, but it
> >
> > > is not widely known, so I'm not sure it would help much to
> >
> > write that
> >
> > > Varnish is an HTTP surrogate rather than an HTTP
> >
> > accelerator (I try to
> >
> > > avoid the term "reverse proxy").
> > >
> > > DES
> >
> > --
> >
> > 	Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
> > 	Rudd-O.com - http://rudd-o.com/
> > 	GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/
> >
> > Now playing, courtesy of Amarok: Voodoo & Serano - Blood is
> > pumping So this is it.  We're going to die.
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev





-- 

	Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
	Rudd-O.com - http://rudd-o.com/
	GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/

Now playing, courtesy of Amarok: Voodoo & Serano - Blood is pumping
Good day for overcoming obstacles.  Try a steeplechase.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20071120/a4fd9131/attachment.bin>

From des at linpro.no  Tue Nov 20 17:48:15 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Tue, 20 Nov 2007 18:48:15 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <200711201241.28401.rudd-o@rudd-o.com> (Manuel Amador's message
	of "Tue, 20 Nov 2007 12:41:23 -0500")
References: <53C652A09719C54DA24741D0157CB2695FF981@TFPRDEXS1.tf1.groupetf1.fr>
	<200711200630.56746.rudd-o@rudd-o.com>
	<53C652A09719C54DA24741D0157CB2695FF99F@TFPRDEXS1.tf1.groupetf1.fr>
	<200711201241.28401.rudd-o@rudd-o.com>
Message-ID: <ujrabp87qqo.fsf@false.linpro.no>

"Manuel Amador (Rudd-O)" <rudd-o at rudd-o.com> writes:
> If you build a description/table of how an RFC2616 cache (I
> understand that to be an HTTP accelerator, but google's kind of slow
> to come up in this slow machine) should act, I promise I will build
> at least a flowchart.

There is no such thing as an HTTP accelerator in RFC2616.  There are
only private and shared caches.  RFC2616 does not come anywhere near
anticipating the need for something like Varnish (or Squid in reverse
mode for that matter) or describing how it should work.

We already have a table of RFC2616 requirements for shared caches
(lifted from the Squid web site) in the wiki, but it is not directly
applicable to Varnish.

The closest there is to a formal description of what Varnish is and
how it should behave is the Edge Architecture Specification, which
unfortunately is far less impressive than its title.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From phk at phk.freebsd.dk  Tue Nov 20 17:54:34 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 20 Nov 2007 17:54:34 +0000
Subject: Support Cache-Control no-cache/private as per RFC2616 ? 
In-Reply-To: Your message of "Tue, 20 Nov 2007 18:48:15 +0100."
	<ujrabp87qqo.fsf@false.linpro.no> 
Message-ID: <26919.1195581274@critter.freebsd.dk>

In message <ujrabp87qqo.fsf at false.linpro.no>, =?iso-8859-1?Q?Dag-Erling_Sm=F8rg
rav?= writes:

>The closest there is to a formal description of what Varnish is and
>how it should behave is the Edge Architecture Specification, which
>unfortunately is far less impressive than its title.

ESI is indeed not impressive, but it is also aimed at a very narrow
market which Varnish is not (3rd party but non-adverserial caching)

The only truly precise way to characterize varnish, IMO, is "A
webserver that uses HTTP to get at its content".

That description on the other hand, is so general as to be almost
without meaning.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From des at linpro.no  Tue Nov 20 17:58:36 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Tue, 20 Nov 2007 18:58:36 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <26919.1195581274@critter.freebsd.dk> (Poul-Henning Kamp's
	message of "Tue, 20 Nov 2007 17:54:34 +0000")
References: <26919.1195581274@critter.freebsd.dk>
Message-ID: <ujr1wak7q9f.fsf@false.linpro.no>

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> Dag-Erling Sm?rgrav <des at linpro.no> writes:
> > The closest there is to a formal description of what Varnish is
> > and how it should behave is the Edge Architecture Specification,
> > which unfortunately is far less impressive than its title.
> ESI is indeed not impressive, but it is also aimed at a very narrow
> market which Varnish is not (3rd party but non-adverserial caching)

I was not referring to ESI.  The Edge Architecture Specification (EAS)
is the document that defines the term "HTTP Surrogate" and the
Surrogate-Control header.  It is the closest we get to a formal
definition of Varnish's niche in the HTTP ecology, and it isn't much.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From jfbustarret at wat.tv  Tue Nov 20 17:58:46 2007
From: jfbustarret at wat.tv (BUSTARRET, Jean-francois)
Date: Tue, 20 Nov 2007 18:58:46 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ? 
In-Reply-To: <26919.1195581274@critter.freebsd.dk>
References: Your message of "Tue,
	20 Nov 2007 18:48:15 +0100."<ujrabp87qqo.fsf@false.linpro.no>
	<26919.1195581274@critter.freebsd.dk>
Message-ID: <53C652A09719C54DA24741D0157CB2695FF9A6@TFPRDEXS1.tf1.groupetf1.fr>

 

> -----Message d'origine-----
> De : varnish-dev-bounces at projects.linpro.no 
> [mailto:varnish-dev-bounces at projects.linpro.no] De la part de 
> Poul-Henning Kamp
> 
> The only truly precise way to characterize varnish, IMO, is 
> "A webserver that uses HTTP to get at its content".

Why not use "HTTP accelerator" ? It is pretty much self-explanatory, and does not imply anything about caching.

Jean-Fran?ois


From des at linpro.no  Tue Nov 20 18:01:14 2007
From: des at linpro.no (=?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?=)
Date: Tue, 20 Nov 2007 19:01:14 +0100
Subject: Support Cache-Control no-cache/private as per RFC2616 ?
In-Reply-To: <53C652A09719C54DA24741D0157CB2695FF9A6@TFPRDEXS1.tf1.groupetf1.fr>
	(Jean-francois BUSTARRET's message of "Tue, 20 Nov 2007 18:58:46
	+0100")
References: <ujrabp87qqo.fsf@false.linpro.no>
	<26919.1195581274@critter.freebsd.dk>
	<53C652A09719C54DA24741D0157CB2695FF9A6@TFPRDEXS1.tf1.groupetf1.fr>
Message-ID: <ujrwssc6bkl.fsf@false.linpro.no>

"BUSTARRET, Jean-francois" <jfbustarret at wat.tv> writes:
> Why not use "HTTP accelerator" ? It is pretty much self-explanatory,
> and does not imply anything about caching.

That's what I've been trying to use consistently since the beginning
(cf. the front page, the SourceForge public profile, the Freshmeat
profile, the FreeBSD / Debian / RedHat package descriptions etc.)

I still think we should define it, and include a mention of the EAS
somewhere.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From mnot at yahoo-inc.com  Tue Nov 20 22:57:50 2007
From: mnot at yahoo-inc.com (Mark Nottingham)
Date: Wed, 21 Nov 2007 09:57:50 +1100
Subject: Support Cache-Control no-cache/private as per RFC2616 ? 
In-Reply-To: <26919.1195581274@critter.freebsd.dk>
References: <26919.1195581274@critter.freebsd.dk>
Message-ID: <62775B34-B1A2-4A83-84D4-549892FEEB6F@yahoo-inc.com>


I brought up the same issue that the original posted did a while  
back, only to be rebuffed in a similar manner.

I'd suggest that the problem here is one of terminology. Defining  
what Varnish does using HTTP terms is at best murky, so extra care  
needs to be taken so that users aren't misled (intentionally or not).

This situation is what caused the caching industry to start using the  
term "surrogate" about six or seven years ago. See <http:// 
www.mnot.net/papers/wcw5-surrogates.pdf>.

After some initial activity, interest waned, because the different  
players couldn't come to any substantial agreement about what a  
surrogate was, how to control it, etc.; the most substantial effort  
was ESI, and as has been pointed out, it was pretty watered down.

Still, I think it's important to distinguish what Varnish does from a  
"normal" HTTP cache, especially since most other "HTTP Accelerators"  
and "reverse proxies" *do* conform to RFC2616 in most ways, because  
they're usually based upon HTTP proxy caches, rather than written  
from the ground up as a accelerator / surrogate.


On 2007/11/21, at 4:54 AM, Poul-Henning Kamp wrote:
> The only truly precise way to characterize varnish, IMO, is "A
> webserver that uses HTTP to get at its content".

This is a good characterisation. It would probably be more correct to  
say "gateway" than "webserver", but most casual readers won't know  
what that means.

Cheers,

--
Mark Nottingham       mnot at yahoo-inc.com




From phk at phk.freebsd.dk  Tue Nov 20 23:12:49 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 20 Nov 2007 23:12:49 +0000
Subject: Support Cache-Control no-cache/private as per RFC2616 ? 
In-Reply-To: Your message of "Wed, 21 Nov 2007 09:57:50 +1100."
	<62775B34-B1A2-4A83-84D4-549892FEEB6F@yahoo-inc.com> 
Message-ID: <86071.1195600369@critter.freebsd.dk>

In message <62775B34-B1A2-4A83-84D4-549892FEEB6F at yahoo-inc.com>, Mark Nottingha
m writes:

>On 2007/11/21, at 4:54 AM, Poul-Henning Kamp wrote:
>> The only truly precise way to characterize varnish, IMO, is "A
>> webserver that uses HTTP to get at its content".
>
>This is a good characterisation. It would probably be more correct to  
>say "gateway" than "webserver", but most casual readers won't know  
>what that means.

Well, I specifically use "webserver" because Varnish should be
RFC2616 compliant as one, seen from the clients side.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From flo at rfc822.org  Thu Nov 22 15:20:08 2007
From: flo at rfc822.org (Florian Lohoff)
Date: Thu, 22 Nov 2007 16:20:08 +0100
Subject: strange errors
Message-ID: <20071122152008.GB10478@paradigm.rfc822.org>


Hi,
i am a new varnish user and i was trying to speed up serving a debian
and kernel org mirror. The webserver is lighttp 1.4.18-1 varnish
1.0.2-2.

Now i see strange things going on when "apt-get install"ing some 
packages:

Failed to fetch http://ftp2.de.debian.org/debian/pool/main/a/apache2/apache2.2-common_2.2.3-4+etch1_i386.deb Size mismatch
 
Then i had a look at "varnishlog" output - logged a couple of seconds
and found this:

   33 SessionClose c remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed
65535 SessionClose   remote closed

Sometimes the SessionClose repeates for hundret of lines. I
can't see a reason for this.

BTW: Are there any tools/scripts to filter through
the varnishlog output and find potential bugs?

Id like to an error log showing misbehaving clients,
misbehaving server and potential 404's etc ...

Flo
-- 
Florian Lohoff                  flo at rfc822.org             +49-171-2280134
	Those who would give up a little freedom to get a little 
          security shall soon have neither - Benjamin Franklin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20071122/66f2bee3/attachment.bin>

From jfbustarret at wat.tv  Fri Nov 23 16:50:12 2007
From: jfbustarret at wat.tv (BUSTARRET, Jean-francois)
Date: Fri, 23 Nov 2007 17:50:12 +0100
Subject: [varnishd] pipe problems...What is the best option ?
Message-ID: <53C652A09719C54DA24741D0157CB2695FF9C2@TFPRDEXS1.tf1.groupetf1.fr>

A few days ago, I encountered the same "Pipe Shut read/write" problems as described in http://projects.linpro.no/pipermail/varnish-misc/2007-July/000599.html (varnishd 1.1.1 stops answering requests, varnishlog reports lots of Pipe Shut read/write messages).
 
I did try with the last 1.1 code from the SVN, but got random and fairly frequent blank pages. I had then to switch off varnish (and revert to lighty/mod_proxy).
 
My site heavily uses AJAX POST request, which are piped, mixed with GET requests.
 
What would be the best option ? What version would be the best for my case (to be able to pass small POST request & have a good pipe stability) ?
 
Thanks for your help
 
Jean-Fran?ois Bustarret
WAT - Responsable technique
http://www.wat.tv
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20071123/d9a51764/attachment.html>