From r.maccallum at imperial.ac.uk  Mon Mar  1 10:16:25 2010
From: r.maccallum at imperial.ac.uk (Bob MacCallum)
Date: Mon, 1 Mar 2010 10:16:25 +0000
Subject: Response Content Modifications
In-Reply-To: <c3623d231002280928o50f0795dx408b1757e35be7a5@mail.gmail.com>
References: <c3623d231002280600v61c38ae4vaa41bc8e972da44d@mail.gmail.com>
	<3289.1267376789@critter.freebsd.dk>
	<c3623d231002280928o50f0795dx408b1757e35be7a5@mail.gmail.com>
Message-ID: <19339.37881.834006.610440@bio-iisrv1.bio.ic.ac.uk>


Apache's mod_substitute may be what you want.

I use it to add Google analytics javascript to HTML from back end services
(tomcat) served through Apache ProxyPass.  Search for </body> replace with
<script ... ></body>

Just make sure you get a recent version with the memory leak fixed.


Moshe Kaplan writes:
 > No, This is a clear case of system integration,
 > 
 > In many production systems, you cannot change the code itself, since it involves in major risks (when you change a single line of code, you never know where do you end at). However, several methods of reverse proxy can still can be very useful in these systems.
 > 
 > I'll be glad for a reference in the code, and will glad to contribute to the code base if it is needed to accomplish this task.
 > 
 > On Sun, Feb 28, 2010 at 7:06 PM, Poul-Henning Kamp <phk at phk.freebsd.dk<mailto:phk at phk.freebsd.dk>> wrote:
 > In message <c3623d231002280600v61c38ae4vaa41bc8e972da44d at mail.gmail.com<mailto:c3623d231002280600v61c38ae4vaa41bc8e972da44d at mail.gmail.com>>, Moshe
 >  Kaplan writes:
 > 
 > >The target is to modify the received content from the backend (for example
 > >changing embedded URLs in HTML, removing parts of the content based on
 > >regular expressions and so on).
 > >
 > >P.S. I don't have control on the back end content, so I cannot use esi.
 > 
 > Sorry for asking a point blank rude question, but what you describes
 > sounds like theft of copyrighted content from somebody else.
 > 
 > Do you have the necessary legal rights to use the contents on the backend ?
 > 
 > Poul-Henning
 > 
 > --
 > Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
 > phk at FreeBSD.ORG         | TCP/IP since RFC 956
 > FreeBSD committer       | BSD since 4.3-tahoe
 > Never attribute to malice what can adequately be explained by incompetence.
 > 
 > <div dir="ltr">No, This is a clear case of system integration,<br><br>In many production systems, you cannot change the code itself, since it involves in major risks (when you change a single line of code, you never know where do you end at). However, several methods of reverse proxy can still can be very useful in these systems.<br>
 > <br>I&#39;ll be glad for a reference in the code, and will glad to contribute to the code base if it is needed to accomplish this task.<br><br><div class="gmail_quote">On Sun, Feb 28, 2010 at 7:06 PM, Poul-Henning Kamp <span dir="ltr">&lt;<a href="mailto:phk at phk.freebsd.dk">phk at phk.freebsd.dk</a>&gt;</span> wrote:<br>
 > <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">In message &lt;<a href="mailto:c3623d231002280600v61c38ae4vaa41bc8e972da44d at mail.gmail.com">c3623d231002280600v61c38ae4vaa41bc8e972da44d at mail.gmail.com</a>&gt;, Moshe<br>
 > 
 > <div class="im">?Kaplan writes:<br>
 > <br>
 > &gt;The target is to modify the received content from the backend (for example<br>
 > &gt;changing embedded URLs in HTML, removing parts of the content based on<br>
 > &gt;regular expressions and so on).<br>
 > &gt;<br>
 > &gt;P.S. I don&#39;t have control on the back end content, so I cannot use esi.<br>
 > <br>
 > </div>Sorry for asking a point blank rude question, but what you describes<br>
 > sounds like theft of copyrighted content from somebody else.<br>
 > <br>
 > Do you have the necessary legal rights to use the contents on the backend ?<br>
 > <br>
 > Poul-Henning<br>
 > <font color="#888888"><br>
 > --<br>
 > Poul-Henning Kamp ? ? ? | UNIX since Zilog Zeus 3.20<br>
 > phk at FreeBSD.ORG ? ? ? ? | TCP/IP since RFC 956<br>
 > FreeBSD committer ? ? ? | BSD since 4.3-tahoe<br>
 > Never attribute to malice what can adequately be explained by incompetence.<br>
 > </font></blockquote></div><br></div>
 > _______________________________________________
 > varnish-dev mailing list
 > varnish-dev at projects.linpro.no
 > http://projects.linpro.no/mailman/listinfo/varnish-dev

-- 
Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups |
Division of Cell and Molecular Biology | Imperial College London |
Phone +442075941945 | Email r.maccallum at imperial.ac.uk


From mokplan at gmail.com  Mon Mar  1 12:56:00 2010
From: mokplan at gmail.com (Moshe Kaplan)
Date: Mon, 1 Mar 2010 14:56:00 +0200
Subject: Response Content Modifications
In-Reply-To: <3289.1267376789@critter.freebsd.dk>
References: <c3623d231002280600v61c38ae4vaa41bc8e972da44d@mail.gmail.com>
	<3289.1267376789@critter.freebsd.dk>
Message-ID: <c3623d231003010456h64a6fbffl2b93a6e1db1f92@mail.gmail.com>

Hi Phk,

I understand that you are well into Varnish, and that you are familiar with
the internal functions (those that are used in the inline C)
Is there any exposure in the code (C and not VCL) to the C pointer to the
response and object content?
Is it available in the http struct? or in any other struct in the system? or
is it just not available anywhere?

Thanks in advance,
Moshe

On Sun, Feb 28, 2010 at 7:06 PM, Poul-Henning Kamp <phk at phk.freebsd.dk>wrote:

> In message <c3623d231002280600v61c38ae4vaa41bc8e972da44d at mail.gmail.com>,
> Moshe
>  Kaplan writes:
>
> >The target is to modify the received content from the backend (for example
> >changing embedded URLs in HTML, removing parts of the content based on
> >regular expressions and so on).
> >
> >P.S. I don't have control on the back end content, so I cannot use esi.
>
> Sorry for asking a point blank rude question, but what you describes
> sounds like theft of copyrighted content from somebody else.
>
> Do you have the necessary legal rights to use the contents on the backend ?
>
> Poul-Henning
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100301/820eda51/attachment.html>

From phk at phk.freebsd.dk  Mon Mar  1 12:59:59 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 01 Mar 2010 12:59:59 +0000
Subject: Response Content Modifications 
In-Reply-To: Your message of "Mon, 01 Mar 2010 14:56:00 +0200."
	<c3623d231003010456h64a6fbffl2b93a6e1db1f92@mail.gmail.com> 
Message-ID: <60057.1267448399@critter.freebsd.dk>

In message <c3623d231003010456h64a6fbffl2b93a6e1db1f92 at mail.gmail.com>, Moshe K
aplan writes:

>I understand that you are well into Varnish, and that you are familiar with
>the internal functions (those that are used in the inline C)
>Is there any exposure in the code (C and not VCL) to the C pointer to the
>response and object content?
>Is it available in the http struct? or in any other struct in the system? or
>is it just not available anywhere?

It is available, but it is far from trivial to substitute a new content,
because of the various reference counts etc.

I think a more conventional proxy process is much better suited to what
you are trying to do.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From slink at schokola.de  Sun Mar  7 11:41:05 2010
From: slink at schokola.de (Nils Goroll)
Date: Sun, 07 Mar 2010 12:41:05 +0100
Subject: more VTLAs
Message-ID: <4B9390D1.8000805@schokola.de>

I've tried to add some more definitions, maybe someone could review those?

http://varnish-cache.org/wiki/VTLA?action=diff&version=5



From phk at phk.freebsd.dk  Sun Mar  7 12:35:42 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sun, 07 Mar 2010 12:35:42 +0000
Subject: more VTLAs 
In-Reply-To: Your message of "Sun, 07 Mar 2010 12:41:05 +0100."
	<4B9390D1.8000805@schokola.de> 
Message-ID: <32475.1267965342@critter.freebsd.dk>

In message <4B9390D1.8000805 at schokola.de>, Nils Goroll writes:

>I've tried to add some more definitions, maybe someone could review those?
>
>http://varnish-cache.org/wiki/VTLA?action=diff&version=5

good catch.

I have expanded the explations a bit.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From slink at schokola.de  Sun Mar  7 13:56:59 2010
From: slink at schokola.de (Nils Goroll)
Date: Sun, 07 Mar 2010 14:56:59 +0100
Subject: Literal ip4/6 addresses in VCL
Message-ID: <4B93B0AB.4080107@schokola.de>

Hi,

I want to add support for assignment to ip address type variables to the VCL. 
VRT support looks fairly straight forward (convert strings to struct sockaddr * 
using inet_pton), but I am uncertain about a good way to represent literal 
addresses in the VCL.

My main question is if they should be tokenized as CSTRs or rather as new 
tokens, e.g. of token type IP (not to be confused with variable type IP).

I favor the idea to add new token types, and to facilitate lexing, I would 
propose to represent IP addresses as [addr], e.g. [123.45.67.8] or [::1].

For literal addresses, the vcc parser would then generate sockaddr contants, 
using the existing backend code, if possible.

This syntax could also be reused for ACLs and backend definitions by allowing 
masks like in [fc00::]/7 (or rather [fc00::/7] ?) and ports like in 
[123.45.67.8]:8080, but at this point it seems more natural to keep the existing 
internal representation for the acl case (struct acl_e) rather than generalizing 
all cases into one.

Does this sound reasonable, or should I this be done differently?

Thanks, Nils


From phk at phk.freebsd.dk  Sun Mar  7 14:03:46 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sun, 07 Mar 2010 14:03:46 +0000
Subject: Literal ip4/6 addresses in VCL 
In-Reply-To: Your message of "Sun, 07 Mar 2010 14:56:59 +0100."
	<4B93B0AB.4080107@schokola.de> 
Message-ID: <32750.1267970626@critter.freebsd.dk>

In message <4B93B0AB.4080107 at schokola.de>, Nils Goroll writes:

>My main question is if they should be tokenized as CSTRs or rather as new 
>tokens, e.g. of token type IP (not to be confused with variable type IP).

The syntax for this is already decided, just not implemented:

	set client.ip = IP(req.http.x-forwarded-for);

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From slink at schokola.de  Sun Mar  7 14:23:38 2010
From: slink at schokola.de (Nils Goroll)
Date: Sun, 07 Mar 2010 15:23:38 +0100
Subject: Literal ip4/6 addresses in VCL
In-Reply-To: <32750.1267970626@critter.freebsd.dk>
References: <32750.1267970626@critter.freebsd.dk>
Message-ID: <4B93B6EA.9010205@schokola.de>

>> My main question is if they should be tokenized as CSTRs or rather as new 
>> tokens, e.g. of token type IP (not to be confused with variable type IP).
> 
> The syntax for this is already decided, just not implemented:
> 
> 	set client.ip = IP(req.http.x-forwarded-for);

Thank you for pointing this out, so I am going to implement this syntax for the 
case where runtime conversion is needed.

Do you have an opinion regarding the suggestion for the VCL compile time case?

IOW, do you favor

	set client.ip = [123.45.67.8];
or
	set client.ip = "123.45.67.8";

or even no compile time support as in

	set client.ip = IP("123.56.78.9");

Thanks, Nils


From phk at phk.freebsd.dk  Sun Mar  7 14:25:46 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sun, 07 Mar 2010 14:25:46 +0000
Subject: Literal ip4/6 addresses in VCL 
In-Reply-To: Your message of "Sun, 07 Mar 2010 15:23:38 +0100."
	<4B93B6EA.9010205@schokola.de> 
Message-ID: <32818.1267971946@critter.freebsd.dk>

In message <4B93B6EA.9010205 at schokola.de>, Nils Goroll writes:

>> The syntax for this is already decided, just not implemented:
>> 
>> 	set client.ip = IP(req.http.x-forwarded-for);
>
>Thank you for pointing this out, so I am going to implement this syntax for the 
>case where runtime conversion is needed.
>
>Do you have an opinion regarding the suggestion for the VCL compile time case?

There should not be two different syntaxes, the above should be used in
both cases.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From slink at schokola.de  Sun Mar  7 14:27:52 2010
From: slink at schokola.de (Nils Goroll)
Date: Sun, 07 Mar 2010 15:27:52 +0100
Subject: Literal ip4/6 addresses in VCL
In-Reply-To: <32818.1267971946@critter.freebsd.dk>
References: <32818.1267971946@critter.freebsd.dk>
Message-ID: <4B93B7E8.8000001@schokola.de>


> There should not be two different syntaxes, the above should be used in
> both cases.

OK, this is going to make life easier for me. And it's true anyway that we can 
put in optimizations later without changing the VCL syntax. Thanks for your advise!

Nils


From slink at schokola.de  Sun Mar  7 17:03:58 2010
From: slink at schokola.de (Nils Goroll)
Date: Sun, 07 Mar 2010 18:03:58 +0100
Subject: Allocating per-session memory in a director
Message-ID: <4B93DC7E.6050606@schokola.de>

Hi,

is it true that the most appropriate ws to make a per-session memory allocation 
from in a director is sp->http->ws?

Thanks, Nils


From slink at schokola.de  Wed Mar 10 20:42:45 2010
From: slink at schokola.de (Nils Goroll)
Date: Wed, 10 Mar 2010 21:42:45 +0100
Subject: set client.ip etc: AF support?
In-Reply-To: <32750.1267970626@critter.freebsd.dk>
References: <32750.1267970626@critter.freebsd.dk>
Message-ID: <4B980445.8080306@schokola.de>

> 	set client.ip = IP(req.http.x-forwarded-for);

Do we need to support changing the address family between IPV4 and IPv6 for this 
case?

I wonder what will happen if I need to replace the existing sockaddr entry with 
something allocated from sp->http->ws to accommodate an IPv6 address when the 
original client address was v4.



From slink at schokola.de  Thu Mar 11 17:59:58 2010
From: slink at schokola.de (Nils Goroll)
Date: Thu, 11 Mar 2010 18:59:58 +0100
Subject: Event ports fix bug#629
In-Reply-To: <4B5A02EC.5010508@schokola.de>
References: <49A71CB3.5060905@schokola.de>	<2202B2DC-E228-44CF-A7CF-E853AB7B7ECC@omniti.com>
	<4B5A02EC.5010508@schokola.de>
Message-ID: <4B992F9E.5030705@schokola.de>

Hi,

IIUC, no regressions have been found for the fix in 
http://varnish-cache.org/ticket/629 and I am using the fix in production since 
beginning of February now, so I would like to ask if this fix could be 
considered for integration.

Thanks, Nils



From slink at schokola.de  Sat Mar 13 15:37:37 2010
From: slink at schokola.de (Nils Goroll)
Date: Sat, 13 Mar 2010 16:37:37 +0100
Subject: pmatch: back-references to subexpressions of matches for VCL
Message-ID: <4B9BB141.4000700@schokola.de>

It was only after the fact that I realized that I had a bit of bad luck 
regarding the assigned ticket id, but I've implemented back references for the 
match operator in VCL and I'd appreciate comments, reviews or (constructive) 
criticism: http://varnish-cache.org/ticket/666

Thank you, Nils



From slink at schokola.de  Sat Mar 13 15:42:41 2010
From: slink at schokola.de (Nils Goroll)
Date: Sat, 13 Mar 2010 16:42:41 +0100
Subject: 1 of 145 tests failed
In-Reply-To: <3BA21E9E032B22479310705FCAE2A010064F4C8B@DRWUSEXCHANGE.US.DRWHoldings.com>
References: <3BA21E9E032B22479310705FCAE2A010064F4C8B@DRWUSEXCHANGE.US.DRWHoldings.com>
Message-ID: <4B9BB271.1040003@schokola.de>

Hi Donny,

> PASS: ./tests/v00024.vtc
> 
> ===============================================
> 
> 1 of 145 tests failed
> 
> Please report to varnish-dev at projects.linpro.no

Could you please post the error message of the failed test and information about 
your platform?

Otherwise, no-one will be able to help you.

Nils



From cloude at instructables.com  Mon Mar 15 18:22:15 2010
From: cloude at instructables.com (Cloude Porteus)
Date: Mon, 15 Mar 2010 10:22:15 -0800
Subject: Out of socket memory errors?
Message-ID: <4a05e1021003151122h4f7fee5bj78a39aa96ae4342d@mail.gmail.com>

Under a peak load for one of our Varnish servers, we were getting a ton of
502 responses. I found these /var/log/messages. I'm not sure if this is
anything related to our Varnish config causing this or our sys config of the
server.

Has anyone run into this kind of thing before?
------------
Mar 11 11:54:44 squid03 kernel: printk: 23 messages suppressed.
Mar 11 11:54:44 squid03 kernel: Out of socket memory
Mar 11 11:54:49 squid03 kernel: printk: 39 messages suppressed.
Mar 11 11:54:49 squid03 kernel: TCP: too many of orphaned sockets
Mar 11 11:54:57 squid03 kernel: printk: 75 messages suppressed.
Mar 11 11:54:57 squid03 kernel: TCP: too many of orphaned sockets
Mar 11 11:54:59 squid03 kernel: printk: 33 messages suppressed.
Mar 11 11:54:59 squid03 kernel: TCP: too many of orphaned sockets
Mar 11 11:55:05 squid03 kernel: printk: 60 messages suppressed.
Mar 11 11:55:05 squid03 kernel: TCP: too many of orphaned sockets
Mar 11 11:55:10 squid03 kernel: printk: 135 messages suppressed.
Mar 11 11:55:10 squid03 kernel: TCP: too many of orphaned sockets
Mar 11 11:55:14 squid03 kernel: printk: 87 messages suppressed.
Mar 11 11:55:14 squid03 kernel: TCP: too many of orphaned sockets
Mar 11 11:55:19 squid03 kernel: printk: 61 messages suppressed.
-----------

best,
cloude
-- 
VP of Product Development
Instructables.com

http://www.instructables.com/member/lebowski
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100315/fb93f115/attachment.html>

From jayasingh.samuel at hotmail.com  Tue Mar 16 08:55:54 2010
From: jayasingh.samuel at hotmail.com (Jaysingh Samuel)
Date: Tue, 16 Mar 2010 14:25:54 +0530
Subject: How to debug Varnish code using gdb.
Message-ID: <SNT134-w3D962E6C259D37FF3F4908A2D0@phx.gbl>


Hi All, 
Iam working on ESI part of Varnish, and to understand the code flow, i was trying to use gdb debugger, but didn't have any luck so far.  
Please help me out in understanding how to debug Varnish source code using gdb.
I checked with the Make i could find that it compiled with -g Option, but still face these errors. 
dev03:/home/varnish/varnish/sbin # gdb ./varnishd GNU gdb 6.4Copyright 2005 Free Software Foundation, Inc.GDB is free software, covered by the GNU General Public License, and you arewelcome to change it and/or distribute copies of it under certain conditions.Type "show copying" to see the conditions.There is absolutely no warranty for GDB.  Type "show warranty" for details.This GDB was configured as "x86_64-suse-linux"...Using host libthread_db library "/lib64/libthread_db.so.1".
(gdb) b esi_handle_include Breakpoint 1 at 0x432533: file cache_vrt_esi.c, line 351.(gdb) r -T 127.0.0.1:200 -a 0.0.0.0:2030 -f /home/varnish/varnish/etc/varnish/varnish.vcl -u varnishStarting program: /home/varnish/varnish/sbin/varnishd -T 127.0.0.1:200 -a 0.0.0.0:2030 -f /home/varnish/varnish/etc/varnish/varnish.vcl -u varnish[Thread debugging using libthread_db enabled][New Thread 47413511004080 (LWP 16317)]Error while mapping shared library sections:./vcl.1P9zoqAU.so: No such file or directory.Error while reading shared library symbols:./vcl.1P9zoqAU.so: No such file or directory.Error while reading shared library symbols:./vcl.1P9zoqAU.so: No such file or directory.storage_file: filename: ./varnish.bjsqBo (unlinked) size 8921 MB.Using old SHMFILE
Program exited normally.(gdb)  Thanks in advance, Jaysingh 		 	   		  
_________________________________________________________________

Green cities and the melting world
http://green.in.msn.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100316/87fee81f/attachment.html>

From frank.gruellich at mapsolute.com  Tue Mar 23 11:02:56 2010
From: frank.gruellich at mapsolute.com (Frank Gruellich)
Date: Tue, 23 Mar 2010 11:02:56 +0000
Subject: Ignoring health checks in statistics
Message-ID: <4BA89FE0.8090406@mapsolute.com>

Hi,

we have two Varnish instances and a load balancer in front.  The load
balancer periodically polls the varnishd's for a static file (lying also
at all the backend servers) to check if varnishd is still working.  It
does that once per second.  This completely messes up statistics like
hit ratio because that static file is served from cache all the time, so
currently our hit ratio is close to 99.9%.  I would like to have counted
this specific file neither as miss nor hit, but just serve it and ignore
it otherwise.  It doesn't even need to be logged.

haproxy has an option monitor-net where you can specify that it does not
need to log requests from that subnet.  Does varnish offer a similar
option?  Or is there some cool VCL snippet I could add?

Thanks in advance.

Kind regards,
-- 
Navteq (DE) GmbH
Frank Gruellich
Map24 Systems and Networks

Duesseldorfer Strasse 40a
65760 Eschborn
Germany

Phone:      +49 6196 77756-414
Fax:        +49 6196 77756-100

HRB 46215, Local Court Frankfurt am Main
Managing Directors: Thomas Golob, Hans Pieter Gieszen, Martin Robert Stockman
USt-ID-No.: DE 197947163



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100323/9432fef2/attachment.bin>

From phk at phk.freebsd.dk  Tue Mar 23 11:22:58 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 23 Mar 2010 11:22:58 +0000
Subject: Ignoring health checks in statistics 
In-Reply-To: Your message of "Tue, 23 Mar 2010 11:02:56 GMT."
	<4BA89FE0.8090406@mapsolute.com> 
Message-ID: <50505.1269343378@critter.freebsd.dk>

In message <4BA89FE0.8090406 at mapsolute.com>, Frank Gruellich writes:

>haproxy has an option monitor-net where you can specify that it does not
>need to log requests from that subnet.  Does varnish offer a similar
>option?  Or is there some cool VCL snippet I could add?

Sorry, not at this time.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From scaunter at topscms.com  Tue Mar 23 12:01:22 2010
From: scaunter at topscms.com (Caunter, Stefan)
Date: Tue, 23 Mar 2010 08:01:22 -0400
Subject: Ignoring health checks in statistics
Message-ID: <064FF286FD17EC418BFB74629578BED1159E24D9@tmg-mail4.torstar.net>

I put the answer to satisfy the load balancer in the error section. Instead of serving the static file, put the text as part of the 404 message and remove the file altogether. 

Stef
--------------------------
Sent using BlackBerry
416 561 4871

----- Original Message -----
From: varnish-dev-bounces at varnish-cache.org <varnish-dev-bounces at varnish-cache.org>
To: varnish-dev at varnish-cache.org <varnish-dev at varnish-cache.org>
Sent: Tue Mar 23 07:02:56 2010
Subject: Ignoring health checks in statistics

Hi,

we have two Varnish instances and a load balancer in front.  The load
balancer periodically polls the varnishd's for a static file (lying also
at all the backend servers) to check if varnishd is still working.  It
does that once per second.  This completely messes up statistics like
hit ratio because that static file is served from cache all the time, so
currently our hit ratio is close to 99.9%.  I would like to have counted
this specific file neither as miss nor hit, but just serve it and ignore
it otherwise.  It doesn't even need to be logged.

haproxy has an option monitor-net where you can specify that it does not
need to log requests from that subnet.  Does varnish offer a similar
option?  Or is there some cool VCL snippet I could add?

Thanks in advance.

Kind regards,
-- 
Navteq (DE) GmbH
Frank Gruellich
Map24 Systems and Networks

Duesseldorfer Strasse 40a
65760 Eschborn
Germany

Phone:      +49 6196 77756-414
Fax:        +49 6196 77756-100

HRB 46215, Local Court Frankfurt am Main
Managing Directors: Thomas Golob, Hans Pieter Gieszen, Martin Robert Stockman
USt-ID-No.: DE 197947163



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100323/f1fe01bb/attachment.html>

From scaunter at topscms.com  Tue Mar 23 14:11:26 2010
From: scaunter at topscms.com (Caunter, Stefan)
Date: Tue, 23 Mar 2010 10:11:26 -0400
Subject: Ignoring health checks in statistics
In-Reply-To: <4BA89FE0.8090406@mapsolute.com>
References: <4BA89FE0.8090406@mapsolute.com>
Message-ID: <064FF286FD17EC418BFB74629578BED117DA7796@tmg-mail4.torstar.net>

Just put this in sub vcl_recv

else {
                 error 404 "I am OK";
     }


Stefan Caunter :: Senior Systems Administrator :: TOPS
e: scaunter at topscms.com  ::  m: (416) 561-4871
www.thestar.com www.topscms.com

-----Original Message-----
From: varnish-dev-bounces at varnish-cache.org
[mailto:varnish-dev-bounces at varnish-cache.org] On Behalf Of Frank
Gruellich
Sent: March-23-10 7:03 AM
To: varnish-dev at varnish-cache.org
Subject: Ignoring health checks in statistics

Hi,

we have two Varnish instances and a load balancer in front.  The load
balancer periodically polls the varnishd's for a static file (lying also
at all the backend servers) to check if varnishd is still working.  It
does that once per second.  This completely messes up statistics like
hit ratio because that static file is served from cache all the time, so
currently our hit ratio is close to 99.9%.  I would like to have counted
this specific file neither as miss nor hit, but just serve it and ignore
it otherwise.  It doesn't even need to be logged.

haproxy has an option monitor-net where you can specify that it does not
need to log requests from that subnet.  Does varnish offer a similar
option?  Or is there some cool VCL snippet I could add?

Thanks in advance.

Kind regards,
--
Navteq (DE) GmbH
Frank Gruellich
Map24 Systems and Networks

Duesseldorfer Strasse 40a
65760 Eschborn
Germany

Phone:      +49 6196 77756-414
Fax:        +49 6196 77756-100

HRB 46215, Local Court Frankfurt am Main Managing Directors: Thomas
Golob, Hans Pieter Gieszen, Martin Robert Stockman
USt-ID-No.: DE 197947163






From slink at schokola.de  Thu Mar 25 15:46:12 2010
From: slink at schokola.de (Nils Goroll)
Date: Thu, 25 Mar 2010 16:46:12 +0100
Subject: How to store per-session director data
Message-ID: <4BAB8544.2050302@schokola.de>

Hi,

I am working on a new director which needs per-session state.

At this point, I have added a pointer to the session state:

struct sess {
        unsigned                magic;
#define SESS_MAGIC              0x2c2f9c5a

	...

        /* chosen director and per-director session data */
        struct director         *director;
        struct director_sess    *director_sess;

Right now, space for director_sess is allocated at runtime when needed (from
sp->http->ws), but it would probably be more efficient to simply embed it in
struct sess.

To support session state for other directors, I see some alternatives:

* simply add elements needed for additional directors to director_sess
* make director_sess a void * to some private, run time allocated data
* make it a union

I don't want to make this unnecessary complex, but I would like to chose a
design supported by the core developers.

Any advise on this?

Nils



From phk at phk.freebsd.dk  Thu Mar 25 16:09:30 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 25 Mar 2010 16:09:30 +0000
Subject: How to store per-session director data 
In-Reply-To: Your message of "Thu, 25 Mar 2010 16:46:12 +0100."
	<4BAB8544.2050302@schokola.de> 
Message-ID: <2126.1269533370@critter.freebsd.dk>

In message <4BAB8544.2050302 at schokola.de>, Nils Goroll writes:
>Hi,
>
>I am working on a new director which needs per-session state.
>
>Any advise on this?

It would probably be a good idea if you started out by telling
me what you are trying to do here...

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From slink at schokola.de  Thu Mar 25 16:59:37 2010
From: slink at schokola.de (Nils Goroll)
Date: Thu, 25 Mar 2010 17:59:37 +0100
Subject: How to store per-session director data / VSLP director design
In-Reply-To: <2126.1269533370@critter.freebsd.dk>
References: <2126.1269533370@critter.freebsd.dk>
Message-ID: <4BAB9679.5080808@schokola.de>

Hi Poul-Henning,

>> I am working on a new director which needs per-session state.
>>
>> Any advise on this?
> 
> It would probably be a good idea if you started out by telling
> me what you are trying to do here...

Oh, yes, absolutely, sorry.

I've written something I would like to call the "Varnish StateLess Persistence"
(VSLP) director: It generalizes some aspects of the hash director and something
I've written before (but not published).

Comment from bin/varnishd/cache_dir_vslp.c:

 * VSLP*) Director - Varnish StateLess Persistence Director
 *
 * The purpose of this director is to provide stateless persistence (or
 * persistency if you prefer) for backend requests based upon this simple
 * concept:
 *
 * - Generate a load balancing key, which will be used to select the backend
 *
 *   The key values should be as uniformly distributed as possible.
 *
 *   For all requests which need to hit the same backend server, the same
 *   key must be generated.
 *
 *   For strings, a hash function is used to generate the key.
 *
 * - Select the preferred backend server as (key mod nservers), nservers being
 *   the number of backend servers for this director.
 *
 * - (Only) If the preferred backend server becomes unavailable, an alternative
 *   backend server is selected using the same method from the set of healhy
 *   servers.
 *
 * This technique allowes to implement persistence without keeping any state,
 * and, in particular, without the need to synchronize state between nodes of a
 * cluster of Varnish servers.
 *
 * Still, all members of a cluster will, if otherwise configured equally,
 * implement the same persistence, in other words, requests sharing a common
 * persistence criterium will be balanced onto the same backend server no matter
 * which Varnish server handles the request.
 *
 * The drawbacks are:
 *
 * - the distribution of requests depends on the number of requests per key and
 *   the uniformity of the distribution of key values. In short, this technique
 *   will generally lead to less good load balancing compared to stateful
 *   techniques.
 *
 * - When a backend server becomes unavailable, every persistence technique has
 *   to reselect a new backend server, but this technique will also switch back
 *   to the preferred server once it becomes healthy again, so the persistence
 *   is generally less stable compared to stateful techniques.
 *
 * - Persistence will generally be lost when the number of hosts configured for
 *   the director is changed (so the configuration should be kept stable).
 *
 * *) The name VSLP was chosen to accompany VTLA as another VETR (Varnish
 *    Exception To the Rule) ;-) http://varnish-cache.org/wiki/VTLA

The purpose I wrote this director for requires the flexibility to chose the
persistence key either directly, generate it from an IP address or from a string
by means of a hash value. The main point is really that it must be possible to
generate the key from some arbitrary data.

Effectively, this resembles the client.id which had been proposed before, and I
would also be happy to chose that notion instead if that's preferred.

Another requirement is to be able to change the hash function for strings on the
fly.

As both the key and the hash function are to be changed in the VCL, I suppose
they should best be linked to the session, so my struct director_sess currently
looks like this:

struct director_sess {
        unsigned                magic;
#define DIRECTOR_SESS_MAGIC     0xd36e6208 /* crc32 */

        /* stateless persistence */
        uint32_t                persistence_key;
        enum vbe_digest_e       digest;
};

In my current implementation, the key can be set using some write-only accessors
and read as an integer value:

# (varnish 2.0 vcc_gen_obj.tcl)

    # XXX: stateless persistence director requirement:

    # XXX: we want to be able to use "any" value as a key, so ideally
    #      we should have some kind of lookahead in the parser and be
    #      able to specify different assignment funktions to overload
    #      assignment / argument passing.
    #      but I don't see how this can be done within the current
    #      framework, so I work around by defining different WO and RO
    #      objects, all accessing the same data
    { dir.key.ip
        WO IP
        {recv pipe pass hash miss hit fetch                        error }
        "struct sess *"
    }
    { dir.key.string
        WO STRING
        {recv pipe pass hash miss hit fetch                        error }
        "struct sess *"
    }
    { dir.key.int
        WO UINT32
        {recv pipe pass hash miss hit fetch                        error }
        "struct sess *"
    }
    # However the key gets set, it is always really an int
    { dir.key
        RO UINT32
        {recv pipe pass hash miss hit fetch                        error }
        "struct sess *"
    }
    # digest alogrithm selection
    { dir.digest
        RW DIGEST
        {recv pipe pass hash miss hit fetch                        error }
        "struct sess *"
    }

Again, I'd really appreciate your advice.

Nils



From peterjanovsky at yahoo.com  Thu Mar 25 19:39:50 2010
From: peterjanovsky at yahoo.com (Peter Janovsky)
Date: Thu, 25 Mar 2010 12:39:50 -0700 (PDT)
Subject: digest
Message-ID: <429157.70343.qm@web51907.mail.re2.yahoo.com>

SET varnish-dev DIGEST


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100325/cad3faa6/attachment.html>

From peterjanovsky at yahoo.com  Thu Mar 25 20:03:50 2010
From: peterjanovsky at yahoo.com (Peter Janovsky)
Date: Thu, 25 Mar 2010 13:03:50 -0700 (PDT)
Subject: varnishd -lssl
Message-ID: <281680.16779.qm@web51905.mail.re2.yahoo.com>

i have a shared library which uses the openssl libraries to perform HMAC-SHA1 and Base64 encoding.  varnishd doesn't allow me to include the -lssl option for my implementation of "default.vcl" registering my shared library.

i'm new to varnish so i may have overlooked critical documentation.  please point me in the direction of documentation pertaining to modifying the varnish gcc build commands.  is it possible to include additional gcc build commands?  thnx in advance....

peter



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100325/c3351bb9/attachment.html>

From tfheen at varnish-software.com  Fri Mar 26 08:06:03 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Fri, 26 Mar 2010 09:06:03 +0100
Subject: varnishd -lssl
In-Reply-To: <281680.16779.qm@web51905.mail.re2.yahoo.com> (Peter Janovsky's
	message of "Thu, 25 Mar 2010 13:03:50 -0700 (PDT)")
References: <281680.16779.qm@web51905.mail.re2.yahoo.com>
Message-ID: <87bpebjyms.fsf@qurzaw.linpro.no>

]] Peter Janovsky 

| i'm new to varnish so i may have overlooked critical documentation.
| please point me in the direction of documentation pertaining to
| modifying the varnish gcc build commands.  is it possible to include
| additional gcc build commands?  thnx in advance....

You can set the exec_cc parameter and add the necessary -l arguments
there.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 54 41 73



From phk at phk.freebsd.dk  Fri Mar 26 08:56:46 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 26 Mar 2010 08:56:46 +0000
Subject: How to store per-session director data / VSLP director design 
In-Reply-To: Your message of "Thu, 25 Mar 2010 17:59:37 +0100."
	<4BAB9679.5080808@schokola.de> 
Message-ID: <6322.1269593806@critter.freebsd.dk>

In message <4BAB9679.5080808 at schokola.de>, Nils Goroll writes:

>I've written something I would like to call the "Varnish StateLess Persistence"
>(VSLP) director: It generalizes some aspects of the hash director and something
>I've written before (but not published).

Don't you overlook the fact that the active VCL may change between
two requests on the same session ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From slink at schokola.de  Fri Mar 26 11:03:48 2010
From: slink at schokola.de (Nils Goroll)
Date: Fri, 26 Mar 2010 12:03:48 +0100
Subject: How to store per-session director data / VSLP director design
In-Reply-To: <6322.1269593806@critter.freebsd.dk>
References: <6322.1269593806@critter.freebsd.dk>
Message-ID: <4BAC9494.40909@schokola.de>

Hi Poul-Henning,

>> I've written something I would like to call the "Varnish StateLess Persistence"
>> (VSLP) director: It generalizes some aspects of the hash director and something
>> I've written before (but not published).
> 
> Don't you overlook the fact that the active VCL may change between
> two requests on the same session ?

Hm. At this point I don't understand what I'd need to consider regarding a
change of the VCL. The current implementation clears sp->director_sess wherever
sp->director is cleared, in other words between requests. If there is anything
else I have overlooked?

As the question of where to put the per-director session state is the one which
I am least happy with at this point, I would really appreciate if that was
answered before publishing the code (which I definitely will do).

Thank you, Nils



From phk at phk.freebsd.dk  Fri Mar 26 11:24:11 2010
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 26 Mar 2010 11:24:11 +0000
Subject: How to store per-session director data / VSLP director design 
In-Reply-To: Your message of "Fri, 26 Mar 2010 12:03:48 +0100."
	<4BAC9494.40909@schokola.de> 
Message-ID: <20056.1269602651@critter.freebsd.dk>

In message <4BAC9494.40909 at schokola.de>, Nils Goroll writes:
>Hi Poul-Henning,
>
>> Don't you overlook the fact that the active VCL may change between
>> two requests on the same session ?
>
>Hm. At this point I don't understand what I'd need to consider regarding a
>change of the VCL.

You may not have the same VCL code, directors or backends for all requests
on a given session.

That opens a number of cans of worms:

1. You cannot cache pointers to any of these bits between requests,
   unless as an opaque token (see uintptr_t use) that does not get
   dereferenced.

2. Memory allocated, needs to be freed.  A different VCL may not know
   how to do that.  Not that VCL is consulted about session retirement
   in the first place anyway.


My plan for the functionality you propose, is to add a "client.identity"
string variable you could set in vcl_recv{} and let the client
director operate on that string.  If the string is not set, client.ip
will be used, like today.

You get to decide what to put in the string: you can pull it out
of a cookie, X-F-F, from the URL or whatever.


I have also pondered a scheme where we cache an opaque cookie to
the last backend used in the session, and let directors attempt to
use this to gain affinity, since that could provide some stability
across a director modification (in a new VCL).

This is likely a lot less useful than you think, as many sites have
a separate backend for images and other primitive static content.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From slink at schokola.de  Fri Mar 26 12:31:28 2010
From: slink at schokola.de (Nils Goroll)
Date: Fri, 26 Mar 2010 13:31:28 +0100
Subject: How to store per-session director data / VSLP director design
In-Reply-To: <20056.1269602651@critter.freebsd.dk>
References: <20056.1269602651@critter.freebsd.dk>
Message-ID: <4BACA920.9060603@schokola.de>

Hi,

>>> Don't you overlook the fact that the active VCL may change between
>>> two requests on the same session ?
>>
>> Hm. At this point I don't understand what I'd need to consider regarding a
>> change of the VCL.
> 
> You may not have the same VCL code, directors or backends for all requests
> on a given session.
> 
> That opens a number of cans of worms:

Thank you for the explanations.

Regarding state, I didn't plan to keep any state across requests, I hope I am
not doing this in error, and I'll check the changes regarding this aspect.

Regarding memory, so far I'm taking allocations from sp->http->ws, not knowing
of any better way within the current framework. As proposed in my OP, I favor
the idea to embed the state into struct session to avoid dynamic allocation in
the first place.

Do you have an opinion about this?

Thinking again about these aspects, for some use cases it would probably be more
efficient to cache state across requests (for the duration of a session), but
the purpose I developed this for actually does require a load
balancing/persistence decision per request anyway.

> My plan for the functionality you propose, is to add a "client.identity"
> string variable you could set in vcl_recv{} and let the client
> director operate on that string.  If the string is not set, client.ip
> will be used, like today.

Yes, I quite liked this concept, but I find it is tied too closely to the idea
of client persistence. There are two reasons why I prefer something like a "load
balancing key" (or maybe persistence key is a better name):

- Conceptually, it makes clear that persistence can be defined upon anything,
not just something client specific (the client.identity could be used anyway, sure)

- The purpose I developed this for requires that the key can be stored in the
client state (Cookie), so making hashing optional is just a natural solution to
extract it again.

> I have also pondered a scheme where we cache an opaque cookie to
> the last backend used in the session, and let directors attempt to
> use this to gain affinity, since that could provide some stability
> across a director modification (in a new VCL).

Agree. For the time being, I believe the simple solution will do, but you're
definitely right that the lack of stability is an issue as long as we don't have
a proper way to keep (and distribute) state.

I must say though that once we start talking about stateful persistence, I favor
the idea of actually distributing state across a cluster of varnish servers, for
instance by using something like memcache and to keep that also across sessions.

IMHO, storing persistency information in a cookie can only be an intermediate
solution because we can't influence load balancing of other requests to other
servers which are in flight at the same time.

> This is likely a lot less useful than you think, as many sites have
> a separate backend for images and other primitive static content.

I don't think I understand what you mean.

Poul-Henning, I really appreciate this fruitful discussion, but could I please
ask you to let me know what you think about the director session state thing?

Cheers, Nils



From peterjanovsky at yahoo.com  Fri Mar 26 16:59:32 2010
From: peterjanovsky at yahoo.com (Peter Janovsky)
Date: Fri, 26 Mar 2010 09:59:32 -0700 (PDT)
Subject: varnishd -lssl
In-Reply-To: <87bpebjyms.fsf@qurzaw.linpro.no>
References: <281680.16779.qm@web51905.mail.re2.yahoo.com>
	<87bpebjyms.fsf@qurzaw.linpro.no>
Message-ID: <977710.38753.qm@web51907.mail.re2.yahoo.com>

tollef,

    please excuse my ignorance as i haven't worked in C for a while.  where would i find the exec_cc parameter?  searching google it appears the parameter is part of the makefile.  i searched the varnish makefiles but didn't come across the parameter.  is this a parameter i need to add to one of the makefiles, if so which makefile?  thank you.

peter





________________________________
From: Tollef Fog Heen <tfheen at varnish-software.com>
To: Peter Janovsky <peterjanovsky at yahoo.com>
Cc: Varnish Development <varnish-dev at varnish-cache.org>
Sent: Fri, March 26, 2010 4:06:03 AM
Subject: Re: varnishd -lssl

]] Peter Janovsky 

| i'm new to varnish so i may have overlooked critical documentation.
| please point me in the direction of documentation pertaining to
| modifying the varnish gcc build commands.  is it possible to include
| additional gcc build commands?  thnx in advance....

You can set the exec_cc parameter and add the necessary -l arguments
there.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 54 41 73



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100326/f07d78c2/attachment.html>

From tfheen at varnish-software.com  Mon Mar 29 07:26:10 2010
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Mon, 29 Mar 2010 09:26:10 +0200
Subject: varnishd -lssl
In-Reply-To: <977710.38753.qm@web51907.mail.re2.yahoo.com> (Peter Janovsky's
	message of "Fri, 26 Mar 2010 09:59:32 -0700 (PDT)")
References: <281680.16779.qm@web51905.mail.re2.yahoo.com>
	<87bpebjyms.fsf@qurzaw.linpro.no>
	<977710.38753.qm@web51907.mail.re2.yahoo.com>
Message-ID: <878w9ba8rx.fsf@qurzaw.linpro.no>

]] Peter Janovsky 

|     please excuse my ignorance as i haven't worked in C for a while.
|     where would i find the exec_cc parameter?  searching google it
|     appears the parameter is part of the makefile.  i searched the
|     varnish makefiles but didn't come across the parameter.  is this a
|     parameter i need to add to one of the makefiles, if so which
|     makefile?  thank you.

You can set it using -p exec_cc="gcc ?"

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 54 41 73



From peterjanovsky at yahoo.com  Mon Mar 29 20:05:49 2010
From: peterjanovsky at yahoo.com (Peter Janovsky)
Date: Mon, 29 Mar 2010 13:05:49 -0700 (PDT)
Subject: varnishd -lssl
In-Reply-To: <878w9ba8rx.fsf@qurzaw.linpro.no>
References: <281680.16779.qm@web51905.mail.re2.yahoo.com>
	<87bpebjyms.fsf@qurzaw.linpro.no>
	<977710.38753.qm@web51907.mail.re2.yahoo.com>
	<878w9ba8rx.fsf@qurzaw.linpro.no>
Message-ID: <946581.13813.qm@web51906.mail.re2.yahoo.com>

great.  thnx for your help thus far.  i'm in the midst of tackling the compilation issue.  from your response and researching the web i've come across a few questions referring to the cc_command parameter and linking/compiling the shared library.  here is what i have done thus far

1.  created a shared library
2.  installed the shared library under /usr/lib (additionally i tried /usr/local/lib)
3.  ran /sbin/ldconfig

i found the following example on the web for the cc_command parameter

/usr/sbin/varnishd -C -f /etc/varnish/default.vcl -p cc_command="exec cc -fpic -shared -Wl,-x -o %o %s -lssl"

when running this command i receive an error indicating an undefined symbol which references a function within the openssl library.  i've changed the position of the -lssl option within the cc_command parameter to determine if it has any effect.  it doesn't.

i've also attempted using the -L and -l options to specify the shared library's location and soname, still no progress.

question is:

when you have a shared library which requires special gcc command line options to compile which syntax shall i use for the cc_command parameter, the syntax i am using or a variation of the syntax?

thnx for all your help. - peter




________________________________
From: Tollef Fog Heen <tfheen at varnish-software.com>
To: Peter Janovsky <peterjanovsky at yahoo.com>
Cc: Varnish Development <varnish-dev at varnish-cache.org>
Sent: Mon, March 29, 2010 3:26:10 AM
Subject: Re: varnishd -lssl

]] Peter Janovsky 

|     please excuse my ignorance as i haven't worked in C for a while.
|     where would i find the exec_cc parameter?  searching google it
|     appears the parameter is part of the makefile.  i searched the
|     varnish makefiles but didn't come across the parameter.  is this a
|     parameter i need to add to one of the makefiles, if so which
|     makefile?  thank you.

You can set it using -p exec_cc="gcc ?"

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 54 41 73



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100329/0cde0222/attachment.html>

From slink at schokola.de  Tue Mar 30 08:28:07 2010
From: slink at schokola.de (Nils Goroll)
Date: Tue, 30 Mar 2010 10:28:07 +0200
Subject: varnishd -lssl
In-Reply-To: <946581.13813.qm@web51906.mail.re2.yahoo.com>
References: <281680.16779.qm@web51905.mail.re2.yahoo.com>	<87bpebjyms.fsf@qurzaw.linpro.no>	<977710.38753.qm@web51907.mail.re2.yahoo.com>	<878w9ba8rx.fsf@qurzaw.linpro.no>
	<946581.13813.qm@web51906.mail.re2.yahoo.com>
Message-ID: <4BB1B617.8080800@schokola.de>

Hi Peter,

> i've also attempted using the -L and -l options to specify the shared
> library's location and soname, still no progress.

Does -R help to specify the runtime linker path?

Otherwise I suggest you post specific error messages and the exact
commands/settings you're using.

Nils



From peterjanovsky at yahoo.com  Wed Mar 31 15:34:37 2010
From: peterjanovsky at yahoo.com (Peter Janovsky)
Date: Wed, 31 Mar 2010 08:34:37 -0700 (PDT)
Subject: varnishd -lssl
In-Reply-To: <4BB1B617.8080800@schokola.de>
References: <281680.16779.qm@web51905.mail.re2.yahoo.com>
	<87bpebjyms.fsf@qurzaw.linpro.no>
	<977710.38753.qm@web51907.mail.re2.yahoo.com>
	<878w9ba8rx.fsf@qurzaw.linpro.no>
	<946581.13813.qm@web51906.mail.re2.yahoo.com>
	<4BB1B617.8080800@schokola.de>
Message-ID: <37972.723.qm@web51906.mail.re2.yahoo.com>

thnx for all your help.  figured out the issue.  it was a problem with locating the shared library.  i went back to scratch, rebuilt the library and ensured it was properly loaded and could be located by ld.  the following command worked like a charm when compiling my vcl

/usr/sbin/varnishd -C -f /etc/varnish/default.vcl -p cc_command="exec cc -L /usr/lib/FULL_NAME_OF_SHARED_LIBRARY -l LIBRARY_NAME-fpic -shared -Wl,-x -o %o %s -lssl"


i may not be 100% familiar with the varnish wiki but is this information posted in one place?  if not i'd be glad to share it as it took me a while to figure out, and was only able to figure it out with the help of users on the listserv.  or is the listserv used as a reference tool?



________________________________
From: Nils Goroll <slink at schokola.de>
To: Peter Janovsky <peterjanovsky at yahoo.com>
Cc: Tollef Fog Heen <tfheen at varnish-software.com>; Varnish Development <varnish-dev at varnish-cache.org>
Sent: Tue, March 30, 2010 4:28:07 AM
Subject: Re: varnishd -lssl

Hi Peter,

> i've also attempted using the -L and -l options to specify the shared
> library's location and soname, still no progress.

Does -R help to specify the runtime linker path?

Otherwise I suggest you post specific error messages and the exact
commands/settings you're using.

Nils



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100331/385e9e33/attachment.html>

From eran at sandler.co.il  Wed Mar 10 11:42:30 2010
From: eran at sandler.co.il (Eran Sandler)
Date: Wed, 10 Mar 2010 11:42:30 -0000
Subject: Purge while taking hostname into account
Message-ID: <e6b3e5a61003100341o68b8a221v65b28c817d4fbc24@mail.gmail.com>

Hello all,

I have a purging question.

I know that when Varnish caches data it takes the Host header into account
so the same file request under different host name should have different
cache entries.
When I want to have a certain URL removed, I can use the method described in
the examples on the wiki to invalidate that specific URL by setting the ttl
to 0.

I am, however, interested in a wildcard purging that won't affect other
caches under different host names.
the purge method seems to accept only a URL.

Is there a way of doing a wildcard purge that will take the Host header into
account?

Thanks,
Eran
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100310/056ea82b/attachment.html>

From djekels at DRWHoldings.com  Sat Mar 13 16:09:08 2010
From: djekels at DRWHoldings.com (Donny Jekels)
Date: Sat, 13 Mar 2010 16:09:08 -0000
Subject: 1 of 145 tests failed
Message-ID: <3BA21E9E032B22479310705FCAE2A010046A1FAC@DRWUSEXCHANGE.US.DRWHoldings.com>

Thanks for the reminder I'll post it Monday when back in office


 
This e-mail and any attachments may contain information that is confidential and proprietary and otherwise protected from disclosure. If you are not the intended recipient of this e-mail, do not read, duplicate or redistribute it by any means. Please immediately delete it and any attachments and notify the sender that you have received it in error. Unintended recipients are prohibited from taking action on the basis of information in this e-mail or any attachments. The DRW Companies make no representations that this e-mail or any attachments are free of computer viruses or other defects. ----- Original Message -----
 

From: Nils Goroll <slink at schokola.de>
To: Donny Jekels
Cc: varnish-dev at varnish-cache.org <varnish-dev at varnish-cache.org>
Sent: Sat Mar 13 09:42:41 2010
Subject: Re: 1 of 145 tests failed

Hi Donny,

> PASS: ./tests/v00024.vtc
> 
> ===============================================
> 
> 1 of 145 tests failed
> 
> Please report to varnish-dev at projects.linpro.no

Could you please post the error message of the failed test and information about 
your platform?

Otherwise, no-one will be able to help you.

Nils



From eran at sandler.co.il  Sun Mar 14 19:47:06 2010
From: eran at sandler.co.il (Eran Sandler)
Date: Sun, 14 Mar 2010 19:47:06 -0000
Subject: Purge while taking hostname into account
In-Reply-To: <e6b3e5a61003100341o68b8a221v65b28c817d4fbc24@mail.gmail.com>
References: <e6b3e5a61003100341o68b8a221v65b28c817d4fbc24@mail.gmail.com>
Message-ID: <e6b3e5a61003141246u3456db3ctc533126fa5ab7f7e@mail.gmail.com>

Nevermind :-) I found the answer I was looking for here:
http://kristian.blog.linpro.no/2010/02/02/varnish-purges/

 <http://kristian.blog.linpro.no/2010/02/02/varnish-purges/>Thanks Kristian
:-)

I also better understand how purging works in Varnish. Quite a cool concept!

Eran

On Wed, Mar 10, 2010 at 1:41 PM, Eran Sandler <eran at sandler.co.il> wrote:

> Hello all,
>
> I have a purging question.
>
> I know that when Varnish caches data it takes the Host header into account
> so the same file request under different host name should have different
> cache entries.
> When I want to have a certain URL removed, I can use the method described
> in the examples on the wiki to invalidate that specific URL by setting the
> ttl to 0.
>
> I am, however, interested in a wildcard purging that won't affect other
> caches under different host names.
> the purge method seems to accept only a URL.
>
> Is there a way of doing a wildcard purge that will take the Host header
> into account?
>
> Thanks,
> Eran
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100314/2a3ad27c/attachment.html>

From frank.gruellich at navteq.com  Tue Mar 16 18:56:39 2010
From: frank.gruellich at navteq.com (Frank Gruellich)
Date: Tue, 16 Mar 2010 18:56:39 -0000
Subject: Ignoring health checks in statistics
Message-ID: <4B9FD456.7070502@navteq.com>

Hi,

we have two Varnish instances and a load balancer in front.  The load
balancer periodically polls the varnishd's for a static file (lying also
at all the backend servers) to check if varnishd is still working.  It
does that once per second.  This completely messes up statistics like
hit ratio because that static file is served from cache all the time, so
currently our hit ratio is close to 99.9%.  I would like to have counted
this specific file neither as miss nor hit, but just serve it and ignore
it otherwise.  It doesn't even need to be logged.

haproxy has an option monitor-net where you can specify that it does not
need to log requests from that subnet.  Does varnish offer a similar
option?  Or is there some cool VCL snippet I could add?

Thanks in advance.

Kind regards,
-- 
Navteq (DE) GmbH
Frank Gruellich
Map24 Systems and Networks

Duesseldorfer Strasse 40a
65760 Eschborn
Germany

Phone:      +49 6196 77756-414
Fax:        +49 6196 77756-100

HRB 46215, Local Court Frankfurt am Main
Managing Directors: Thomas Golob, Hans Pieter Gieszen, Martin Robert Stockman
USt-ID-No.: DE 197947163

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20100316/4cc49eec/attachment.bin>