From numard at gmail.com  Thu Aug  1 22:53:21 2013
From: numard at gmail.com (Norberto Meijome)
Date: Fri, 2 Aug 2013 08:53:21 +1000
Subject: Sudden hit_for_pass
In-Reply-To: <51F92453.6060103@talkunafraid.co.uk>
References: <51F92453.6060103@talkunafraid.co.uk>
Message-ID: <CACj2-4+L=0vUXBnPbXEoLnc2+-fh02FTvSRm-tY-dUPK+8YB_Q@mail.gmail.com>

Has your backend code changed, and cookies or other headers being sent back
now?
On 01/08/2013 1:05 AM, "James Harrison" <james at talkunafraid.co.uk> wrote:

> Hi,
>
> I have a varnish server sat in front of a website which was previously
> sitting at 60% hitrate, no hit_for_pass. With no configuration change
> aside from adding a single ban rule (which I have since removed) keyed
> to a hostname the site runs (but which accounts for a tiny fraction of
> the traffic), this has gone to 1% hitrate, 99% hit_for_pass. Headers
> being sent out suggest caching is still behaving as expected but the shm
> tools all report this high hit_for_pass rate. Looking at the number of
> objects, there's now nothing but object heads in the store.
>
> Any thoughts on what could be going on here?
>
> --
> Cheers,
> James Harrison
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130802/de8ba15e/attachment.html>

From fabio at dataspace.com.br  Fri Aug  2 19:49:47 2013
From: fabio at dataspace.com.br (Fabio Fraga [DS])
Date: Fri, 2 Aug 2013 16:49:47 -0300
Subject: Some doubts
Message-ID: <CADsNsA_RjqXxTE6pRAaz5Henv7KPr=KHeJod5Wf0E0p=pU1Xgw@mail.gmail.com>

Folks,

It my first time here, i'm Fabio.
My apologizes if my questions looks like a children :D
I'm a newbie in Varnish Cache, but I'm studying hard.

I manage a web server with one single domain (popular fitness/health
wordpress blog in Brazil) which receives 50 K page views/day. When the
person goes to tv shows, we receive peaks of 230 K page views/day with 5 or
6 K simultaneous users.
I've got a very good hit ratio, and all pages are showing fast.

Everything works ok, except when the cache is purged after a new post or
causeless (I just can not understand it yet).
The css of home page gone away and wordpress admin not works properly.

When i clear the cache (with command: varnishadm "ban req.url ~ /"),
everything works ok again, but not for long time.

Can u help me, saying if i forgot or missed some syntax or parameter in my
configuration, which goes below.

#######################
default.vcl
#######################

backend default {
        .host = "127.0.0.1";
        .port = "8081";
                .connect_timeout = 300s;
                .first_byte_timeout = 300s;
                .between_bytes_timeout = 300s;
                .url = "/";
                .interval = 5s;
                .timeout = 1 s;
                .window = 5;
                .threshold = 3;
}

#acl purge {
#        "localhost";
#}
sub vcl_recv {

        # Allow the back-end to serve up stale content if it is responding
slowly.
        set req.grace = 2m;

        # Always cache the following file types for all users.
        if ( req.url ~
"(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$" ) {
                unset req.http.cookie;
        }

        # Don't serve cached pages to logged in users
        if ( req.http.cookie ~ "wordpress_logged_in" || req.url ~
"vaultpress=true" ) {
                return( pass );
        }

        # Drop any cookies sent to WordPress.
        if ( ! ( req.url ~ "wp-(login|admin)" ) ) {
                unset req.http.cookie;
        }
# Purge Cache
#        if (req.request == "PURGE") {
#                if (!client.ip ~ purge) {
#                        error 405 "Not allowed.";
#                }
#                return (lookup);
#        }

        # Handle compression correctly. Different browsers send different
        # "Accept-Encoding" headers, even though they mostly all support
the same
        # compression mechanisms. By consolidating these compression
headers into
        # a consistent format, we can reduce the size of the cache and get
more hits.
        # @see: http://varnish.projects.linpro.no/wiki/FAQ/Compression
        if ( req.http.Accept-Encoding ) {

                if ( req.http.Accept-Encoding ~ "gzip" ) {
                        # If the browser supports it, we'll use gzip.
                        set req.http.Accept-Encoding = "gzip";
                }

                else if ( req.http.Accept-Encoding ~ "deflate" ) {
                        # Next, try deflate if it is supported.
                        set req.http.Accept-Encoding = "deflate";
                }

                else {
                        # Unknown algorithm. Remove it and send unencoded.
                        unset req.http.Accept-Encoding;
                }

        }
  if ( req.request == "POST") {
        ban("req.http.host == " + req.http.Host);
        return(pass);
    }
}

sub vcl_fetch {

        # Allow items to be stale if needed.
        set beresp.grace = 2m;

        # Tempo de cache depois de transferido do Backend (Apache)
        set beresp.ttl = 48h;
                return (deliver);

        # Drop any cookies WordPress tries to send back to the client.
        if ( ! req.url ~ "wp-(login|admin)" && ! req.http.cookie ~
"wordpress_logged_in" ) {
                unset beresp.http.set-cookie;
        }

}

sub vcl_hit {
  if (req.request == "PURGE") {
    purge;
    error 200 "Purged.";
  }
}

sub vcl_miss {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged.";
        }
}

sub vcl_deliver {
  if (obj.hits > 0) {
    set resp.http.X-Varnish-Cache = "HIT";
  }
  else {
    set resp.http.X-Varnish-Cache = "MISS";
  }
}

######################
varnishstat
######################

[root~]# varnishstat -1
client_conn             256621        11.19 Client connections accepted
client_drop                  0         0.00 Connection dropped, no sess/wrk
client_req              878216        38.28 Client requests received
cache_hit               672456        29.31 Cache hits
cache_hitpass                0         0.00 Cache hits for pass
cache_miss              170160         7.42 Cache misses
backend_conn              5702         0.25 Backend conn. success
backend_unhealthy            0         0.00 Backend conn. not attempted
backend_busy                 0         0.00 Backend conn. too many
backend_fail                76         0.00 Backend conn. failures
backend_reuse           199994         8.72 Backend conn. reuses
backend_toolate           2152         0.09 Backend conn. was closed
backend_recycle         202152         8.81 Backend conn. recycles
backend_retry               11         0.00 Backend conn. retry
fetch_head                   0         0.00 Fetch head
fetch_length            201283         8.77 Fetch with Length
fetch_chunked             3968         0.17 Fetch chunked
fetch_eof                    0         0.00 Fetch EOF
fetch_bad                    0         0.00 Fetch had bad headers
fetch_close                 60         0.00 Fetch wanted close
fetch_oldhttp                0         0.00 Fetch pre HTTP/1.1 closed
fetch_zero                   0         0.00 Fetch zero len
fetch_failed                34         0.00 Fetch failed
fetch_1xx                    0         0.00 Fetch no body (1xx)
fetch_204                    0         0.00 Fetch no body (204)
fetch_304                  319         0.01 Fetch no body (304)
n_sess_mem                 220          .   N struct sess_mem
n_sess                     128          .   N struct sess
n_object                  7354          .   N struct object
n_vampireobject              0          .   N unresurrected objects
n_objectcore              7435          .   N struct objectcore
n_objecthead              7449          .   N struct objecthead
n_waitinglist               88          .   N struct waitinglist
n_vbc                        5          .   N struct vbc
n_wrk                      100          .   N worker threads
n_wrk_create               100         0.00 N worker threads created
n_wrk_failed                 0         0.00 N worker threads not created
n_wrk_max                    0         0.00 N worker threads limited
n_wrk_lqueue                 0         0.00 work request queue length
n_wrk_queued                 0         0.00 N queued work requests
n_wrk_drop                   0         0.00 N dropped work requests
n_backend                    1          .   N backends
n_expired               162687          .   N expired objects
n_lru_nuked                  0          .   N LRU nuked objects
n_lru_moved             457757          .   N LRU moved objects
losthdr                      0         0.00 HTTP header overflows
n_objsendfile                0         0.00 Objects sent with sendfile
n_objwrite              746317        32.53 Objects sent with write
n_objoverflow                0         0.00 Objects overflowing workspace
s_sess                  256609        11.18 Total Sessions
s_req                   878216        38.28 Total Requests
s_pipe                      32         0.00 Total pipe
s_pass                   35568         1.55 Total pass
s_fetch                 205596         8.96 Total fetch
s_hdrbytes           301843193     13156.22 Total header bytes
s_bodybytes        26851914941   1170375.06 Total body bytes
sess_closed              17724         0.77 Session Closed
sess_pipeline             6099         0.27 Session Pipeline
sess_readahead            2246         0.10 Session Read Ahead
sess_linger             856500        37.33 Session Linger
sess_herd               861705        37.56 Session herd
shm_records           46079248      2008.42 SHM records
shm_writes             2958209       128.94 SHM writes
shm_flushes                  0         0.00 SHM flushes due to overflow
shm_cont                  1658         0.07 SHM MTX contention
shm_cycles                  21         0.00 SHM cycles through buffer
sms_nreq                   132         0.01 SMS allocator requests
sms_nobj                     0          .   SMS outstanding allocations
sms_nbytes                   0          .   SMS outstanding bytes
sms_balloc               55176          .   SMS bytes allocated
sms_bfree                55176          .   SMS bytes freed
backend_req             205663         8.96 Backend requests made
n_vcl                        1         0.00 N vcl total
n_vcl_avail                  1         0.00 N vcl available
n_vcl_discard                0         0.00 N vcl discarded
n_ban                      661          .   N total active bans
n_ban_gone                 659          .   N total gone bans
n_ban_add                  661         0.03 N new bans added
n_ban_retire                 0         0.00 N old bans deleted
n_ban_obj_test          162881         7.10 N objects tested
n_ban_re_test           162891         7.10 N regexps tested against
n_ban_dups                 658         0.03 N duplicate bans removed
hcb_nolock              842631        36.73 HCB Lookups without lock
hcb_lock                  7470         0.33 HCB Lookups with lock
hcb_insert                7470         0.33 HCB Inserts
esi_errors                   0         0.00 ESI parse errors (unlock)
esi_warnings                 0         0.00 ESI parse warnings (unlock)
accept_fail                  0         0.00 Accept failures
client_drop_late             0         0.00 Connection dropped late
uptime                   22943         1.00 Client uptime
dir_dns_lookups              0         0.00 DNS director lookups
dir_dns_failed               0         0.00 DNS director failed lookups
dir_dns_hit                  0         0.00 DNS director cached lookups hit
dir_dns_cache_full           0         0.00 DNS director full dnscache
vmods                        0          .   Loaded VMODs
n_gzip                       0         0.00 Gzip operations
n_gunzip                 26083         1.14 Gunzip operations
sess_pipe_overflow           0          .   Dropped sessions due to session
pipe overflow
LCK.sms.creat                1         0.00 Created locks
LCK.sms.destroy              0         0.00 Destroyed locks
LCK.sms.locks              396         0.02 Lock Operations
LCK.sms.colls                0         0.00 Collisions
LCK.smp.creat                0         0.00 Created locks
LCK.smp.destroy              0         0.00 Destroyed locks
LCK.smp.locks                0         0.00 Lock Operations
LCK.smp.colls                0         0.00 Collisions
LCK.sma.creat                1         0.00 Created locks
LCK.sma.destroy              0         0.00 Destroyed locks
LCK.sma.locks            84162         3.67 Lock Operations
LCK.sma.colls                0         0.00 Collisions
LCK.smf.creat                1         0.00 Created locks
LCK.smf.destroy              0         0.00 Destroyed locks
LCK.smf.locks           689712        30.06 Lock Operations
LCK.smf.colls                0         0.00 Collisions
LCK.hsl.creat                0         0.00 Created locks
LCK.hsl.destroy              0         0.00 Destroyed locks
LCK.hsl.locks                0         0.00 Lock Operations
LCK.hsl.colls                0         0.00 Collisions
LCK.hcb.creat                1         0.00 Created locks
LCK.hcb.destroy              0         0.00 Destroyed locks
LCK.hcb.locks             7712         0.34 Lock Operations
LCK.hcb.colls                0         0.00 Collisions
LCK.hcl.creat                0         0.00 Created locks
LCK.hcl.destroy              0         0.00 Destroyed locks
LCK.hcl.locks                0         0.00 Lock Operations
LCK.hcl.colls                0         0.00 Collisions
LCK.vcl.creat                1         0.00 Created locks
LCK.vcl.destroy              0         0.00 Destroyed locks
LCK.vcl.locks              928         0.04 Lock Operations
LCK.vcl.colls                0         0.00 Collisions
LCK.stat.creat               1         0.00 Created locks
LCK.stat.destroy             0         0.00 Destroyed locks
LCK.stat.locks          256709        11.19 Lock Operations
LCK.stat.colls               0         0.00 Collisions
LCK.sessmem.creat            1         0.00 Created locks
LCK.sessmem.destroy            0         0.00 Destroyed locks
LCK.sessmem.locks         259356        11.30 Lock Operations
LCK.sessmem.colls              0         0.00 Collisions
LCK.wstat.creat                1         0.00 Created locks
LCK.wstat.destroy              0         0.00 Destroyed locks
LCK.wstat.locks            46183         2.01 Lock Operations
LCK.wstat.colls                0         0.00 Collisions
LCK.herder.creat               1         0.00 Created locks
LCK.herder.destroy             0         0.00 Destroyed locks
LCK.herder.locks               1         0.00 Lock Operations
LCK.herder.colls               0         0.00 Collisions
LCK.wq.creat                   2         0.00 Created locks
LCK.wq.destroy                 0         0.00 Destroyed locks
LCK.wq.locks             1805936        78.71 Lock Operations
LCK.wq.colls                   0         0.00 Collisions
LCK.objhdr.creat            7557         0.33 Created locks
LCK.objhdr.destroy           108         0.00 Destroyed locks
LCK.objhdr.locks         3858977       168.20 Lock Operations
LCK.objhdr.colls               0         0.00 Collisions
LCK.exp.creat                  1         0.00 Created locks
LCK.exp.destroy                0         0.00 Destroyed locks
LCK.exp.locks             518339        22.59 Lock Operations
LCK.exp.colls                  0         0.00 Collisions
LCK.lru.creat                  2         0.00 Created locks
LCK.lru.destroy                0         0.00 Destroyed locks
LCK.lru.locks             332730        14.50 Lock Operations
LCK.lru.colls                  0         0.00 Collisions
LCK.cli.creat                  1         0.00 Created locks
LCK.cli.destroy                0         0.00 Destroyed locks
LCK.cli.locks               7663         0.33 Lock Operations
LCK.cli.colls                  0         0.00 Collisions
LCK.ban.creat                  1         0.00 Created locks
LCK.ban.destroy                0         0.00 Destroyed locks
LCK.ban.locks             357251        15.57 Lock Operations
LCK.ban.colls                  0         0.00 Collisions
LCK.vbp.creat                  1         0.00 Created locks
LCK.vbp.destroy                0         0.00 Destroyed locks
LCK.vbp.locks                  0         0.00 Lock Operations
LCK.vbp.colls                  0         0.00 Collisions
LCK.vbe.creat                  1         0.00 Created locks
LCK.vbe.destroy                0         0.00 Destroyed locks
LCK.vbe.locks              11551         0.50 Lock Operations
LCK.vbe.colls                  0         0.00 Collisions
LCK.backend.creat              1         0.00 Created locks
LCK.backend.destroy            0         0.00 Destroyed locks
LCK.backend.locks         421628        18.38 Lock Operations
LCK.backend.colls              0         0.00 Collisions
SMF.s0.c_req              339798        14.81 Allocator requests
SMF.s0.c_fail                  0         0.00 Allocator failures
SMF.s0.c_bytes       14666334208    639250.94 Bytes allocated
SMF.s0.c_freed       14026391552    611358.22 Bytes freed
SMF.s0.g_alloc             14721          .   Allocations outstanding
SMF.s0.g_bytes         639942656          .   Bytes outstanding
SMF.s0.g_space        2006028288          .   Bytes available
SMF.s0.g_smf               16753          .   N struct smf
SMF.s0.g_smf_frag           1982          .   N small free smf
SMF.s0.g_smf_large            50          .   N large free smf
SMA.Transient.c_req        41668         1.82 Allocator requests
SMA.Transient.c_fail           0         0.00 Allocator failures
SMA.Transient.c_bytes    215142059      9377.24 Bytes allocated
SMA.Transient.c_freed    215142059      9377.24 Bytes freed
SMA.Transient.g_alloc            0          .   Allocations outstanding
SMA.Transient.g_bytes            0          .   Bytes outstanding
SMA.Transient.g_space            0          .   Bytes available
VBE.default(127.0.0.1,,8081).vcls            1          .   VCL references
VBE.default(127.0.0.1,,8081).happy           0          .   Happy health
probes


Atenciosamente,
F?bio Fraga Machado

: Telefone: (48) 4052-9252
: MSN:  fabio at dataspace.com.br
: GTalk: fabio at dataspace.com.br
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130802/2a8c28e6/attachment.html>

From smwood4 at gmail.com  Mon Aug  5 21:05:05 2013
From: smwood4 at gmail.com (Stephen Wood)
Date: Mon, 5 Aug 2013 14:05:05 -0700
Subject: Change client.ip based on x-forwaded-for headers?
Message-ID: <CA+pzux-vYe-5jCMPPEJ38akJFC-BNRr7B3zs8quEuQTW6+_dwQ@mail.gmail.com>

Is there a way to change client.ip?

For example, I would like to set up an ACL to block certain IPs, but since
the instances are behind a load balancer the only thing I have to work with
is the x-forwarded-for header. I've tried something like this:

sub vcl_recv {
  if (req.http.X-Forwarded-For) {
    set client.ip = req.http.X-Forwarded-For;
  }
}

But of course that kind of thing doesn't compile.

Can someone please advise me on this issue or the broader question about
implementing ACLs via x-forwarded-for headers for those of us stuck behind
another load balancer?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130805/0a4591ff/attachment.html>

From r.egglestone at auckland.ac.nz  Mon Aug  5 22:13:29 2013
From: r.egglestone at auckland.ac.nz (Robert Egglestone)
Date: Mon, 5 Aug 2013 22:13:29 +0000
Subject: Change client.ip based on x-forwaded-for headers?
In-Reply-To: <CA+pzux-vYe-5jCMPPEJ38akJFC-BNRr7B3zs8quEuQTW6+_dwQ@mail.gmail.com>
References: <CA+pzux-vYe-5jCMPPEJ38akJFC-BNRr7B3zs8quEuQTW6+_dwQ@mail.gmail.com>
Message-ID: <EDA44FFDD9129945880F2759A34FF53E67EE5EB5@uxcn10-1.UoA.auckland.ac.nz>

Hi Stephen,

The configuration from here can do that:
http://zcentric.com/tag/header/


C{
#include <netinet/in.h>
#include <string.h>
#include <sys/socket.h>
#include <arpa/inet.h>
}C

acl vpn {
  "192.168.0.0"/16;
}

sub vcl_recv {

C{
//
// This is a hack from Igor Gariev (gariev hotmail com):
// Copy IP address from "X-Forwarded-For" header
// into Varnish's client_ip structure.
// This works with Varnish 3.0.1; test with other versions
//
// Trusted "X-Forwarded-For" header is a must!
// No commas are allowed. If your load balancer something other
// than a single IP, then use a regsub() to fix it.
//
struct sockaddr_storage *client_ip_ss = VRT_r_client_ip(sp);
struct sockaddr_in *client_ip_si = (struct sockaddr_in *) client_ip_ss;
struct in_addr *client_ip_ia = &(client_ip_si->sin_addr);
char *xff_ip = VRT_GetHdr(sp, HDR_REQ, "\020X-Forwarded-For:");

if (xff_ip != NULL) {
// Copy the ip address into the struct's sin_addr.
inet_pton(AF_INET, xff_ip, client_ip_ia);
}
}C

  if (client.ip ~ vpn) {
    # do something here
  }

  return(pass);
}


Kind regards,
Robert Egglestone | Application Engineering Team Lead | The University of Auckland
r.egglestone at auckland.ac.nz | ext: 84624 | m: +64 21 240 8992 | Level 2, 58 Symonds St, Auckland, NZ

On 6/08/2013, at 9:05 AM, Stephen Wood <smwood4 at gmail.com> wrote:

> Is there a way to change client.ip?
> 
> For example, I would like to set up an ACL to block certain IPs, but since the instances are behind a load balancer the only thing I have to work with is the x-forwarded-for header. I've tried something like this:
> 
> sub vcl_recv {
>   if (req.http.X-Forwarded-For) {
>     set client.ip = req.http.X-Forwarded-For;
>   }
> }
> 
> But of course that kind of thing doesn't compile.
> 
> Can someone please advise me on this issue or the broader question about implementing ACLs via x-forwarded-for headers for those of us stuck behind another load balancer?
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130805/24d2ca31/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4690 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130805/24d2ca31/attachment.bin>

From lkarsten at varnish-software.com  Tue Aug  6 09:00:41 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 6 Aug 2013 11:00:41 +0200
Subject: Change client.ip based on x-forwaded-for headers?
In-Reply-To: <CA+pzux-vYe-5jCMPPEJ38akJFC-BNRr7B3zs8quEuQTW6+_dwQ@mail.gmail.com>
References: <CA+pzux-vYe-5jCMPPEJ38akJFC-BNRr7B3zs8quEuQTW6+_dwQ@mail.gmail.com>
Message-ID: <20130806090040.GA9209@immer.varnish-software.com>

On Mon, Aug 05, 2013 at 02:05:05PM -0700, Stephen Wood wrote:
> Is there a way to change client.ip?

Yes, you can do this with the ipcast vmod:

	https://github.com/lkarsten/libvmod-ipcast


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From smwood4 at gmail.com  Tue Aug  6 16:08:40 2013
From: smwood4 at gmail.com (Stephen Wood)
Date: Tue, 6 Aug 2013 09:08:40 -0700
Subject: Change client.ip based on x-forwaded-for headers?
In-Reply-To: <20130806090040.GA9209@immer.varnish-software.com>
References: <CA+pzux-vYe-5jCMPPEJ38akJFC-BNRr7B3zs8quEuQTW6+_dwQ@mail.gmail.com>
 <20130806090040.GA9209@immer.varnish-software.com>
Message-ID: <CA+pzux90074bGreidEeXMYNA2ZWJX_N5dE-NF77Vq_DAVVgduw@mail.gmail.com>

That's perfect, Lasse! Thank you for the information.

I had settled on putting the VCL in a jinja template and iterating over a
list of ips (x-forward-for headers), each one creating a separate entry
into sub vcl_recv to send an error 403.


On Tue, Aug 6, 2013 at 2:00 AM, Lasse Karstensen <
lkarsten at varnish-software.com> wrote:

> On Mon, Aug 05, 2013 at 02:05:05PM -0700, Stephen Wood wrote:
> > Is there a way to change client.ip?
>
> Yes, you can do this with the ipcast vmod:
>
>         https://github.com/lkarsten/libvmod-ipcast
>
>
> --
> With regards,
> Lasse Karstensen
> Varnish Software AS
>



-- 
Stephen Wood
Dev/Ops Engineer
Moz, Inc.
Website: www.heystephenwood.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130806/4eff1544/attachment.html>

From numard at gmail.com  Tue Aug  6 23:01:38 2013
From: numard at gmail.com (Norberto Meijome)
Date: Wed, 7 Aug 2013 09:01:38 +1000
Subject: Sudden hit_for_pass
In-Reply-To: <52011857.5050007@talkunafraid.co.uk>
References: <51F92453.6060103@talkunafraid.co.uk>
 <CACj2-4+L=0vUXBnPbXEoLnc2+-fh02FTvSRm-tY-dUPK+8YB_Q@mail.gmail.com>
 <52011857.5050007@talkunafraid.co.uk>
Message-ID: <CACj2-4LzvdofVm_iXA3mhj1PbWevNQUUWepi5pRd1JwOWer-dg@mail.gmail.com>

Please make sure you CC the list ;-)
On 07/08/2013 1:38 AM, "James Harrison" <james at talkunafraid.co.uk> wrote:

> On 01/08/13 23:53, Norberto Meijome wrote:
> > Has your backend code changed, and cookies or other headers being sent
> > back now?
> >
>
> No.
>
> In addition to this sudden increase in hit_for_pass, my monitoring of
> uptime is now showing a flat line at 23675208.05 days, transfer rates
> are in the kilobytes rather than the usual 180M, and apparently I've got
> 1.8 million worker threads running. This all seems a bit unlikely, to
> say the least, so I'm thinking shm bug or something along those lines?
>
>
> --
> Cheers,
> James Harrison
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130807/53422705/attachment.html>

From marco at nucleus.it  Wed Aug  7 11:00:52 2013
From: marco at nucleus.it (Marco Felettigh)
Date: Wed, 7 Aug 2013 13:00:52 +0200
Subject: varnishstat and memory  usage
Message-ID: <20130807130052.0f5662e3@lobo.lobo.dom>

Hi to all,
can someone explain please what this values mean ?

varnish-3.0.2

from varnishstat -1 :

SMA.s0.c_bytes   4421961276     14330.40 Bytes allocated
Bytes allocated from Varnish for all the things;
cache , thread etc . Right ?

SMA.s0.c_freed   3336892882     10813.98 Bytes freed
freed ? that means that in the uptime of my Varnish instance it freed
3,3GB of bytes from the cache ?
So this counter can go beyond Bytes allocated ?

SMA.s0.g_bytes   1085068394          .   Bytes outstanding
These are the bytes used by cache ?

SMA.s0.g_space   2136157078          .   Bytes available
Ok this is self explained. I hope :)

Thanks and have a nice day

Marco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130807/f31199ef/attachment.bin>

From russ at eatnumber1.com  Thu Aug  8 18:11:32 2013
From: russ at eatnumber1.com (Russell Harmon)
Date: Thu, 8 Aug 2013 11:11:32 -0700
Subject: Requests with the same ID interleaving in varnishlog output
Message-ID: <CA+zrezSQQYr=LppPDbFJ=shmZgwNszPOirc9OZdTiujddd6H6g@mail.gmail.com>

I'm writing a parser for varnishlog output and am seeing some odd behavior.
See the log here:
https://gist.github.com/eatnumber1/1dc55700324d9ee4dfb4which shows a
request start with xid 1254156398, later followed by a
request end with xid 1254156353 (which is the xid of the request before
1254156398), later (not shown) followed by the ReqEnd of 1254156353. What's
the deal? How can I tell what request all the lines with id 7 are for?

--
Russell Harmon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130808/7a710987/attachment.html>

From omegabk at gmail.com  Thu Aug  8 19:10:35 2013
From: omegabk at gmail.com (omega bavedila katumua)
Date: Thu, 8 Aug 2013 21:10:35 +0200
Subject: Requests with the same ID interleaving in varnishlog output
In-Reply-To: <CA+zrezSQQYr=LppPDbFJ=shmZgwNszPOirc9OZdTiujddd6H6g@mail.gmail.com>
References: <CA+zrezSQQYr=LppPDbFJ=shmZgwNszPOirc9OZdTiujddd6H6g@mail.gmail.com>
Message-ID: <09E5EC2E-8836-4BF5-A9E8-319A42C089A3@gmail.com>

Use varnishlog -o

Cheers

Sent from my iPhone

On Aug 8, 2013, at 8:11 PM, Russell Harmon <russ at eatnumber1.com> wrote:

> I'm writing a parser for varnishlog output and am seeing some odd behavior. See the log here: https://gist.github.com/eatnumber1/1dc55700324d9ee4dfb4 which shows a request start with xid 1254156398, later followed by a request end with xid 1254156353 (which is the xid of the request before 1254156398), later (not shown) followed by the ReqEnd of 1254156353. What's the deal? How can I tell what request all the lines with id 7 are for?
> 
> --
> Russell Harmon
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130808/6271d19c/attachment.html>

From russ at eatnumber1.com  Thu Aug  8 19:12:08 2013
From: russ at eatnumber1.com (Russell Harmon)
Date: Thu, 8 Aug 2013 12:12:08 -0700
Subject: Requests with the same ID interleaving in varnishlog output
In-Reply-To: <09E5EC2E-8836-4BF5-A9E8-319A42C089A3@gmail.com>
References: <CA+zrezSQQYr=LppPDbFJ=shmZgwNszPOirc9OZdTiujddd6H6g@mail.gmail.com>
 <09E5EC2E-8836-4BF5-A9E8-319A42C089A3@gmail.com>
Message-ID: <CA+zrezSeLPRwEjRKrdV0m8MpbuvKcBKkbqwWn4LTnWeNoxSOcg@mail.gmail.com>

-o Ignored for compatibility with earlier versions.

that's the option you're referring to?

I have tried it both with, and without -O

--
Russell Harmon


On Thu, Aug 8, 2013 at 12:10 PM, omega bavedila katumua
<omegabk at gmail.com>wrote:

> Use varnishlog -o
>
> Cheers
>
> Sent from my iPhone
>
> On Aug 8, 2013, at 8:11 PM, Russell Harmon <russ at eatnumber1.com> wrote:
>
> I'm writing a parser for varnishlog output and am seeing some odd
> behavior. See the log here:
> https://gist.github.com/eatnumber1/1dc55700324d9ee4dfb4 which shows a
> request start with xid 1254156398, later followed by a request end with xid
> 1254156353 (which is the xid of the request before 1254156398), later (not
> shown) followed by the ReqEnd of 1254156353. What's the deal? How can I
> tell what request all the lines with id 7 are for?
>
> --
> Russell Harmon
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130808/e26efc57/attachment.html>

From lohmann.andre at gmail.com  Sun Aug 11 08:21:28 2013
From: lohmann.andre at gmail.com (Andre Lohmann)
Date: Sun, 11 Aug 2013 10:21:28 +0200
Subject: Varnish for large Videofiles with Range requests
Message-ID: <CAJa8y-A+xAdvLF=4fLNL+QKRysDO_ycAstguZtSgt7ZVzJw-NA@mail.gmail.com>

Hi, I read through some explanations about varnish, but still have
questions, that I didn't understand yet.

I want to use a varnish farm as some kind of loadbalancing in front of a
Videoserver.

My Idea is, that the varnish clients shall be hit, when a File is
requested. If this file was allready cached by the hit instance, it will be
delivered by the cache, otherwise it will be delivered from the origin an
being chached for the next requests.

The problem I see here are within the range requests. If I got it right,
when the first (range) request hits varnish, before the file is cached,
this file will have a big latency, as it first needs to be fully cached. Is
it possible to pipe all range requests to the origin, until the file is
fully cached?

For highly frequented files it is a possible way, to prefetch the file, so
it will be cached, before the first requests hit that file. But there are
also archived files, which get requested not that often and that will fill
up the cache otherwise, if I need to cache them all too.

My Idea was, that high frequented files stay longer in the cache, as they
are reuqested more often and older, archived files are drop from the cache,
as they are requested much less.

-- 
Andre Lohmann

Alte G?rtnerei 20a
24232 Sch?nkirchen

Tel.: +49 (0) 431 260 93 833
mobil: +49 (0) 176 55 910 913
mail: lohmann.andre at gmail.com

LinkedIn: http://de.linkedin.com/pub/andre-lohmann/26/a60/a92
Xing: http://www.xing.com/profile/Andre_Lohmann
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130811/5d60f07c/attachment.html>

From hugo.cisneiros at gmail.com  Mon Aug 12 02:07:50 2013
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Sun, 11 Aug 2013 23:07:50 -0300
Subject: Varnish for large Videofiles with Range requests
In-Reply-To: <CAJa8y-A+xAdvLF=4fLNL+QKRysDO_ycAstguZtSgt7ZVzJw-NA@mail.gmail.com>
References: <CAJa8y-A+xAdvLF=4fLNL+QKRysDO_ycAstguZtSgt7ZVzJw-NA@mail.gmail.com>
Message-ID: <CA+KACL=gytXUREwp=F2DXQR_nVtDQUSa5XXUKgOa9NoBGBRBfQ@mail.gmail.com>

On Sun, Aug 11, 2013 at 5:21 AM, Andre Lohmann <lohmann.andre at gmail.com> wrote:
> My Idea is, that the varnish clients shall be hit, when a File is requested. If this file was allready cached by the hit instance, it will be delivered by the cache, otherwise it will be delivered from the origin an being chached for the next requests.
>
> The problem I see here are within the range requests. If I got it right, when the first (range) request hits varnish, before the file is cached, this file will have a big latency, as it first needs to be fully cached. Is it possible to pipe all range requests to the origin, until the file is fully cached?

Did you try the streaming fork (usually called "s" fork, like 3.0.2s)?
It works very well and solves your problem.

Information: https://www.varnish-software.com/blog/http-streaming-varnish
(part "The new and improved streaming implementation")
Test it (only found the 3.0.2):
http://repo.varnish-cache.org/test/3.0.2+streaming/

The original varnish trunk can't do this.

> For highly frequented files it is a possible way, to prefetch the file, so it will be cached, before the first requests hit that file. But there are also archived files, which get requested not that often and that will fill up the cache otherwise, if I need to cache them all too.
>
> My Idea was, that high frequented files stay longer in the cache, as they are reuqested more often and older, archived files are drop from the cache, as they are requested much less.

I don't know if this can be done easily since you can't store
variables across many requests (unless there's a vmod that I don't
know). If anyone knows, I would like to know too :)

But you can process log files, check which files are requested more
frequently and prefetch them the same way as "warming the cache" (look
for it at the documentation). For example: you get the most accessed
files into a text file and do some warming using a header that does
req.hash_always_miss on vcl_recv before the TTL expires. This may work
and these files should be always on the cache.

-- 
[]'s
Hugo
www.devin.com.br



From superoverdrive at gmx.de  Mon Aug 12 09:49:12 2013
From: superoverdrive at gmx.de (superoverdrive at gmx.de)
Date: Mon, 12 Aug 2013 11:49:12 +0200 (CEST)
Subject: Website migration using Varnish?
Message-ID: <trinity-77da20ac-74fe-42ec-9f32-67087f9712b3-1376300952522@3capp-gmx-bs57>

An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130812/a79117b8/attachment.html>

From tobias.eichelbroenner at lamp-solutions.de  Mon Aug 12 13:13:35 2013
From: tobias.eichelbroenner at lamp-solutions.de (=?ISO-8859-1?Q?Tobias_Eichelbr=F6nner?=)
Date: Mon, 12 Aug 2013 15:13:35 +0200
Subject: Website migration using Varnish?
In-Reply-To: <trinity-77da20ac-74fe-42ec-9f32-67087f9712b3-1376300952522@3capp-gmx-bs57>
References: <trinity-77da20ac-74fe-42ec-9f32-67087f9712b3-1376300952522@3capp-gmx-bs57>
Message-ID: <5208DF7F.1020903@lamp-solutions.de>

Hi,

> My plan is to use varnish in order to fetch Tomcat (or Apache->Tomcat)
> pages from the new application and deliver all urls not found on the new
> application from the old php application.

my first idea would to build a custom error page and redirect all failed
requests from the Java-Website to the old php website.
This is easier if you can use different (sub)domains for the old and the
new page.

The error vcl-code would look something like this (untested)

sub vcl_error {
   if(obj.http.host == "my_java_backend.tld
    set obj.http.Content-Type = "text/html; charset=utf-8";
    set obj.http.Location = "http://my_php_backend.tld/" + req.url;
    set obj.status = 301;#moved permanently
    return(deliver);
  }
}

Sincerely,

Tobias

-- 
LAMP solutions GmbH
Gostenhofer Hauptstrasse 35
90443 Nuernberg

Amtsgericht Nuernberg: HRB 22366
Geschaeftsfuehrer: Heiko Schubert

Es gelten unsere allgemeinen Geschaeftsbedingungen.
http://www.lamp-solutions.de/agbs/

Telefon      : 0911 / 376 516 0
Fax          : 0911 / 376 516 11
E-Mail       : support at lamp-solutions.de
Web          : www.lamp-solutions.de
Facebook     : http://www.facebook.com/LAMPsolutions
Twitter      : http://twitter.com/#!/lampsolutions



From lkarsten at varnish-software.com  Mon Aug 12 14:31:02 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 12 Aug 2013 16:31:02 +0200
Subject: Storing state between requests (Was: Re: Varnish for large
 Videofiles with Range requests)
In-Reply-To: <CA+KACL=gytXUREwp=F2DXQR_nVtDQUSa5XXUKgOa9NoBGBRBfQ@mail.gmail.com>
References: <CAJa8y-A+xAdvLF=4fLNL+QKRysDO_ycAstguZtSgt7ZVzJw-NA@mail.gmail.com>
 <CA+KACL=gytXUREwp=F2DXQR_nVtDQUSa5XXUKgOa9NoBGBRBfQ@mail.gmail.com>
Message-ID: <20130812143102.GA4162@immer.varnish-software.com>

On Sun, Aug 11, 2013 at 11:07:50PM -0300, Hugo Cisneiros (Eitch) wrote:
[..]
> I don't know if this can be done easily since you can't store
> variables across many requests (unless there's a vmod that I don't
> know). If anyone knows, I would like to know too :)

The variable vmod can do this for you.

    https://github.com/varnish/libvmod-var


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From hugo.cisneiros at gmail.com  Mon Aug 12 14:39:56 2013
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Mon, 12 Aug 2013 11:39:56 -0300
Subject: Website migration using Varnish?
In-Reply-To: <trinity-77da20ac-74fe-42ec-9f32-67087f9712b3-1376300952522@3capp-gmx-bs57>
References: <trinity-77da20ac-74fe-42ec-9f32-67087f9712b3-1376300952522@3capp-gmx-bs57>
Message-ID: <CA+KACLkmJw9+Ae-aZE6YBC=GoCprJ+jUxsj=iq9HiR98pkEH=Q@mail.gmail.com>

On Mon, Aug 12, 2013 at 6:49 AM,  <superoverdrive at gmx.de> wrote:
> I would like to migrate a very old PHP website to Java - without loosing all
> internal pages...however I also do not want to migrate 100% of the old
> website, only abour 80%.
>
> My plan is to use varnish in order to fetch Tomcat (or Apache->Tomcat) pages
> from the new application and deliver all urls not found on the new
> application from the old php application.

Setup two backends:

backend be_java {
   [...]
}

backend be_php {
   [...]
}

On vcl_recv, the default backend will be the java one, meaning all
requests will go to the java backend first. But if you receive the
request if a custom header, you will serve from the PHP backend. This
custom header will be appended to the request by varnish when it gets
a 404 object response from the java backend:

sub vcl_recv {
  set req.backend = be_java;

  # if custom header is present, use the php backend
  if (req.http.X-Serve-From-Old == "1") {
    unset req.http.X-Serve-From-Old;
    set req.backend = be_php;
  }

  [...]
}

sub vcl_fetch {
  if (req.backend == be_java && beresp.status == 404) {
    # set custom header and restart request.
    set req.http.X-Serve-From-Old = "1";
    return (restart);
  }

  [...]
}

This should do the trick.

> Would you recommend to use Tomcat without Apache or with Apache?

It really depends if you're using apache features or not :) People use
apache in front of Tomcat to mix static and dynamic items, to use
mod_rewrite features easily, to virtual hosts with different dynamic
languages and so on. But if you're just using the application, keep it
simple.

-- 
[]'s
Hugo
www.devin.com.br



From hugo.cisneiros at gmail.com  Mon Aug 12 15:12:45 2013
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Mon, 12 Aug 2013 12:12:45 -0300
Subject: Website migration using Varnish?
In-Reply-To: <trinity-f3b598b4-bd17-4d71-9c44-7eee268dc0d6-1376319514855@3capp-gmx-bs57>
References: <trinity-77da20ac-74fe-42ec-9f32-67087f9712b3-1376300952522@3capp-gmx-bs57>
 <CA+KACLkmJw9+Ae-aZE6YBC=GoCprJ+jUxsj=iq9HiR98pkEH=Q@mail.gmail.com>
 <trinity-f3b598b4-bd17-4d71-9c44-7eee268dc0d6-1376319514855@3capp-gmx-bs57>
Message-ID: <CA+KACLnjt-4h=MhrecFBawOAxhYjo8-ha21vMy8OByMEgXv_Pw@mail.gmail.com>

On Mon, Aug 12, 2013 at 11:58 AM,  <superoverdrive at gmx.de> wrote:
> What if there is several domain names and the response depends on the
> domain?
>
> Will Varnish pass the request domain name on to tomcat and apache?

Yes. Varnish will send the request to the backend as it received. This
includes the Host header that specifies the domain name.

We set up a custom header in vcl_fetch only if it's a 404 status and
the backend is the java one. If you need, you can add as many
conditions as you like to the logic and it'll fallback to the php
backend.

-- 
[]'s
Hugo
www.devin.com.br



From maximilian.herrmann at jambit.com  Mon Aug 12 20:25:44 2013
From: maximilian.herrmann at jambit.com (Maximilian Herrmann)
Date: Mon, 12 Aug 2013 20:25:44 +0000
Subject: Error: possible SYN flooding
Message-ID: <29A8CC44441EC34F885176945C633FDD566354E1@exc1.jambit.com>

Hi,

i`m currently having issues with my Varnish load balancers. For some users, the access to our site is very slow, for others it isn`t reachable.
I`ve observed something strange at the dmesg-Logfile, it is saying "possible SYN flooding on port 80. Sending cookies." about every 20 minutes. I don`t think that this is a real SYN flood, as this is some very high traffic system.
The system is running at CentOS 6.3, we`re using Varnish 3.0.4 (from rpm).

We have optimized the following kernel parameters already:
/proc/sys/net/ipv4/tcp_tw_reuse (now set to 1)
/proc/sys/net/ipv4/tcp_tw_recycle (now set to 1)
/proc/sys/net/ipv4/tcp_fin_timeout (now set to 30)
/proc/sys/net/ipv4/tcp_keepalive_intvl (now set to 40)
/proc/sys/net/ipv4/tcp_keepalive_time (now set to 1800)
/proc/sys/net/ipv4/tcp_syn_retries (not set to 2)
/proc/sys/net/ipv4/tcp_synack_retries (now set to 2)
/proc/sys/net/ipv4/tcp_no_metrics_save (now set to 1)
/proc/sys/net/ipv4/tcp_rmem (now 4096   87380   16777216)
/proc/sys/net/core/wmem_max (now 16777216)
/proc/sys/net/core/rmem_max (now 16777216)
/proc/sys/net/ipv4/tcp_max_syn_backlog (now 4096)
/proc/sys/net/ipv4/tcp_max_syn_backlog (now 4096)
/proc/sys/net/core/netdev_max_backlog (now 5000)
/proc/sys/net/ipv4/ip_local_port_range (now 1024 65536)


I could not find anything at Google. May somebody give me tips how to resolve this issue?

Thanks,

Maxi
________________________________

Maximilian Herrmann, System Administrator
Phone: +49 89 45 23 47 - 628
Fingerprint: 4BDD 79EA 8B40 F21C 9667 DC6C F257 0841 13DB 6EC5

jambit GmbH
Erika-Mann-Str. 63, 80636 M?nchen
Phone: +49 89 45 23 47 - 0 Fax: +49 89 45 23 47 - 70

http://www.jambit.com where innovation works

Gesch?ftsf?hrer: Peter F. Fellinger, Markus Hartinger
Sitz: M?nchen; Registergericht: M?nchen, HRB 129139
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130812/08548a0a/attachment.html>

From lkarsten at varnish-software.com  Tue Aug 13 10:05:36 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 13 Aug 2013 12:05:36 +0200
Subject: Error: possible SYN flooding
In-Reply-To: <29A8CC44441EC34F885176945C633FDD566354E1@exc1.jambit.com>
References: <29A8CC44441EC34F885176945C633FDD566354E1@exc1.jambit.com>
Message-ID: <20130813100536.GC4162@immer.varnish-software.com>

On Mon, Aug 12, 2013 at 08:25:44PM +0000, Maximilian Herrmann wrote:
> i`m currently having issues with my Varnish load balancers. For some users, the access to our site is very slow, for others it isn`t reachable.
> I`ve observed something strange at the dmesg-Logfile, it is saying "possible SYN flooding on port 80. Sending cookies." about every 20 minutes. I don`t think that this is a real SYN flood, as this is some very high traffic system.
> The system is running at CentOS 6.3, we`re using Varnish 3.0.4 (from rpm).

>From what I gathered last time I looked into this for a customer, you need to
increase netdev_max_backlog to avoid the synflood message in dmesg.

You can see with netstat -s if your syncookies are in fact returned from the
client, which hints on whether you are being synflooded or not.


> We have optimized the following kernel parameters already:
> /proc/sys/net/ipv4/tcp_tw_reuse (now set to 1)
> /proc/sys/net/ipv4/tcp_tw_recycle (now set to 1)
> /proc/sys/net/ipv4/tcp_fin_timeout (now set to 30)
> /proc/sys/net/ipv4/tcp_keepalive_intvl (now set to 40)
> /proc/sys/net/ipv4/tcp_keepalive_time (now set to 1800)
> /proc/sys/net/ipv4/tcp_syn_retries (not set to 2)
> /proc/sys/net/ipv4/tcp_synack_retries (now set to 2)
> /proc/sys/net/ipv4/tcp_no_metrics_save (now set to 1)
> /proc/sys/net/ipv4/tcp_rmem (now 4096   87380   16777216)
> /proc/sys/net/core/wmem_max (now 16777216)
> /proc/sys/net/core/rmem_max (now 16777216)
> /proc/sys/net/ipv4/tcp_max_syn_backlog (now 4096)
> /proc/sys/net/ipv4/tcp_max_syn_backlog (now 4096)
> /proc/sys/net/core/netdev_max_backlog (now 5000)
> /proc/sys/net/ipv4/ip_local_port_range (now 1024 65536)


I'd recommend keeping TCP stack tuning to a minimum.

tcp_tw_recycle is dangerous. Don't enable it.

At least you should be sure you understand what each setting means before
changing it. Please don't just copy something from a blog somewhere.


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From tobias.eichelbroenner at lamp-solutions.de  Tue Aug 13 10:35:36 2013
From: tobias.eichelbroenner at lamp-solutions.de (=?ISO-8859-1?Q?Tobias_Eichelbr=F6nner?=)
Date: Tue, 13 Aug 2013 12:35:36 +0200
Subject: Error: possible SYN flooding
In-Reply-To: <29A8CC44441EC34F885176945C633FDD566354E1@exc1.jambit.com>
References: <29A8CC44441EC34F885176945C633FDD566354E1@exc1.jambit.com>
Message-ID: <520A0BF8.20009@lamp-solutions.de>

Hi,

> I could not find anything at Google. May somebody give me tips how to
> resolve this issue?

If you have many connections to your loadbalancer, a slow backend and a
low cache hitrate the number of connection hold open may cause problems.

My first step would be to check if the number of syn-connections fits to
the number of connections

so if

netstat -nap | grep SYN | wc -l

is high compared to the number of requests per second maybe this is a
real SYN flood attack after all.
I checked on a system of my own serving about 70 req/s the number of
open syn requests is always below 10.

If there is no attack and you cannot speed up your backend or increase
the cache hitrate, maybe you can use dns round robin to split the
requests between more than one load balancer.

Sincerely,

Tobias

-- 
LAMP solutions GmbH
Gostenhofer Hauptstrasse 35
90443 Nuernberg

Amtsgericht Nuernberg: HRB 22366
Geschaeftsfuehrer: Heiko Schubert

Es gelten unsere allgemeinen Geschaeftsbedingungen.
http://www.lamp-solutions.de/agbs/

Telefon      : 0911 / 376 516 0
Fax          : 0911 / 376 516 11
E-Mail       : support at lamp-solutions.de
Web          : www.lamp-solutions.de
Facebook     : http://www.facebook.com/LAMPsolutions
Twitter      : http://twitter.com/#!/lampsolutions



From hugues at betabrand.com  Wed Aug 14 03:46:50 2013
From: hugues at betabrand.com (Hugues Alary)
Date: Tue, 13 Aug 2013 20:46:50 -0700
Subject: varnish and mod_pagespeed
In-Reply-To: <CAGeq0-cdbBaNNr-Q4H5KnXjcXS8EbpQ9i5jgOfh0OQ7NQOqfRQ@mail.gmail.com>
References: <CADOBX8i+-jozGaSMmkiUabQSNmtUSLs6iAsH7XtNPZdsY688aA@mail.gmail.com>
 <51C2AC9F.70900@ibotty.net>
 <CAGeq0-cdbBaNNr-Q4H5KnXjcXS8EbpQ9i5jgOfh0OQ7NQOqfRQ@mail.gmail.com>
Message-ID: <CAL_utwa_8O7sn_6LTfjafvVqKR_zMx+TvgW7KZ6uxtMRJtcP_A@mail.gmail.com>

Hi there,

In order to save some time to people who might find this thread, here is
how to force mod_pagespeed to send a fully optimized version to varnish:

In your .vcl file:

sub vcl_recv() {
       set req.http.X-PSA-Blocking-Rewrite = "whatever-you-want";
}

In your pagespeed configuration:

ModPagespeedBlockingRewriteKey   "whatever-you-want"

It is worth noting that a feature meant to make reverse proxies like
varnish happy with mod_pagespeed is currently in beta, here is the relevant
link:
https://developers.google.com/speed/pagespeed/module/downstream-caching

-Hugues
Betabrand.com




On Thu, Jun 20, 2013 at 1:07 AM, ?ukasz Rysiak <lury at critical.pl> wrote:

> David, please read this thread on google mps forum, where i described the
> same problem and got the solution from mps developers.
> https://groups.google.com/forum/#!topic/mod-pagespeed-discuss/j7X4uSckXlg
>
> You can add header which will force mps to do full optimisation before
> sending a response to varnish.
>
>
>
>
> Regards,
>
> ?ukasz Rysiak
> Critical - Conversion Experts
> tel. 792 448 678
> http://critical.pl
>
>
>
> 2013/6/20 <me at ibotty.net>
>
> hi,
>>
>> > The problem is that varnish caches the first request - an unoptimized
>> > page. I wonder whether somebody has found a way to tell varnish to
>> > ignore the first request after the ttl is over and cache the second
>> > one?
>>
>> maybe you can configure mod_pagespeed to set a header when it is only
>> passing through the page.
>>
>> and i thought the recommended setup is to have mod_pagespeed before
>> varnish, but i've never set it up.
>>
>> good luck,
>>  tobias florek
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130813/5d8e946c/attachment.html>

From hugues at betabrand.com  Wed Aug 14 04:09:56 2013
From: hugues at betabrand.com (Hugues Alary)
Date: Tue, 13 Aug 2013 21:09:56 -0700
Subject: Passing cookies to ESI includes
In-Reply-To: <CABtDKm4JQgdAkHQcNGfC07T1NFX14nt0tRKFm1SUv1me67z58w@mail.gmail.com>
References: <CABtDKm4JQgdAkHQcNGfC07T1NFX14nt0tRKFm1SUv1me67z58w@mail.gmail.com>
Message-ID: <CAL_utwZErQF5+8me5Aj1sMghoB=Nw9pKnE2-g=CTsSn6MU83aA@mail.gmail.com>

One way to alleviate, a bit, this problem is to make sure that a cookie is
always set on the first request.

To do that, you can include on every page a fake pixel image that will hit
a page that will return a response with a Set-Cookie header.

E.g:

Let's say you are serving www.example.com/mylandingpage.html
Have a script hosted at www.example.com/pixel.php and add an <img /> tag
pointing to it on all your pages.

Pixel.php contains:
header("Content-type: image/gif");
sprintf
("%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c",71,73,70,56,57,97,1,0,1,0,128,255,0,192,192,192,0,0,0,33,249,4,1,0,0,0,0,44,0,0,0,0,1,0,1,0,0,2,2,68,1,0,59));

mylandingpage.html:
<body>
    <img src="//pixel.php" />
</body>

your .vcl file:
sub vcl_recv()
{
    // Never cache the pixel... you wouldn't like your customers to share
the same cookie.
    if(req.url ~ "pixel.php")
    {
        return (pass);
    }
}

I'm however not sure this will make the *very* *first* request not create
garbage sessions, but it will prevent your customer from not having any
cookie until they hit a non-cached page.

-Hugues
betabrand.com


On Wed, Jul 3, 2013 at 4:50 AM, Dridi Boukelmoune <
dridi.boukelmoune at zenika.com> wrote:

> Hi all,
>
> I hope someone can help me with this ESI problem I have with one of my
> clients.
>
> One of the applications here use Varnish in front of two backends, and
> we see a lot of session garbage allocated because of ESI not passing
> cookies back-and-forth.
>
> The two backends in a nutshell:
> - front office (drupal cms)
> - back office (java webapp, connected to other services of the company)
>
> For the sake of simplicity, assume the front and back offices send
> respectively PHPSESSID and JSESSIONID cookies.
>
> The setup is also twisted, since you can actually access to both front
> and back offices (varnish does the routing) and each can esi:include
> fragments from the other. I haven't got through the 5 whys with the
> team yet...
>
> The worst case scenario:
>
> - Access to the front office without any cookie
> => drupal adds a set-cookie header for PHPSESSID
> => varnish finds N esi:include tags
> ==> the back office creates N different JSESSIONID set-cookie headers
> (one per request)
>
> It means N wasted sessions on the back office since none of the
> cookies will be retrieved by the user agent. And this will happen
> until you actually hit a page directly on the back office (with a
> shiny JSESSIONID).
>
> The ideal solution:
>
> - Access to the front office without any cookie
> => drupal adds a set-cookie header for PHPSESSID
> => varnish finds N esi:include tags
> ==> varnish fires the first esi request
> ===> the backoffice adds a set-cookie header for JSESSIONID
> ===> varnish adds the JSESSIONID cookie to the existing set-cookie header
> ===> varnish adds the JSESSIONID cookie header to the next requests
> ==> varnish fires the other N-1 esi requests with the JSESSIONID cookie
> ===> the back office doesn't create additional sessions
> ==> varnish answers with both PHPSESSID and JSESSIONID cookies
> => champagne \o/
>
> I'm open to any kind of solution, including (no pun intended) inline C
> or open source vmods. Any help appreciated :)
>
> One more thing:
> I'm aware of libvmod-header, which solves the (implicit) multiple
> Set-Cookie headers issue, but doesn't help cross the ESI boundaries.
>
> Best Regards,
> Dridi
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130813/2f0d6262/attachment.html>

From Paul.McInerney at faredge.com.au  Wed Aug 14 05:56:47 2013
From: Paul.McInerney at faredge.com.au (Paul McInerney)
Date: Wed, 14 Aug 2013 15:56:47 +1000
Subject: varnish 3 - odd behaviour with using PURGE against cached objects
Message-ID: <CCD0821AE781994EB8BAB510676B5EF50120CD12A083@ZEUS.faredge.local>

Hi all,

I've got varnish currently configured to server cached content for 300 secs.

I've also allowed varnish to accept purge requests (coming from a trusted list of IPs)

Watching varnish (using varnishlog), I can see a  PURGE request get successfully handled (I am sending the purge via php/curl_init  - even if I telnet directly to varnish and issue a direct http request, I still get the same 'success')

In this example, I want to clear the front page for a joomla site, from cache:

   11 SessionOpen  c 10.2.35.5 36978 :6081
   11 ReqStart     c 10.2.35.5 36978 806561070
   11 RxRequest    c PURGE
   11 RxURL        c /
   11 RxProtocol   c HTTP/1.0
   11 RxHeader     c Host: staging.site.com.au<http://staging.site.com.au/>
   11 RxHeader     c X-Real-IP: 117.53.165.104
   11 RxHeader     c X-Forwarded-For: 117.53.165.104
   11 RxHeader     c Connection: close
   11 RxHeader     c Accept: */*
   11 VCL_call     c recv
   11 VCL_acl      c MATCH purge 10.2.35.5
   11 VCL_return   c lookup
   11 VCL_call     c hash
   11 Hash         c /
   11 Hash         c staging.site.com.au<http://staging.site.com.au/>
   11 VCL_return   c hash
   11 Hit          c 806560950
   11 VCL_call     c hit error
   11 VCL_call     c error deliver
   11 VCL_call     c deliver deliver
   11 TxProtocol   c HTTP/1.1
   11 TxStatus     c 200
   11 TxResponse   c Purged.
   11 TxHeader     c Server: Varnish
   11 TxHeader     c Content-Type: text/html; charset=utf-8
   11 TxHeader     c Retry-After: 5
   11 TxHeader     c Content-Length: 382
   11 TxHeader     c Accept-Ranges: bytes
   11 TxHeader     c Date: Tue, 13 Aug 2013 06:17:36 GMT
   11 TxHeader     c X-Varnish: 806561070
   11 TxHeader     c Age: 0
   11 TxHeader     c Via: 1.1 varnish
   11 TxHeader     c Connection: close
   11 TxHeader     c X-Cache: MISS
   11 Length       c 382
   11 ReqEnd       c 806561070 1376374656.189619780 1376374656.189839363 0.000064850 0.000157595 0.000061989
   11 SessionClose c error
   11 StatSess     c 10.2.35.5 36978 0 1 1 0 0 0 259 382


BUT, if I then immediately hit that site, I serves up a page from cache that should have just been purged.

   11 SessionOpen  c 10.2.35.5 36979 :6081
   11 ReqStart     c 10.2.35.5 36979 806561071
   11 RxRequest    c GET
   11 RxURL        c /
   11 RxProtocol   c HTTP/1.0
   11 RxHeader     c Host: staging.site.com.au<http://staging.site.com.au/>
   11 RxHeader     c X-Real-IP: 150.101.113.35
   11 RxHeader     c X-Forwarded-For: 150.101.113.35
   11 RxHeader     c Connection: close
   11 RxHeader     c Cache-Control: no-cache
   11 RxHeader     c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
   11 RxHeader     c Pragma: no-cache
   11 RxHeader     c User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36
   11 RxHeader     c Accept-Encoding: gzip,deflate,sdch
   11 RxHeader     c Accept-Language: en-US,en;q=0.8
   11 RxHeader     c Cookie: __gads=ID=1f70b5d5bf2eb903:T=1373344524:S=ALNI_MYS1RK-nxaklQLEIiCBMgSvpVDOXg; __utma=4009333.1369734685.1373238769.1373587007.1373846521.4; __utmc=4009333; __utmz=4009333.1373238769.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); the_cookie194=
   11 VCL_call     c recv lookup
   11 VCL_call     c hash
   11 Hash         c /
   11 Hash         c staging.site.com.au<http://staging.site.com.au/>
   11 VCL_return   c hash
   11 Hit          c 806560950
   11 VCL_call     c hit deliver
   11 VCL_call     c deliver deliver
   11 TxProtocol   c HTTP/1.1
   11 TxStatus     c 200
   11 TxResponse   c OK
   11 TxHeader     c Server: Apache/2.2.22 (Ubuntu)
   11 TxHeader     c X-Powered-By: PHP/5.3.10-1ubuntu3.7
   11 TxHeader     c X-Logged-In: False
   11 TxHeader     c P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
   11 TxHeader     c Content-Encoding: gzip
   11 TxHeader     c X-Content-Encoded-By: Joomla! 2.5
   11 TxHeader     c Content-Type: text/html; charset=utf-8
   11 TxHeader     c Content-Length: 12287
   11 TxHeader     c Accept-Ranges: bytes
   11 TxHeader     c Date: Tue, 13 Aug 2013 06:17:49 GMT
   11 TxHeader     c X-Varnish: 806561071 806560950
   11 TxHeader     c Age: 288
   11 TxHeader     c Via: 1.1 varnish
   11 TxHeader     c Connection: close
   11 TxHeader     c X-Cache: HIT
   11 TxHeader     c X-Cache-Hits: 4
   11 Length       c 12287
   11 ReqEnd       c 806561071 1376374669.010172367 1376374669.010374069 0.000091553 0.000092268 0.000109434
   11 SessionClose c Connection: close

Here are the relevant snippets from the default.vcl

sub vcl_recv {
....
..
.
    if (req.request == "PURGE") {
        if (!client.ip ~ purge) {
        error 405 "Not allowed.";
    }
         return (lookup);
}

.....
..
.
}


sub vcl_hit {
   if (req.request == "PURGE") {
    error 200 "Purged.";
    }
}

sub vcl_miss {
    if (req.request == "PURGE") {
    error 404 "Not in cache.";
  }
}

The cached content isn't something that has been generated after the cache being cleared, as its age precedes the PURGE being issued.

Unless someone out there knows of a varnish plugin for joomla (similar to the 'varnish http purge' available for wordpress), then I need to be able to manually issue cache purges.

Cheers,

--
Paul McInerney
Senior Systems Administrator
Faredge Technology

P:   +2  8425 1410
F:   +2  8425 1489
M: +61 435 348 435
E:   paul.mcinerney at faredge.com.au

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130814/2fd2c6be/attachment.html>

From perbu at varnish-software.com  Wed Aug 14 06:59:33 2013
From: perbu at varnish-software.com (Per Buer)
Date: Wed, 14 Aug 2013 08:59:33 +0200
Subject: varnish 3 - odd behaviour with using PURGE against cached objects
In-Reply-To: <CCD0821AE781994EB8BAB510676B5EF50120CD12A083@ZEUS.faredge.local>
References: <CCD0821AE781994EB8BAB510676B5EF50120CD12A083@ZEUS.faredge.local>
Message-ID: <CAOXZevA_fQCAMQGDqzQLya18GZXFPkTMnBcH9OU8CM=cyokpYQ@mail.gmail.com>

Hi Paul,


On Wed, Aug 14, 2013 at 7:56 AM, Paul McInerney <
Paul.McInerney at faredge.com.au> wrote:

> Hi all,****
>
> ** **
>
> I've got varnish currently configured to server cached content for 300
> secs. ****
>
> ** **
>
> I've also allowed varnish to accept purge requests (coming from a trusted
> list of IPs)****
>
> ** **
>
> Watching varnish (using varnishlog), I can see a  PURGE request get
> successfully handled (I am sending the purge via php/curl_init  - even if I
> telnet directly to varnish and issue a direct http request, I still get the
> same 'success')
>
(..)**
>
> Here are the relevant snippets from the default.vcl****
>
> ** **
>
> (..)
>
> sub vcl_hit {****
>
>    if (req.request == "PURGE") {****
>
>     error 200 "Purged.";****
>
>     }****
>
> }****
>
> ** **
>
> sub vcl_miss {****
>
>     if (req.request == "PURGE") {****
>
>     error 404 "Not in cache.";****
>
>   }****
>
> }****
>
> **
>

You need to add purge statements here. Make it look like this:


sub vcl_hit {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged.";
        }
}

sub vcl_miss {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged.";
        }
}



-- 
 <http://www.varnish-software.com/> *Per Buer*
CTO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130814/336dbdcc/attachment.html>

From mattias at nucleus.be  Wed Aug 14 07:21:24 2013
From: mattias at nucleus.be (Mattias Geniar)
Date: Wed, 14 Aug 2013 07:21:24 +0000
Subject: varnish 3 - odd behaviour with using PURGE against cached objects
In-Reply-To: <CCD0821AE781994EB8BAB510676B5EF50120CD12A083@ZEUS.faredge.local>
Message-ID: <CE30FC29.787D1%mattias@nucleus.be>

> The cached content isn't something that has been generated after the
>cache being cleared,
> as its age precedes the PURGE being issued.
> 
> Unless someone out there knows of a varnish plugin for joomla (similar
>to the 'varnish http
> purge' available for wordpress), then I need to be able to manually
>issue cache purges.
 

You are performing a purge of the cache from the non-gzipped content. You
purge command does not have the 'Accept-Encoding: gzip,deflate,sdch'
header, whereas your normal request does. Varnish is simply returning the
gzipped version of the cache where you only flushed the non-gzipped
version.

Easiest fix would be to do 2 purges, one with the Accept-Encoding header
(to flush gzipped content) and one without (to flush non-compressed
content).

Regards,
Mattias




From apj at mutt.dk  Wed Aug 14 07:36:14 2013
From: apj at mutt.dk (Andreas Plesner Jacobsen)
Date: Wed, 14 Aug 2013 09:36:14 +0200
Subject: varnish 3 - odd behaviour with using PURGE against cached objects
In-Reply-To: <CCD0821AE781994EB8BAB510676B5EF50120CD12A083@ZEUS.faredge.local>
References: <CCD0821AE781994EB8BAB510676B5EF50120CD12A083@ZEUS.faredge.local>
Message-ID: <20130814073614.GJ19694@nerd.dk>

On Wed, Aug 14, 2013 at 03:56:47PM +1000, Paul McInerney wrote:
> 
> sub vcl_hit {
>    if (req.request == "PURGE") {
>     error 200 "Purged.";
>     }
> }
> 
> sub vcl_miss {
>     if (req.request == "PURGE") {
>     error 404 "Not in cache.";
>   }
> }


You're missing the purge command in hit and miss.

-- 
Andreas



From apj at mutt.dk  Wed Aug 14 08:33:06 2013
From: apj at mutt.dk (Andreas Plesner Jacobsen)
Date: Wed, 14 Aug 2013 10:33:06 +0200
Subject: varnish 3 - odd behaviour with using PURGE against cached objects
In-Reply-To: <CE30FC29.787D1%mattias@nucleus.be>
References: <CCD0821AE781994EB8BAB510676B5EF50120CD12A083@ZEUS.faredge.local>
 <CE30FC29.787D1%mattias@nucleus.be>
Message-ID: <20130814083306.GK19694@nerd.dk>

On Wed, Aug 14, 2013 at 07:21:24AM +0000, Mattias Geniar wrote:
> > 
> > Unless someone out there knows of a varnish plugin for joomla (similar
> >to the 'varnish http
> > purge' available for wordpress), then I need to be able to manually
> >issue cache purges.
>  
> You are performing a purge of the cache from the non-gzipped content. You
> purge command does not have the 'Accept-Encoding: gzip,deflate,sdch'
> header, whereas your normal request does. Varnish is simply returning the
> gzipped version of the cache where you only flushed the non-gzipped
> version.

In varnish 3 there's only one copy. gzip is handled on the fly.

(the purge keyword will also purge all variants)

-- 
Andreas



From james at talkunafraid.co.uk  Wed Aug 14 08:57:05 2013
From: james at talkunafraid.co.uk (James Harrison)
Date: Wed, 14 Aug 2013 09:57:05 +0100
Subject: Sudden hit_for_pass
In-Reply-To: <CACj2-4LzvdofVm_iXA3mhj1PbWevNQUUWepi5pRd1JwOWer-dg@mail.gmail.com>
References: <51F92453.6060103@talkunafraid.co.uk>
 <CACj2-4+L=0vUXBnPbXEoLnc2+-fh02FTvSRm-tY-dUPK+8YB_Q@mail.gmail.com>
 <52011857.5050007@talkunafraid.co.uk>
 <CACj2-4LzvdofVm_iXA3mhj1PbWevNQUUWepi5pRd1JwOWer-dg@mail.gmail.com>
Message-ID: <520B4661.1020804@talkunafraid.co.uk>

On 07/08/13 00:01, Norberto Meijome wrote:
> Please make sure you CC the list ;-)
> 
> On 07/08/2013 1:38 AM, "James Harrison" <james at talkunafraid.co.uk
> <mailto:james at talkunafraid.co.uk>> wrote:
> 
>     On 01/08/13 23:53, Norberto Meijome wrote:
>     > Has your backend code changed, and cookies or other headers being sent
>     > back now?
>     >
> 
>     No.
> 
>     In addition to this sudden increase in hit_for_pass, my monitoring of
>     uptime is now showing a flat line at 23675208.05 days, transfer rates
>     are in the kilobytes rather than the usual 180M, and apparently I've got
>     1.8 million worker threads running. This all seems a bit unlikely, to
>     say the least, so I'm thinking shm bug or something along those lines?
> 
> 

Doh on my part.

Was indeed something odd going on with shm - a restart of Varnish
cleared the symptoms and everything is back to normal.


-- 
Cheers,
James Harrison



From gdarbinyan at bk.ru  Wed Aug 14 13:53:58 2013
From: gdarbinyan at bk.ru (=?UTF-8?B?R2V2b3JnIERhcmJpbnlhbg==?=)
Date: Wed, 14 Aug 2013 17:53:58 +0400
Subject: No subject
Message-ID: <1376488438.154450502@f358.i.mail.ru>

 Hi, welcome to you, ?how I can unsubscribe from your website.Thanks

-- 
Gevorg Darbinyan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130814/1fb001c4/attachment.html>

From perbu at varnish-software.com  Wed Aug 14 16:05:14 2013
From: perbu at varnish-software.com (Per Buer)
Date: Wed, 14 Aug 2013 18:05:14 +0200
Subject: No subject
In-Reply-To: <1376488438.154450502@f358.i.mail.ru>
References: <1376488438.154450502@f358.i.mail.ru>
Message-ID: <CAOXZevCydknWs5CVRkRwsP6ArA74mLdu5u21+obqsyPQrbRmgw@mail.gmail.com>

Hi,

Just click this link and press unsubscribe.
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc


On Wed, Aug 14, 2013 at 3:53 PM, Gevorg Darbinyan <gdarbinyan at bk.ru> wrote:

> Hi, welcome to you,  how I can unsubscribe from your website.Thanks
>
> --
> Gevorg Darbinyan
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
 <http://www.varnish-software.com/> *Per Buer*
CTO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130814/bf77dff8/attachment.html>

From rolmos at endertech.com  Thu Aug 15 04:34:28 2013
From: rolmos at endertech.com (Rob Olmos)
Date: Wed, 14 Aug 2013 21:34:28 -0700
Subject: Use ETag or Last-Modified to utilize 304 Not Modified until ESI
 fragment has expired then send a 200
Message-ID: <520C5A54.8000909@endertech.com>

I believe this may be more of a feature request because I'm unable to 
find anything in regards to this but basically I would like to utilize 
"304 Not Modified" responses until an ESI fragment has expired and then 
send a "200 OK" with the full document and updated ESI fragment.

Right now I have a parent document that has a couple ESI fragments 
within it. Essentially I would like to set a cache header, such as an 
ETag, on the parent document so that client browsers can utilize 
If-None-Match or If-Modified-Since and receive "304 Not Modified" 
instead of a 200 with the whole document thus saving bandwidth.

Is there a way to check within the VCL parent document request that an 
ESI fragment has expired?

I'm thinking that would allow me to update the Etag or Last-Modified 
headers of the parent document. However, it would also need to update 
the parent document cached object with these new values as well and 
Varnish would need to be aware of that so it doesn't send a 304 but 
instead a 200.

Thanks

-- 
Rob Olmos
EnderTech



From leiwang at rhapsody.com  Thu Aug 15 18:09:13 2013
From: leiwang at rhapsody.com (Lei Wang)
Date: Thu, 15 Aug 2013 11:09:13 -0700
Subject: varnish forward to wrong backend
Message-ID: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>

I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
multi-site in one configuration. Sometimes Varnish switched to use the
first backend in the configuration for all the requests to the other sites.
As in the sample configuration below, the request to backend2, backend3,
backend4 sometimes forwarded to backend1. Does anyone have the same problem
or know any solution, suggestion?

The varnish backend configuration is quite simple like below:

backend backend1 {
  .host = "backend1.myserver.com";
  .port = "80";
  .connect_timeout = 20s;
  .first_byte_timeout = 60s;
  .between_bytes_timeout = 60s;
}

backend backend2 {
  .host = "backend2.myserver.com";
  .port = "80";
  .connect_timeout = 20s;
  .first_byte_timeout = 60s;
  .between_bytes_timeout = 60s;
}

backend backend3 {
  .host = "backend3.myserver.com";
  .port = "80";
  .connect_timeout = 20s;
  .first_byte_timeout = 60s;
  .between_bytes_timeout = 60s;
}

backend backend4 {
  .host = "backend4.myserver.com";
  .port = "80";
  .connect_timeout = 20s;
  .first_byte_timeout = 60s;
  .between_bytes_timeout = 60s;
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130815/c2fd4577/attachment.html>

From james at ifixit.com  Thu Aug 15 21:12:24 2013
From: james at ifixit.com (James Pearson)
Date: Thu, 15 Aug 2013 14:12:24 -0700
Subject: varnish forward to wrong backend
In-Reply-To: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
References: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
Message-ID: <1376601092-sup-7048@geror.local>

Excerpts from Lei Wang's message of 2013-08-15 11:09:13 -0700:
> I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
> multi-site in one configuration. Sometimes Varnish switched to use the
> first backend in the configuration for all the requests to the other sites.
> As in the sample configuration below, the request to backend2, backend3,
> backend4 sometimes forwarded to backend1. Does anyone have the same problem
> or know any solution, suggestion?
> 
> The varnish backend configuration is quite simple like below:
> 
> [snip]

What does the rest of your config look like, specifically the part that assigns
a backend?
 - P



From leiwang at rhapsody.com  Thu Aug 15 22:11:09 2013
From: leiwang at rhapsody.com (Lei Wang)
Date: Thu, 15 Aug 2013 15:11:09 -0700
Subject: varnish forward to wrong backend
In-Reply-To: <1376601092-sup-7048@geror.local>
References: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
 <1376601092-sup-7048@geror.local>
Message-ID: <CAD_kORdJ_nnd5GzJ_3d6fO5eZ0yVg+X4Bx=S6=Jox22kACKAZQ@mail.gmail.com>

Hi James,

It is very simple like below:
sub vcl_recv {
  if (req.url ~ "^/my-home") {
    set req.http.host = "www.mydns1.com";
    set req.backend = backend1 ;
  } elsif (req.http.host ~ "^www.mydns2.com") {
    set req.http.host = "www.mydns2.com";
    set req.backend = backend2
  } elsif (req.http.host ~ "^www.mydns3.com") {
    unset req.http.Accept;
    set req.backend = backend3;
  } elsif (req.http.host ~ "^www.mydns4.com") {
    unset req.http.Accept;
    set req.backend = backend4;
  } else {
    error 404 "Unknown virtual host";
  }
}


Thanks,

Lei


On Thu, Aug 15, 2013 at 2:12 PM, James Pearson <james at ifixit.com> wrote:

> Excerpts from Lei Wang's message of 2013-08-15 11:09:13 -0700:
> > I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
> > multi-site in one configuration. Sometimes Varnish switched to use the
> > first backend in the configuration for all the requests to the other
> sites.
> > As in the sample configuration below, the request to backend2, backend3,
> > backend4 sometimes forwarded to backend1. Does anyone have the same
> problem
> > or know any solution, suggestion?
> >
> > The varnish backend configuration is quite simple like below:
> >
> > [snip]
>
> What does the rest of your config look like, specifically the part that
> assigns
> a backend?
>  - P
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130815/2080eea0/attachment.html>

From Travis.Crowder at penton.com  Thu Aug 15 22:35:40 2013
From: Travis.Crowder at penton.com (Crowder, Travis)
Date: Thu, 15 Aug 2013 22:35:40 +0000
Subject: varnish forward to wrong backend
In-Reply-To: <CAD_kORdJ_nnd5GzJ_3d6fO5eZ0yVg+X4Bx=S6=Jox22kACKAZQ@mail.gmail.com>
References: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
 <1376601092-sup-7048@geror.local>,
 <CAD_kORdJ_nnd5GzJ_3d6fO5eZ0yVg+X4Bx=S6=Jox22kACKAZQ@mail.gmail.com>
Message-ID: <F7A3490F-1E31-4068-824B-1B9347A5FFB9@penton.com>

It is probably that the other domains have the same URL.  Check for www.mydns1.com<http://www.mydns1.com> in your first conditional.

-Travis Crowder

Sent from my iPhone

On Aug 15, 2013, at 5:20 PM, "Lei Wang" <leiwang at rhapsody.com<mailto:leiwang at rhapsody.com>> wrote:

Hi James,

It is very simple like below:
sub vcl_recv {
  if (req.url ~ "^/my-home") {
    set req.http.host = "www.mydns1.com<http://www.mydns1.com>";
    set req.backend = backend1 ;
  } elsif (req.http.host ~ "^www.mydns2.com<http://www.mydns2.com>") {
    set req.http.host = "www.mydns2.com<http://www.mydns2.com>";
    set req.backend = backend2
  } elsif (req.http.host ~ "^www.mydns3.com<http://www.mydns3.com>") {
    unset req.http.Accept;
    set req.backend = backend3;
  } elsif (req.http.host ~ "^www.mydns4.com<http://www.mydns4.com>") {
    unset req.http.Accept;
    set req.backend = backend4;
  } else {
    error 404 "Unknown virtual host";
  }
}


Thanks,

Lei


On Thu, Aug 15, 2013 at 2:12 PM, James Pearson <james at ifixit.com<mailto:james at ifixit.com>> wrote:
Excerpts from Lei Wang's message of 2013-08-15 11:09:13 -0700:
> I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
> multi-site in one configuration. Sometimes Varnish switched to use the
> first backend in the configuration for all the requests to the other sites.
> As in the sample configuration below, the request to backend2, backend3,
> backend4 sometimes forwarded to backend1. Does anyone have the same problem
> or know any solution, suggestion?
>
> The varnish backend configuration is quite simple like below:
>
> [snip]

What does the rest of your config look like, specifically the part that assigns
a backend?
 - P

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130815/56aa3823/attachment.html>

From leiwang at rhapsody.com  Thu Aug 15 23:05:17 2013
From: leiwang at rhapsody.com (Lei Wang)
Date: Thu, 15 Aug 2013 16:05:17 -0700
Subject: varnish forward to wrong backend
Message-ID: <h0xdmv3gahl7k8k397eebx9g.1376607917412@email.android.com>

No. That is not the case.

Thanks,

Lei

-------- Original message --------
From: "Crowder, Travis" <Travis.Crowder at penton.com> 
Date: 08/15/2013  3:35 PM  (GMT-08:00) 
To: Lei Wang <leiwang at rhapsody.com> 
Cc: James Pearson <james at ifixit.com>,varnish-misc <varnish-misc at varnish-cache.org> 
Subject: Re: varnish forward to wrong backend 
 
It is probably that the other domains have the same URL.? Check for www.mydns1.com<http://www.mydns1.com> in your first conditional.

-Travis Crowder

Sent from my iPhone

On Aug 15, 2013, at 5:20 PM, "Lei Wang" <leiwang at rhapsody.com<mailto:leiwang at rhapsody.com>> wrote:

Hi James,

It is very simple like below:
sub vcl_recv {
? if (req.url ~ "^/my-home") {
??? set req.http.host = "www.mydns1.com<http://www.mydns1.com>";
??? set req.backend = backend1 ;
? } elsif (req.http.host ~ "^www.mydns2.com<http://www.mydns2.com>") {
??? set req.http.host = "www.mydns2.com<http://www.mydns2.com>";
??? set req.backend = backend2
? } elsif (req.http.host ~ "^www.mydns3.com<http://www.mydns3.com>") {
??? unset req.http.Accept;
??? set req.backend = backend3;
? } elsif (req.http.host ~ "^www.mydns4.com<http://www.mydns4.com>") {
??? unset req.http.Accept;
??? set req.backend = backend4;
? } else {
??? error 404 "Unknown virtual host";
? }
}


Thanks,

Lei


On Thu, Aug 15, 2013 at 2:12 PM, James Pearson <james at ifixit.com<mailto:james at ifixit.com>> wrote:
Excerpts from Lei Wang's message of 2013-08-15 11:09:13 -0700:
> I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
> multi-site in one configuration. Sometimes Varnish switched to use the
> first backend in the configuration for all the requests to the other sites.
> As in the sample configuration below, the request to backend2, backend3,
> backend4 sometimes forwarded to backend1. Does anyone have the same problem
> or know any solution, suggestion?
>
> The varnish backend configuration is quite simple like below:
>
> [snip]

What does the rest of your config look like, specifical
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130815/80dd4aba/attachment.html>

From alex at babyonline.com.sg  Fri Aug 16 04:26:49 2013
From: alex at babyonline.com.sg (Alex Soo)
Date: Fri, 16 Aug 2013 12:26:49 +0800
Subject: Varnish restart a few times a day
Message-ID: <C281C498-DEC4-406B-BF31-B5D001B4293F@babyonline.com.sg>

Hi,

I am new to varnish and has very limited knowledge in programing or UNIX

I am hosting my own web site on a Mac Mini server. It is all setup and install by freelance programer.

I have notice that varnish is restarting many times a day through log file.

Below is an example :

Process:         varnishd [8259]
Path:            /usr/local/sbin/varnishd
Identifier:      varnishd
Version:         0
Code Type:       X86-64 (Native)
Parent Process:  varnishd [173]
User ID:         -2

Date/Time:       2013-08-16 04:53:47.253 +0800
OS Version:      Mac OS X 10.8.4 (12E55)
Report Version:  10

Crashed Thread:  16

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000

Application Specific Information:
abort() called

Thread 0:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff8f5e4f96 poll + 10
1   libvarnish.dylib              	0x0000000105597ec8 VCLS_Poll + 376 (cli_serve.c:519)
2   varnishd                      	0x0000000105512501 CLI_Run + 129 (cache_cli.c:113)
3   varnishd                      	0x00000001055261de child_main + 398 (cache_main.c:140)
4   varnishd                      	0x0000000105539c04 start_child + 948 (mgt_child.c:345)
5   varnishd                      	0x000000010553abb8 mgt_sigchld + 1496 (mgt_child.c:524)
6   libvarnish.dylib              	0x000000010559a9e5 vev_sched_signal + 149
7   libvarnish.dylib              	0x000000010559ad7a vev_schedule_one + 746 (vev.c:474)
8   libvarnish.dylib              	0x000000010559b0c8 vev_schedule + 200 (vev.c:363)
9   varnishd                      	0x000000010553a535 MGT_Run + 773 (mgt_child.c:604)
10  varnishd                      	0x000000010554a523 main + 4003 (varnishd.c:650)
11  libdyld.dylib                 	0x00007fff8da497e1 start + 1

Thread 1:
0   libsystem_kernel.dylib        	0x00007fff8f5e4386 __semwait_signal + 10
1   libsystem_c.dylib             	0x00007fff94839800 nanosleep + 163
2   libvarnish.dylib              	0x0000000105599838 TIM_sleep + 56 (time.c:177)
3   varnishd                      	0x0000000105528c70 wrk_herdtimer_thread + 528 (cache_pool.c:446)
4   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
5   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 2:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105528ec0 wrk_herder_thread + 176 (cache_pool.c:526)
4   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
5   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 3:
0   libsystem_kernel.dylib        	0x00007fff8f5e4386 __semwait_signal + 10
1   libsystem_c.dylib             	0x00007fff94839800 nanosleep + 163
2   libvarnish.dylib              	0x0000000105599838 TIM_sleep + 56 (time.c:177)
3   varnishd                      	0x000000010551a64e exp_timer + 142 (cache_expire.c:351)
4   varnishd                      	0x00000001055286ae wrk_bgthread + 286 (cache_pool.c:564)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 4:
0   libsystem_kernel.dylib        	0x00007fff8f5e4386 __semwait_signal + 10
1   libsystem_c.dylib             	0x00007fff94839800 nanosleep + 163
2   libvarnish.dylib              	0x0000000105599838 TIM_sleep + 56 (time.c:177)
3   varnishd                      	0x0000000105538cfc hcb_cleaner + 348 (hash_critbit.c:358)
4   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
5   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 5:
0   libsystem_kernel.dylib        	0x00007fff8f5e4386 __semwait_signal + 10
1   libsystem_c.dylib             	0x00007fff94839800 nanosleep + 163
2   libvarnish.dylib              	0x0000000105599838 TIM_sleep + 56 (time.c:177)
3   varnishd                      	0x000000010550b7d8 ban_lurker + 472 (cache_ban.c:922)
4   varnishd                      	0x00000001055286ae wrk_bgthread + 286 (cache_pool.c:564)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 6:
0   libsystem_kernel.dylib        	0x00007fff8f5e4d16 kevent + 10
1   varnishd                      	0x000000010550415e vca_kqueue_main + 382 (cache_waiter_kqueue.c:168)
2   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
3   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 7:
0   libsystem_kernel.dylib        	0x00007fff8f5e4f96 poll + 10
1   varnishd                      	0x00000001055033d8 vca_acct + 1000 (cache_acceptor.c:272)
2   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
3   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 8:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105529991 wrk_thread_real + 1297
4   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 9:
0   libsystem_kernel.dylib        	0x00007fff8f5e4f96 poll + 10
1   varnishd                      	0x000000010550d4d0 CNT_Session + 1392 (cache_center.c:102)
2   varnishd                      	0x0000000105529a77 wrk_thread_real + 1527
3   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
4   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
5   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 10:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105529991 wrk_thread_real + 1297
4   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 11:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105529991 wrk_thread_real + 1297
4   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 12:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105529991 wrk_thread_real + 1297
4   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 13:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105529991 wrk_thread_real + 1297
4   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 14:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105529991 wrk_thread_real + 1297
4   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 15:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105529991 wrk_thread_real + 1297
4   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 16 Crashed:
0   libsystem_kernel.dylib        	0x00007fff8f5e4212 __pthread_kill + 10
1   libsystem_c.dylib             	0x00007fff947b0b54 pthread_kill + 90
2   libsystem_c.dylib             	0x00007fff947f4dce abort + 143
3   varnishd                      	0x0000000105527c57 pan_ic + 2679
4   varnishd                      	0x0000000105503e2a VCA_Prep + 1226 (cache_acceptor.c:184)
5   varnishd                      	0x000000010550d642 CNT_Session + 1762 (cache_center.c:974)
6   varnishd                      	0x0000000105529a77 wrk_thread_real + 1527
7   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
8   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
9   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 17:
0   libsystem_kernel.dylib        	0x00007fff8f5e40fa __psynch_cvwait + 10
1   libsystem_c.dylib             	0x00007fff947b3fe9 _pthread_cond_wait + 869
2   varnishd                      	0x0000000105526d99 Lck_CondWait + 281 (cache_lck.c:151)
3   varnishd                      	0x0000000105529991 wrk_thread_real + 1297
4   varnishd                      	0x000000010552920f wrk_thread + 287 (cache_pool.c:235)
5   libsystem_c.dylib             	0x00007fff947af7a2 _pthread_start + 327
6   libsystem_c.dylib             	0x00007fff9479c1e1 thread_start + 13

Thread 16 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000000000006  rcx: 0x000000010b2f6298  rdx: 0x0000000000000000
  rdi: 0x0000000000002913  rsi: 0x0000000000000006  rbp: 0x000000010b2f62c0  rsp: 0x000000010b2f6298
   r8: 0x00007fff7e458278   r9: 0x000000010b2f61d0  r10: 0x0000000020000000  r11: 0x0000000000000206
  r12: 0x0000000105552701  r13: 0x0000000105552710  r14: 0x000000010b30a000  r15: 0x00000001055c8488
  rip: 0x00007fff8f5e4212  rfl: 0x0000000000000206  cr2: 0x00007fff7e4600b0
Logical CPU: 0

Binary Images:
       0x105502000 -        0x105568fe7 +varnishd (0) <04E977E5-715A-30FA-949C-C962BDA3A4C8> /usr/local/sbin/varnishd
       0x105594000 -        0x1055a3fe7 +libvarnish.dylib (0) <01EF19F6-8B8A-3DD1-9F45-34A57BF5365A> /usr/local/lib/varnish/libvarnish.dylib
       0x1055b1000 -        0x1055b1fff +libvarnishcompat.dylib (0) <9F1C26EE-15DC-3E11-B971-A056011EB26F> /usr/local/lib/varnish/libvarnishcompat.dylib
       0x1055b6000 -        0x1055d0ff7 +libvcl.dylib (0) <11184ED1-A285-356E-BF03-6AA94F76C3DD> /usr/local/lib/varnish/libvcl.dylib
       0x1055da000 -        0x1055ebfef +libvgz.dylib (0) <F6834A25-2001-32A9-A5C6-07AE002E150E> /usr/local/lib/varnish/libvgz.dylib
       0x1055f0000 -        0x105607fef +libpcre.0.dylib (0) <6169CF5A-DB62-34BB-9976-549B6C65629E> /usr/local/lib/libpcre.0.dylib
       0x10ab4b000 -        0x10ab4dff7 +vcl.JJrPdxqR.so (0) <4FF44DF6-ED37-3877-A1B7-C8D53FE73DF0> /usr/local/var/varnish/miniserver.local/vcl.JJrPdxqR.so
    0x7fff65102000 -     0x7fff6513693f  dyld (210.2.3) <6900F2BA-DB48-3B78-B668-58FC0CF6BCB8> /usr/lib/dyld
    0x7fff8d7d8000 -     0x7fff8d7e0fff  liblaunch.dylib (442.26.2) <2F71CAF8-6524-329E-AC56-C506658B4C0C> /usr/lib/system/liblaunch.dylib
    0x7fff8da47000 -     0x7fff8da4aff7  libdyld.dylib (210.2.3) <F59367C9-C110-382B-A695-9035A6DD387E> /usr/lib/system/libdyld.dylib
    0x7fff8da4b000 -     0x7fff8da4dff7  libunc.dylib (25) <92805328-CD36-34FF-9436-571AB0485072> /usr/lib/system/libunc.dylib
    0x7fff8e471000 -     0x7fff8e472ff7  libremovefile.dylib (23.2) <6763BC8E-18B8-3AD9-8FFA-B43713A7264F> /usr/lib/system/libremovefile.dylib
    0x7fff8e4e6000 -     0x7fff8e535ff7  libcorecrypto.dylib (106.2) <CE0C29A3-C420-339B-ADAA-52F4683233CC> /usr/lib/system/libcorecrypto.dylib
    0x7fff8e956000 -     0x7fff8e957ff7  libdnsinfo.dylib (453.19) <14202FFB-C3CA-3FCC-94B0-14611BF8692D> /usr/lib/system/libdnsinfo.dylib
    0x7fff8f5d2000 -     0x7fff8f5edff7  libsystem_kernel.dylib (2050.24.15) <A9F97289-7985-31D6-AF89-151830684461> /usr/lib/system/libsystem_kernel.dylib
    0x7fff90f5f000 -     0x7fff90f61fff  libquarantine.dylib (52.1) <143B726E-DF47-37A8-90AA-F059CFD1A2E4> /usr/lib/system/libquarantine.dylib
    0x7fff912a4000 -     0x7fff912a9fff  libcache.dylib (57) <65187C6E-3FBF-3EB8-A1AA-389445E2984D> /usr/lib/system/libcache.dylib
    0x7fff91504000 -     0x7fff91529ff7  libc++abi.dylib (26) <D86169F3-9F31-377A-9AF3-DB17142052E4> /usr/lib/libc++abi.dylib
    0x7fff91580000 -     0x7fff915e8ff7  libc++.1.dylib (65.1) <20E31B90-19B9-3C2A-A9EB-474E08F9FE05> /usr/lib/libc++.1.dylib
    0x7fff922df000 -     0x7fff922eafff  libsystem_notify.dylib (98.5) <C49275CC-835A-3207-AFBA-8C01374927B6> /usr/lib/system/libsystem_notify.dylib
    0x7fff92336000 -     0x7fff92382ff7  libauto.dylib (185.4) <AD5A4CE7-CB53-313C-9FAE-673303CC2D35> /usr/lib/libauto.dylib
    0x7fff9479b000 -     0x7fff94867ff7  libsystem_c.dylib (825.26) <4C9EB006-FE1F-3F8F-8074-DFD94CF2CE7B> /usr/lib/system/libsystem_c.dylib
    0x7fff94df6000 -     0x7fff94dfcff7  libunwind.dylib (35.1) <21703D36-2DAB-3D8B-8442-EAAB23C060D3> /usr/lib/system/libunwind.dylib
    0x7fff94fdb000 -     0x7fff94ff0ff7  libdispatch.dylib (228.23) <D26996BF-FC57-39EB-8829-F63585561E09> /usr/lib/system/libdispatch.dylib
    0x7fff959d8000 -     0x7fff959d9fff  libsystem_blocks.dylib (59) <D92DCBC3-541C-37BD-AADE-ACC75A0C59C8> /usr/lib/system/libsystem_blocks.dylib
    0x7fff95b07000 -     0x7fff95b0cfff  libcompiler_rt.dylib (30) <08F8731D-5961-39F1-AD00-4590321D24A9> /usr/lib/system/libcompiler_rt.dylib
    0x7fff95ce6000 -     0x7fff95ce7fff  libDiagnosticMessagesClient.dylib (8) <8548E0DC-0D2F-30B6-B045-FE8A038E76D8> /usr/lib/libDiagnosticMessagesClient.dylib
    0x7fff95df4000 -     0x7fff95e2afff  libsystem_info.dylib (406.17) <4FFCA242-7F04-365F-87A6-D4EFB89503C1> /usr/lib/system/libsystem_info.dylib
    0x7fff95e3e000 -     0x7fff95e4cff7  libsystem_network.dylib (77.10) <0D99F24E-56FE-380F-B81B-4A4C630EE587> /usr/lib/system/libsystem_network.dylib
    0x7fff9734d000 -     0x7fff97355ff7  libsystem_dnssd.dylib (379.38.1) <BDCB8566-0189-34C0-9634-35ABD3EFE25B> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff97e42000 -     0x7fff97f5a92f  libobjc.A.dylib (532.2) <90D31928-F48D-3E37-874F-220A51FD9E37> /usr/lib/libobjc.A.dylib
    0x7fff98589000 -     0x7fff9858ffff  libmacho.dylib (829) <BF332AD9-E89F-387E-92A4-6E1AB74BD4D9> /usr/lib/system/libmacho.dylib
    0x7fff987f7000 -     0x7fff98819ff7  libxpc.dylib (140.43) <70BC645B-6952-3264-930C-C835010CCEF9> /usr/lib/system/libxpc.dylib
    0x7fff989bb000 -     0x7fff989bcff7  libsystem_sandbox.dylib (220.3) <B739DA63-B675-387A-AD84-412A651143C0> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff99674000 -     0x7fff99682fff  libcommonCrypto.dylib (60027) <BAAFE0C9-BB86-3CA7-88C0-E3CBA98DA06F> /usr/lib/system/libcommonCrypto.dylib
    0x7fff99bd8000 -     0x7fff99bd9ff7  libSystem.B.dylib (169.3) <FF25248A-574C-32DB-952F-B948C389B2A4> /usr/lib/libSystem.B.dylib
    0x7fff9a0c1000 -     0x7fff9a0c8fff  libcopyfile.dylib (89) <876573D0-E907-3566-A108-577EAD1B6182> /usr/lib/system/libcopyfile.dylib
    0x7fff9a3c1000 -     0x7fff9a3c1fff  libkeymgr.dylib (25) <CC9E3394-BE16-397F-926B-E579B60EE429> /usr/lib/system/libkeymgr.dylib
    0x7fff9a3c2000 -     0x7fff9a3f0ff7  libsystem_m.dylib (3022.6) <B434BE5C-25AB-3EBD-BAA7-5304B34E3441> /usr/lib/system/libsystem_m.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 420
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 178951
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=57.6M resident=28.6M(50%) swapped_out_or_unallocated=29.1M(50%)
Writable regions: Total=92.9M written=7040K(7%) resident=8376K(9%) swapped_out=0K(0%) unallocated=84.7M(91%)
 
REGION TYPE                      VIRTUAL
===========                      =======
MALLOC                             75.9M
MALLOC guard page                    16K
STACK GUARD                        56.1M
Stack                              16.6M
VM_ALLOCATE                           4K
__DATA                              684K
__LINKEDIT                         52.4M
__TEXT                             5340K
mapped file                        81.0M
shared memory                        12K
===========                      =======
TOTAL                             287.9M

Any advise where I should start looking at? Thanks in advance.

Thanks
Alex Soo





From perbu at varnish-software.com  Fri Aug 16 08:10:07 2013
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 16 Aug 2013 10:10:07 +0200
Subject: varnish forward to wrong backend
In-Reply-To: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
References: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
Message-ID: <CAOXZevB44eG8uaVa0HY21o16FjhDvfdGHQnO1dX5+8uQaKqouw@mail.gmail.com>

Hi Lei,

On Thu, Aug 15, 2013 at 8:09 PM, Lei Wang <leiwang at rhapsody.com> wrote:

> I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
> multi-site in one configuration. Sometimes Varnish switched to use the
> first backend in the configuration for all the requests to the other sites.
> As in the sample configuration below, the request to backend2, backend3,
> backend4 sometimes forwarded to backend1. Does anyone have the same problem
> or know any solution, suggestion?
>

Could you see if you can post a varnishlog statement showing this
behaviour? It should be fairly easy to find one if it happens regularly.


Cheers,

Per.

-- 
 <http://www.varnish-software.com/> *Per Buer*
CTO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130816/f05832f5/attachment.html>

From leiwang at rhapsody.com  Fri Aug 16 12:21:35 2013
From: leiwang at rhapsody.com (Lei Wang)
Date: Fri, 16 Aug 2013 05:21:35 -0700
Subject: varnish forward to wrong backend
In-Reply-To: <CAOXZevB44eG8uaVa0HY21o16FjhDvfdGHQnO1dX5+8uQaKqouw@mail.gmail.com>
References: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
 <CAOXZevB44eG8uaVa0HY21o16FjhDvfdGHQnO1dX5+8uQaKqouw@mail.gmail.com>
Message-ID: <CAD_kORfhpoenB6W37vTGKXLGki9Gy2k5bB5GFBYRzuNxmT5xuw@mail.gmail.com>

Hi Per,

There are not in varnishlog and even varnishncsa log. There only trace may
useful is the captured network tcpdump. It is quite big ans shows load
balance sent correct header and url to varnish listen port, then passed to
wrong backend.

Thanks,

Lei


On Fri, Aug 16, 2013 at 1:10 AM, Per Buer <perbu at varnish-software.com>wrote:

> Hi Lei,
>
> On Thu, Aug 15, 2013 at 8:09 PM, Lei Wang <leiwang at rhapsody.com> wrote:
>
>> I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
>> multi-site in one configuration. Sometimes Varnish switched to use the
>> first backend in the configuration for all the requests to the other sites.
>> As in the sample configuration below, the request to backend2, backend3,
>> backend4 sometimes forwarded to backend1. Does anyone have the same problem
>> or know any solution, suggestion?
>>
>
> Could you see if you can post a varnishlog statement showing this
> behaviour? It should be fairly easy to find one if it happens regularly.
>
>
> Cheers,
>
> Per.
>
> --
>  <http://www.varnish-software.com/> *Per Buer*
> CTO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130816/aea39150/attachment.html>

From perbu at varnish-software.com  Fri Aug 16 13:20:41 2013
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 16 Aug 2013 15:20:41 +0200
Subject: varnish forward to wrong backend
In-Reply-To: <CAD_kORfhpoenB6W37vTGKXLGki9Gy2k5bB5GFBYRzuNxmT5xuw@mail.gmail.com>
References: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
 <CAOXZevB44eG8uaVa0HY21o16FjhDvfdGHQnO1dX5+8uQaKqouw@mail.gmail.com>
 <CAD_kORfhpoenB6W37vTGKXLGki9Gy2k5bB5GFBYRzuNxmT5xuw@mail.gmail.com>
Message-ID: <CAOXZevDPKKHCeUi2nQnObhb+JOpgviR4gt0XVHiqFe=ghm2qKA@mail.gmail.com>

Hi Lei,

It sounds like they are somehow bypassing your Varnish installation. If
they where handled by Varnish they would be in varnishlog. Maybe the
loadbalancer is doing something weird? Or the webservers have an exposed IP
address outside the LB?

Per.


On Fri, Aug 16, 2013 at 2:21 PM, Lei Wang <leiwang at rhapsody.com> wrote:

> Hi Per,
>
> There are not in varnishlog and even varnishncsa log. There only trace may
> useful is the captured network tcpdump. It is quite big ans shows load
> balance sent correct header and url to varnish listen port, then passed to
> wrong backend.
>
> Thanks,
>
> Lei
>
>
> On Fri, Aug 16, 2013 at 1:10 AM, Per Buer <perbu at varnish-software.com>wrote:
>
>> Hi Lei,
>>
>> On Thu, Aug 15, 2013 at 8:09 PM, Lei Wang <leiwang at rhapsody.com> wrote:
>>
>>> I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
>>> multi-site in one configuration. Sometimes Varnish switched to use the
>>> first backend in the configuration for all the requests to the other sites.
>>> As in the sample configuration below, the request to backend2, backend3,
>>> backend4 sometimes forwarded to backend1. Does anyone have the same problem
>>> or know any solution, suggestion?
>>>
>>
>> Could you see if you can post a varnishlog statement showing this
>> behaviour? It should be fairly easy to find one if it happens regularly.
>>
>>
>> Cheers,
>>
>> Per.
>>
>> --
>>  <http://www.varnish-software.com/> *Per Buer*
>> CTO | Varnish Software AS
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>> Winner of the Red Herring Top 100 Europe Award 2013
>>
>>
>>
>


-- 
 <http://www.varnish-software.com/> *Per Buer*
CTO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130816/74b427a9/attachment.html>

From bstillwell at photobucket.com  Fri Aug 16 17:29:46 2013
From: bstillwell at photobucket.com (Bryan Stillwell)
Date: Fri, 16 Aug 2013 11:29:46 -0600
Subject: High read IOPS with file backend
Message-ID: <CADAfL_hV2-PyTfaQkX6n8ENMGM+knLYSEaR8d_f-+Pf=C+qB5g@mail.gmail.com>

I'm looking at converting all of our nginx caching servers over to
varnish, but I'm seeing some oddities with the 'file' storage backend
that I'm hoping someone could shed some light on.

Hardware specs for both machines:

Dell PowerEdge R610
  Dual Xeon E5620 @ 2.40GHz
  64GiB memory
  4x 240GB SSDs in a RAID5 attached to a PERC H700 RAID controller

The following image compares CPU, memory, and IOPS for both a machine
running varnish (left), and one running nginx (right):

http://i1217.photobucket.com/albums/dd391/bstillwell_pb/Graphs/varnish-vs-nginx.png

At the most recent sample, each machine was handling ~350 requests/sec.

As you can see the varnish machine has a lot more CPU time dedicated to
I/O wait, which matches up with ~5x higher IOPS numbers.  However, the
biggest difference is that varnish is using ~25x more read IOPS than
nginx.

As for the jumps in the IOPS graph, I believe they can be explained by:

Wednesday @ 10:00a: Started taking traffic
Wednesday @ 11:00a: Memory cache filled, started using SSDs
Wednesday @  4:00p: TTL of 6 hours was hit, objects start expiring
Wednesday @  7:15p: SSD cache filled

I pre-allocated the storage with fallocate (fallocate -l 450g
/cache/varnish_storage.bin) to make sure that wasn't helping contribute
to the issue.

Any ideas on what could be tuned to reduce the number of read IOPS to be
more inline with nginx?

Thanks,
Bryan



From leiwang at rhapsody.com  Fri Aug 16 18:08:57 2013
From: leiwang at rhapsody.com (Lei Wang)
Date: Fri, 16 Aug 2013 11:08:57 -0700
Subject: varnish forward to wrong backend
In-Reply-To: <CAOXZevDPKKHCeUi2nQnObhb+JOpgviR4gt0XVHiqFe=ghm2qKA@mail.gmail.com>
References: <CAD_kORdw2Ewvm8FH+mmUt+m2WX0drCu_QQJB5xG9Mp1SADkp_A@mail.gmail.com>
 <CAOXZevB44eG8uaVa0HY21o16FjhDvfdGHQnO1dX5+8uQaKqouw@mail.gmail.com>
 <CAD_kORfhpoenB6W37vTGKXLGki9Gy2k5bB5GFBYRzuNxmT5xuw@mail.gmail.com>
 <CAOXZevDPKKHCeUi2nQnObhb+JOpgviR4gt0XVHiqFe=ghm2qKA@mail.gmail.com>
Message-ID: <CAD_kORe0qwq6Z12LMpbX092+9xi9d_Vja7sg1oNbCWaDf9JEOA@mail.gmail.com>

Hi Per,

>From the tcpdump, the request was passed to the varnish listening port (80)
and then passed to the wrong backend. The weird thing was all the bad
transaction passed to backend do not have any varnish related header as it
should be.

Good one
GET /6/3/3/6/4/221446336.ea HTTP/1.1
User-Agent: Java/1.4.2_19
Host: cache.internal.myserver.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
X-Forwarded-For: 10.150.4.118, 10.150.0.220
X-Varnish: 570940624

Bad one
GET /6/3/3/6/4/221446336.ea HTTP/1.1
User-Agent: Java/1.4.2_19
Host: cache.internal.myserver.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
X-Forwarded-For: 10.150.0.220

Thanks,

Lei


On Fri, Aug 16, 2013 at 6:20 AM, Per Buer <perbu at varnish-software.com>wrote:

> Hi Lei,
>
> It sounds like they are somehow bypassing your Varnish installation. If
> they where handled by Varnish they would be in varnishlog. Maybe the
> loadbalancer is doing something weird? Or the webservers have an exposed IP
> address outside the LB?
>
> Per.
>
>
> On Fri, Aug 16, 2013 at 2:21 PM, Lei Wang <leiwang at rhapsody.com> wrote:
>
>> Hi Per,
>>
>> There are not in varnishlog and even varnishncsa log. There only trace
>> may useful is the captured network tcpdump. It is quite big ans shows load
>> balance sent correct header and url to varnish listen port, then passed to
>> wrong backend.
>>
>> Thanks,
>>
>> Lei
>>
>>
>> On Fri, Aug 16, 2013 at 1:10 AM, Per Buer <perbu at varnish-software.com>wrote:
>>
>>> Hi Lei,
>>>
>>> On Thu, Aug 15, 2013 at 8:09 PM, Lei Wang <leiwang at rhapsody.com> wrote:
>>>
>>>> I have varnish 3 (varnish-3.0.4 revision 9f83e8f) installed to cache
>>>> multi-site in one configuration. Sometimes Varnish switched to use the
>>>> first backend in the configuration for all the requests to the other sites.
>>>> As in the sample configuration below, the request to backend2, backend3,
>>>> backend4 sometimes forwarded to backend1. Does anyone have the same problem
>>>> or know any solution, suggestion?
>>>>
>>>
>>> Could you see if you can post a varnishlog statement showing this
>>> behaviour? It should be fairly easy to find one if it happens regularly.
>>>
>>>
>>> Cheers,
>>>
>>> Per.
>>>
>>> --
>>>  <http://www.varnish-software.com/> *Per Buer*
>>> CTO | Varnish Software AS
>>> Phone: +47 958 39 117 | Skype: per.buer
>>> We Make Websites Fly!
>>>
>>> Winner of the Red Herring Top 100 Europe Award 2013
>>>
>>>
>>>
>>
>
>
> --
>  <http://www.varnish-software.com/> *Per Buer*
> CTO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130816/bfab1e23/attachment.html>

From Travis.Crowder at penton.com  Fri Aug 16 18:10:34 2013
From: Travis.Crowder at penton.com (Crowder, Travis)
Date: Fri, 16 Aug 2013 18:10:34 +0000
Subject: High read IOPS with file backend
In-Reply-To: <CADAfL_hV2-PyTfaQkX6n8ENMGM+knLYSEaR8d_f-+Pf=C+qB5g@mail.gmail.com>
Message-ID: <CE33D4EA.1A51E%travis.crowder@penton.com>

Can you publish your testing methods and configurations for both Varnish
and Nginx?  Can you include the Network graphs and/or the Connections
graphs for Varnish and Nginx?

-Travis

On 8/16/13 12:29 PM, "Bryan Stillwell" <bstillwell at photobucket.com> wrote:

>I'm looking at converting all of our nginx caching servers over to
>varnish, but I'm seeing some oddities with the 'file' storage backend
>that I'm hoping someone could shed some light on.
>
>Hardware specs for both machines:
>
>Dell PowerEdge R610
>  Dual Xeon E5620 @ 2.40GHz
>  64GiB memory
>  4x 240GB SSDs in a RAID5 attached to a PERC H700 RAID controller
>
>The following image compares CPU, memory, and IOPS for both a machine
>running varnish (left), and one running nginx (right):
>
>http://i1217.photobucket.com/albums/dd391/bstillwell_pb/Graphs/varnish-vs-
>nginx.png
>
>At the most recent sample, each machine was handling ~350 requests/sec.
>
>As you can see the varnish machine has a lot more CPU time dedicated to
>I/O wait, which matches up with ~5x higher IOPS numbers.  However, the
>biggest difference is that varnish is using ~25x more read IOPS than
>nginx.
>
>As for the jumps in the IOPS graph, I believe they can be explained by:
>
>Wednesday @ 10:00a: Started taking traffic
>Wednesday @ 11:00a: Memory cache filled, started using SSDs
>Wednesday @  4:00p: TTL of 6 hours was hit, objects start expiring
>Wednesday @  7:15p: SSD cache filled
>
>I pre-allocated the storage with fallocate (fallocate -l 450g
>/cache/varnish_storage.bin) to make sure that wasn't helping contribute
>to the issue.
>
>Any ideas on what could be tuned to reduce the number of read IOPS to be
>more inline with nginx?
>
>Thanks,
>Bryan
>
>_______________________________________________
>varnish-misc mailing list
>varnish-misc at varnish-cache.org
>https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>




From bstillwell at photobucket.com  Fri Aug 16 21:01:05 2013
From: bstillwell at photobucket.com (Bryan Stillwell)
Date: Fri, 16 Aug 2013 15:01:05 -0600
Subject: High read IOPS with file backend
In-Reply-To: <CE33D4EA.1A51E%travis.crowder@penton.com>
References: <CADAfL_hV2-PyTfaQkX6n8ENMGM+knLYSEaR8d_f-+Pf=C+qB5g@mail.gmail.com>
 <CE33D4EA.1A51E%travis.crowder@penton.com>
Message-ID: <CADAfL_gO0FFca2MJzY=LgMWDbKvf5JsC2vLHOOHt_djap-r+sg@mail.gmail.com>

Travis,

Here are a couple more graphs showing the requests/sec and network
traffic for both varnish and nginx:

http://i1217.photobucket.com/albums/dd391/bstillwell_pb/Graphs/varnish-vs-nginx2.png

As for testing method, that involved having both servers equally
weighted on an F5 load balancer taking production traffic.  The F5 was
configured to re-use connections when possible with one-connect, and
to use persistent hashing to always send the same requests to the same
caching server.

As for the configurations for both I'll send those to you in a private email.

Bryan

On Fri, Aug 16, 2013 at 12:10 PM, Crowder, Travis
<Travis.Crowder at penton.com> wrote:
> Can you publish your testing methods and configurations for both Varnish
> and Nginx?  Can you include the Network graphs and/or the Connections
> graphs for Varnish and Nginx?
>
> -Travis
>
> On 8/16/13 12:29 PM, "Bryan Stillwell" <bstillwell at photobucket.com> wrote:
>
>>I'm looking at converting all of our nginx caching servers over to
>>varnish, but I'm seeing some oddities with the 'file' storage backend
>>that I'm hoping someone could shed some light on.
>>
>>Hardware specs for both machines:
>>
>>Dell PowerEdge R610
>>  Dual Xeon E5620 @ 2.40GHz
>>  64GiB memory
>>  4x 240GB SSDs in a RAID5 attached to a PERC H700 RAID controller
>>
>>The following image compares CPU, memory, and IOPS for both a machine
>>running varnish (left), and one running nginx (right):
>>
>>http://i1217.photobucket.com/albums/dd391/bstillwell_pb/Graphs/varnish-vs-
>>nginx.png
>>
>>At the most recent sample, each machine was handling ~350 requests/sec.
>>
>>As you can see the varnish machine has a lot more CPU time dedicated to
>>I/O wait, which matches up with ~5x higher IOPS numbers.  However, the
>>biggest difference is that varnish is using ~25x more read IOPS than
>>nginx.
>>
>>As for the jumps in the IOPS graph, I believe they can be explained by:
>>
>>Wednesday @ 10:00a: Started taking traffic
>>Wednesday @ 11:00a: Memory cache filled, started using SSDs
>>Wednesday @  4:00p: TTL of 6 hours was hit, objects start expiring
>>Wednesday @  7:15p: SSD cache filled
>>
>>I pre-allocated the storage with fallocate (fallocate -l 450g
>>/cache/varnish_storage.bin) to make sure that wasn't helping contribute
>>to the issue.
>>
>>Any ideas on what could be tuned to reduce the number of read IOPS to be
>>more inline with nginx?
>>
>>Thanks,
>>Bryan
>>
>>_______________________________________________
>>varnish-misc mailing list
>>varnish-misc at varnish-cache.org
>>https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>



-- 


Bryan Stillwell
SENIOR SYSTEM ADMINISTRATOR

E: bstillwell at photobucket.com
O: 303.228.5109
M: 970.310.6085



From james at ifixit.com  Fri Aug 16 22:14:03 2013
From: james at ifixit.com (James Pearson)
Date: Fri, 16 Aug 2013 15:14:03 -0700
Subject: Varnish restart a few times a day
In-Reply-To: <C281C498-DEC4-406B-BF31-B5D001B4293F@babyonline.com.sg>
References: <C281C498-DEC4-406B-BF31-B5D001B4293F@babyonline.com.sg>
Message-ID: <1376691013-sup-86@geror.local>

Excerpts from Alex Soo's message of 2013-08-15 21:26:49 -0700:
> Hi,
> 
> I am new to varnish and has very limited knowledge in programing or UNIX
> 
> I am hosting my own web site on a Mac Mini server. It is all setup and install by freelance programer.
> 
> I have notice that varnish is restarting many times a day through log file.
> 
> Below is an example :
> 
> [snip]
> 
> Any advise where I should start looking at? Thanks in advance.

It'd probably be helpful to know the version of Varnish (varnishd -V), as well
as any other information about the installation - do you know how varnish was
installed (downloaded a binary, compiled it), how it's being run (through an
init-script, directly in a screen session, through a monitoring process like
Monit)?  How much traffic are you getting on the machine?  Is anything unusual
happening, like other processes crashing or the machine running out of
available memory?  Are the crash times consistent?  Do they correlate with
anything else?
 - P




From maillists0 at gmail.com  Sat Aug 17 00:24:04 2013
From: maillists0 at gmail.com (maillists0 at gmail.com)
Date: Fri, 16 Aug 2013 20:24:04 -0400
Subject: Interpreting sar output on a Varnish server.
Message-ID: <CAB+OxSASoPs+GTNLFQPCx4E8XYE=47GRk5d4GzLW4EyiAZDsrA@mail.gmail.com>

I have a fairly busy 2.1 box that stopped serving traffic a couple of times
recently. Since it's actively serving our sites, it ended up getting
bounced before we could look at Varnish to see what was going on. But
restarting Varnish did get things back to working.

In the process of trying to troubleshoot this, I came across something I
don't understand. If I run 'sar -n ALL', I get this for open sockets in the
period leading up to the crisis:


                            totsck    tcpsck    udpsck    rawsck
ip-frag    tcp-tw

    11:45:01 AM     14921     13657         0         0         0       118
    11:55:01 AM     23930     20664         0         0         0       152
    12:05:02 PM     32092     25969         0         0         0       113
    12:15:01 PM     39461     30043         0         0         0       113
    12:25:02 PM     46668     33715         0         0         0        94
    12:35:01 PM     54069     36689         0         0         0       107
    12:45:01 PM     61508     39127         0         0         0        90
    12:55:03 PM     68697     40981         0         0         0       101
    01:05:01 PM     75922     42634         0         0         0        93
    01:15:02 PM     82843     43848         0         0         0        98
    01:25:01 PM     89889     45174         0         0         0       103
    01:35:01 PM     97185     46296         0         0         0        94
    01:45:01 PM     100184     39404         0         0         0        85

Varnish became unresponsive near the 100,000 sockets point.

My question is this: if totsck is total sockets open, how is it that the
numbers in the other columns don't sum to equal totsck? What kind of socket
might be missing from this output? Does this even tell me anything useful
at all?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130816/887ab951/attachment.html>

From james at talkunafraid.co.uk  Mon Aug 19 09:47:37 2013
From: james at talkunafraid.co.uk (James Harrison)
Date: Mon, 19 Aug 2013 10:47:37 +0100
Subject: Varnish segfaulting after upgrade?
Message-ID: <5211E9B9.6090201@talkunafraid.co.uk>

Hi all,

I upgraded Varnish to 3.0.4 a couple of days ago, having not updated it
for a little while (I'm afraid I don't know what version I was running
before other than it was 3.0.x).

Every night since then it has segfaulted.

http://stats.talkunafraid.co.uk/derpibooru.org/cadance.derpibooru.org/varnish_objects-week.png

This is a standard Ubuntu package install from the normal repositories.
varnishd (varnish-3.0.4 revision 9f83e8f), package version: 3.0.4-1~precise

All I can find in the logs is the following:

Aug 17 15:20:38 cadance kernel: [1736198.007167] varnishd[10824]:
segfault at 7f9cf0d0245d ip 00007f9e807bba99 sp 00007f93f94e3190 error 4
in libvgz.so[7f9e807b4000+13000]

Any ideas?
-- 
Cheers,
James Harrison



From bstillwell at photobucket.com  Mon Aug 19 22:30:40 2013
From: bstillwell at photobucket.com (Bryan Stillwell)
Date: Mon, 19 Aug 2013 16:30:40 -0600
Subject: High read IOPS with file backend
In-Reply-To: <CADAfL_gO0FFca2MJzY=LgMWDbKvf5JsC2vLHOOHt_djap-r+sg@mail.gmail.com>
References: <CADAfL_hV2-PyTfaQkX6n8ENMGM+knLYSEaR8d_f-+Pf=C+qB5g@mail.gmail.com>
 <CE33D4EA.1A51E%travis.crowder@penton.com>
 <CADAfL_gO0FFca2MJzY=LgMWDbKvf5JsC2vLHOOHt_djap-r+sg@mail.gmail.com>
Message-ID: <CADAfL_ieNvrbWyiOVpi41NyBsZikSrTFzURSayPMqCO_SegOyg@mail.gmail.com>

I decided to give the persistent storage backend a try today to see if
it behaved any differently, but it still appears to be quite
experimental.  Within minutes I had varnishd restart itself a couple
times before I disabled it in the load balancer.

Here are the reasons reported in /var/log/messages:

Aug 19 16:08:47 v01 varnishd[30007]: Child (30008) not responding to
CLI, killing it.
Aug 19 16:08:47 v01 varnishd[30007]: Child (30008) died signal=6 (core dumped)
Aug 19 16:08:47 v01 varnishd[30007]: Child (30008) Panic message:
Assert error in HSH_Unbusy(), cache_hash.c line 645:#012
Condition(oc_getobj(sp->wrk, oc) == o) not true.#012thread =
(cache-worker)#012ident =
Linux,2.6.32-358.14.1.el6.x86_64,x86_64,-spersistent,-smalloc,-hcritbit,epoll#012Backtrace:#012
 0x42f728: /usr/sbin/varnishd() [0x42f728]#012  0x418175:
/usr/sbin/varnishd() [0x418175]#012  0x419195:
/usr/sbin/varnishd(CNT_Session+0x655) [0x419195]#012  0x431471:
/usr/sbin/varnishd() [0x431471]#012  0x3b89607851:
/lib64/libpthread.so.0() [0x3b89607851]#012  0x3b892e890d:
/lib64/libc.so.6(clone+0x6d) [0x3b892e890d]#012sp = 0x7ef14445a008
{#012  fd = 230, id = 230, xid = 1762400248,#012  client = XX.X.XX.XX
57031,#012  step = STP_FETCHBODY,#012  handling = deliver,#012
err_code = 200, err_reason = (null),#012  restarts = 0, esi_level =
0#012  flags =  is_gunzip#012  bodystatus = 4#012  ws = 0x7ef14445a080
{ #012    id = "sess",#012    {s,f,r,e} =
{0x7ef14445ac78,+3104,(nil),+65536},#012  },#012  http[req] = {#012
ws = 0x7ef14445a080[sess]#012      "GET",#012
"/XXXXXX/XXXX/XXXXXXXXXXXX/XXXXXXXXX.jpg",#012      "HTTP/1.1",#012
  "Cache-Control: max-age=0",#012      "Connection:
Persist,Keep-alive",#012      "Date: Mon, 19 Aug 2013 22:08:44
GMT",#012      "Via: 1.1 192.221.115.43:80 (Footprint 4.8/FPMCP)",#012
     "Accept: image/webp,*/*;q=0.8",#012      "Accept-Language:
tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4",#012      "If-Modified-Since:
Mon, 31 Mar 2008 19:38:05 GMT",#012      "If-None-Match:
"dd7-449c0ca64e140"",#012      "Referer:
http://XXXXX.XXX/XXXXXXXXXXXXXXXXXXXX",#012      "User-Agent:
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/28.0.1500.95 Safari/537.36",#012      "TE: chunked",#012
"x-cache-request: Level3",#012      "host: i123.photobucket.com",#012
    "X-Forwarded-For: XX.XXX.XXX.XXX, XX.X.XX.XX",#012
"Accept-Encoding: gzip",#012  },#012  worker = 0x7ef10872ea90 {#012
ws = 0x7ef10872ecc8 { #012      id = "wrk",#012      {s,f,r,e}
Aug 19 16:08:47 v01 varnishd[30007]: child (31550) Started
Aug 19 16:08:47 v01 varnishd[30007]: Child (31550) said Child starts
Aug 19 16:08:47 v01 varnishd[30007]: Child (31550) said Dropped 0
segments to make free_reserve
Aug 19 16:08:47 v01 varnishd[30007]: Child (31550) said Silo completely loaded


Aug 19 16:10:14 v01 varnishd[30007]: Child (31550) not responding to
CLI, killing it.
Aug 19 16:10:14 v01 varnishd[30007]: Child (31550) died signal=6 (core dumped)
Aug 19 16:10:14 v01 varnishd[30007]: Child (31550) Panic message:
Assert error in smp_oc_getobj(), storage_persistent_silo.c line
401:#012  Condition((o)->magic == 0x32851d42) not true.#012thread =
(cache-worker)#012ident =
Linux,2.6.32-358.14.1.el6.x86_64,x86_64,-spersistent,-smalloc,-hcritbit,epoll#012Backtrace:#012
 0x42f728: /usr/sbin/varnishd() [0x42f728]#012  0x44cc2f:
/usr/sbin/varnishd() [0x44cc2f]#012  0x44cf3b: /usr/sbin/varnishd()
[0x44cf3b]#012  0x41814e: /usr/sbin/varnishd() [0x41814e]#012
0x419195: /usr/sbin/varnishd(CNT_Session+0x655) [0x419195]#012
0x431471: /usr/sbin/varnishd() [0x431471]#012  0x3b89607851:
/lib64/libpthread.so.0() [0x3b89607851]#012  0x3b892e890d:
/lib64/libc.so.6(clone+0x6d) [0x3b892e890d]#012sp = 0x7ef110279008
{#012  fd = 301, id = 301, xid = 1780069529,#012  client = XX.X.XX.XX
56677,#012  step = STP_FETCHBODY,#012  handling = deliver,#012
err_code = 200, err_reason = (null),#012  restarts = 0, esi_level =
0#012  flags =  is_gunzip#012  bodystatus = 4#012  ws = 0x7ef110279080
{ #012    id = "sess",#012    {s,f,r,e} =
{0x7ef110279c78,+848,(nil),+65536},#012  },#012  http[req] = {#012
ws = 0x7ef110279080[sess]#012      "GET",#012
"/XXXXXX/XXXXX/XXXXXXXXXX/XXXXXXXXXXX/XXXXXXXXXXXXXXXX.jpg",#012
"HTTP/1.1",#012      "Cache-Control: max-age=31536000",#012
"Connection: Persist,Keep-alive",#012      "Date: Mon, 19 Aug 2013
22:10:10 GMT",#012      "Via: 1.1 8.12.204.11:80 (Footprint
4.8/FPMCP)",#012      "Accept: image/png,image/*;q=0.8,*/*;q=0.5",#012
     "Accept-Language: en-US,en;q=0.5",#012      "Referer:
http://XXX.XXXXXXX.XXX/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXX/",#012
     "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0)
Gecko/20100101 Firefox/23.0",#012      "TE: chunked",#012      "DNT:
1",#012      "x-cache-request: Level3",#012      "host:
i123.photobucket.com",#012      "X-Forwarded-For: XX.XXX.XXX.XXX,
XX.X.XX.XX",#012      "Accept-Encoding: gzip",#012  },#012  worker =
0x7ef0fee14a90 {#012    ws =
Aug 19 16:10:14 v01 varnishd[30007]: child (32678) Started
Aug 19 16:10:14 v01 varnishd[30007]: Child (32678) said Child starts
Aug 19 16:10:14 v01 varnishd[30007]: Child (32678) said Dropped 0
segments to make free_reserve
Aug 19 16:10:14 v01 varnishd[30007]: Child (32678) said Silo completely loaded


I would prefer to use the persistent backend, so if I can help
troubleshoot any problems with it let me know.

Bryan


On Fri, Aug 16, 2013 at 3:01 PM, Bryan Stillwell
<bstillwell at photobucket.com> wrote:
>
> Travis,
>
> Here are a couple more graphs showing the requests/sec and network
> traffic for both varnish and nginx:
>
> http://i1217.photobucket.com/albums/dd391/bstillwell_pb/Graphs/varnish-vs-nginx2.png
>
> As for testing method, that involved having both servers equally
> weighted on an F5 load balancer taking production traffic.  The F5 was
> configured to re-use connections when possible with one-connect, and
> to use persistent hashing to always send the same requests to the same
> caching server.
>
> As for the configurations for both I'll send those to you in a private email.
>
> Bryan
>
> On Fri, Aug 16, 2013 at 12:10 PM, Crowder, Travis
> <Travis.Crowder at penton.com> wrote:
> > Can you publish your testing methods and configurations for both Varnish
> > and Nginx?  Can you include the Network graphs and/or the Connections
> > graphs for Varnish and Nginx?
> >
> > -Travis
> >
> > On 8/16/13 12:29 PM, "Bryan Stillwell" <bstillwell at photobucket.com> wrote:
> >
> >>I'm looking at converting all of our nginx caching servers over to
> >>varnish, but I'm seeing some oddities with the 'file' storage backend
> >>that I'm hoping someone could shed some light on.
> >>
> >>Hardware specs for both machines:
> >>
> >>Dell PowerEdge R610
> >>  Dual Xeon E5620 @ 2.40GHz
> >>  64GiB memory
> >>  4x 240GB SSDs in a RAID5 attached to a PERC H700 RAID controller
> >>
> >>The following image compares CPU, memory, and IOPS for both a machine
> >>running varnish (left), and one running nginx (right):
> >>
> >>http://i1217.photobucket.com/albums/dd391/bstillwell_pb/Graphs/varnish-vs-
> >>nginx.png
> >>
> >>At the most recent sample, each machine was handling ~350 requests/sec.
> >>
> >>As you can see the varnish machine has a lot more CPU time dedicated to
> >>I/O wait, which matches up with ~5x higher IOPS numbers.  However, the
> >>biggest difference is that varnish is using ~25x more read IOPS than
> >>nginx.
> >>
> >>As for the jumps in the IOPS graph, I believe they can be explained by:
> >>
> >>Wednesday @ 10:00a: Started taking traffic
> >>Wednesday @ 11:00a: Memory cache filled, started using SSDs
> >>Wednesday @  4:00p: TTL of 6 hours was hit, objects start expiring
> >>Wednesday @  7:15p: SSD cache filled
> >>
> >>I pre-allocated the storage with fallocate (fallocate -l 450g
> >>/cache/varnish_storage.bin) to make sure that wasn't helping contribute
> >>to the issue.
> >>
> >>Any ideas on what could be tuned to reduce the number of read IOPS to be
> >>more inline with nginx?
> >>
> >>Thanks,
> >>Bryan
> >>
> >>_______________________________________________
> >>varnish-misc mailing list
> >>varnish-misc at varnish-cache.org
> >>https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From marco at nucleus.it  Tue Aug 20 07:50:42 2013
From: marco at nucleus.it (Marco Felettigh)
Date: Tue, 20 Aug 2013 09:50:42 +0200
Subject: varnishstat and memory  usage
In-Reply-To: <20130807130052.0f5662e3@lobo.lobo.dom>
References: <20130807130052.0f5662e3@lobo.lobo.dom>
Message-ID: <20130820095042.346978a3@lobo.lobo.dom>

no one ? :(


On Wed, 7 Aug 2013 13:00:52 +0200
Marco Felettigh <marco at nucleus.it> wrote:

> Hi to all,
> can someone explain please what this values mean ?
> 
> varnish-3.0.2
> 
> from varnishstat -1 :
> 
> SMA.s0.c_bytes   4421961276     14330.40 Bytes allocated
> Bytes allocated from Varnish for all the things;
> cache , thread etc . Right ?
> 
> SMA.s0.c_freed   3336892882     10813.98 Bytes freed
> freed ? that means that in the uptime of my Varnish instance it freed
> 3,3GB of bytes from the cache ?
> So this counter can go beyond Bytes allocated ?
> 
> SMA.s0.g_bytes   1085068394          .   Bytes outstanding
> These are the bytes used by cache ?
> 
> SMA.s0.g_space   2136157078          .   Bytes available
> Ok this is self explained. I hope :)
> 
> Thanks and have a nice day
> 
> Marco



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130820/fb36bd1e/attachment.bin>

From maillists0 at gmail.com  Wed Aug 21 16:07:57 2013
From: maillists0 at gmail.com (maillists0 at gmail.com)
Date: Wed, 21 Aug 2013 12:07:57 -0400
Subject: ICP Emulation
Message-ID: <CAB+OxSBi=y_wdcirfue0Q988P7jQrTwAEOksCFhZoDJQGK5GUg@mail.gmail.com>

An older blog post on the Varnish site says this:

"... you can emulate ICP-like behaviour by writing VCL code to check your
neighouring cache for the request that you are looking for. Setups like
these have been discussed on
varnish-misc<http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc/>several
times and implementing it should be fairly simple."

I must be searching the wrong terms, because I can't find any descriptions
of this.

It looks to me like the setup would be little more than setting the peer as
a backend and then setting req.hash_ignore_busy on one of the servers to
avoid race conditions, as described on the wiki. Is there anything I'm
missing, or does someone have a reference on how to set this up? Any help
is appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130821/d4292a5e/attachment.html>

From russ at eatnumber1.com  Thu Aug 22 19:09:03 2013
From: russ at eatnumber1.com (Russell Harmon)
Date: Thu, 22 Aug 2013 12:09:03 -0700
Subject: Varnishlog Truncating Long Lines
Message-ID: <CA+zrezSq+w1n2rw4M4=W3iMswjC8z+61oxa7imb1yFSSDxYnxg@mail.gmail.com>

Is it possible to get varnishlog to not truncate long lines?
--
Russell Harmon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130822/66fc36f3/attachment.html>

From smwood4 at gmail.com  Thu Aug 22 19:24:24 2013
From: smwood4 at gmail.com (Stephen Wood)
Date: Thu, 22 Aug 2013 12:24:24 -0700
Subject: Varnishlog Truncating Long Lines
In-Reply-To: <CA+zrezSq+w1n2rw4M4=W3iMswjC8z+61oxa7imb1yFSSDxYnxg@mail.gmail.com>
References: <CA+zrezSq+w1n2rw4M4=W3iMswjC8z+61oxa7imb1yFSSDxYnxg@mail.gmail.com>
Message-ID: <CA+pzux_BaCG36MuYirani3+xEKJaxebZsVjTNevDH4xdtgs2Hw@mail.gmail.com>

You can adjust this by updating the shm_reclen parameter. The default is
255 bytes.




On Thu, Aug 22, 2013 at 12:09 PM, Russell Harmon <russ at eatnumber1.com>wrote:

> Is it possible to get varnishlog to not truncate long lines?
> --
> Russell Harmon
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
Stephen Wood
Dev/Ops Engineer
Moz, Inc.
Website: www.heystephenwood.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130822/7b684bdd/attachment.html>

From russ at eatnumber1.com  Thu Aug 22 20:23:50 2013
From: russ at eatnumber1.com (Russell Harmon)
Date: Thu, 22 Aug 2013 13:23:50 -0700
Subject: Varnishlog Truncating Long Lines
In-Reply-To: <CA+pzux_BaCG36MuYirani3+xEKJaxebZsVjTNevDH4xdtgs2Hw@mail.gmail.com>
References: <CA+zrezSq+w1n2rw4M4=W3iMswjC8z+61oxa7imb1yFSSDxYnxg@mail.gmail.com>
 <CA+pzux_BaCG36MuYirani3+xEKJaxebZsVjTNevDH4xdtgs2Hw@mail.gmail.com>
Message-ID: <CA+zrezS_CYpD2BQjyr-p9Rm-2gUYad9X5BRruUw9Ff92UCMWxg@mail.gmail.com>

That's exactly what I was looking for. Thanks :)

--
Russell Harmon


On Thu, Aug 22, 2013 at 12:24 PM, Stephen Wood <smwood4 at gmail.com> wrote:

> You can adjust this by updating the shm_reclen parameter. The default is
> 255 bytes.
>
>
>
>
> On Thu, Aug 22, 2013 at 12:09 PM, Russell Harmon <russ at eatnumber1.com>wrote:
>
>> Is it possible to get varnishlog to not truncate long lines?
>> --
>> Russell Harmon
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
>
>
> --
> Stephen Wood
> Dev/Ops Engineer
> Moz, Inc.
> Website: www.heystephenwood.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130822/60d9feab/attachment.html>

From yabuki at sraoss.co.jp  Mon Aug 26 05:31:16 2013
From: yabuki at sraoss.co.jp (YABUKI Youichi)
Date: Mon, 26 Aug 2013 14:31:16 +0900
Subject: varnish 2.1.5 assert error in http_SetH()
Message-ID: <201308260531.OAA30565@osspc4.sra.co.jp>

[I have reported a bug such as following to the varnish-bug list August 13,
but I cannot get any responses yet. ]

My customers varnish 2.1.5 aborted with Assert error in http_SetH().
The related logs in /var/log/messages are as follow.

This problem seems like #726.
https://www.varnish-cache.org/lists/pipermail/varnish-bugs/2010-June/002888.html
The page says this is fixed in trunk, but I cannot find the fix for 2.1.5.
Is my fix bellow OK?

/var/log/messages:
-----------------------------------------------------------------------------
Jul 25 06:12:22 XXXXXXXXXX varnishd[4883]: Child (4898) died signal=6
Jul 25 06:12:22 XXXXXXXXXX varnishd[4883]: Child (4898) Panic message: Assert error in http_SetH(), cache_http.c line 656:   Condition((fm) != 0) not true. thread = (cache-worker) ident = Linux,2.6.18-194.el5,x86_64,-sfile,-hclassic,epoll Backtrace:   0x42bad9: pan_backtrace+16   0x42bd42: pan_ic+164   0x427006: http_SetH+81   0x4352ac: VRT_l_obj_status+d8   0x2acd5b411ce5: _end+2acd5ad8547d   0x2acd5b414cac: _end+2acd5ad88444   0x4328cf: VCL_error_method+6a   0x4150eb: cnt_error+2fa   0x418558: CNT_Session+5e1   0x42d7e9: wrk_do_cnt_sess+12a sp = 0x2acd57f02008 {   fd = 20, id = 20, xid = 4240864234,   client = XX.XX.XX.XX 46773,   step = STP_ERROR,   handling = deliver,   err_code = 854, err_reason = redirection,   restarts = 0, esis = 0   ws = 0x2acd57f02080 {      id = "sess",     {s,f,r,e} = {0x2acd57f02cd8,+1464,(nil),+131072},   },   http[req] = {     ws = 0x2acd57f02080[sess]       "GET",       "/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
 X!
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Jul 25 06:12:22 XXXXXXXXXX varnishd[4883]: Child cleanup complete
Jul 25 06:12:22 XXXXXXXXXX varnishd[4883]: child (2652) Started
Jul 25 06:12:22 XXXXXXXXXX varnishd[4883]: Child (2652) said 
Jul 25 06:12:22 XXXXXXXXXX varnishd[4883]: Child (2652) said Child starts
Jul 25 06:12:22 XXXXXXXXXX varnishd[4883]: Child (2652) said managed to mmap 107374182400 bytes of 107374182400
-----------------------------------------------------------------------------

patch:
-----------------------------------------------------------------------------
--- cache_vrt.c.org	2013-08-07 13:00:12.000000000 +0900
+++ cache_vrt.c	2013-08-08 10:36:24.000000000 +0900
@@ -275,12 +275,13 @@ VRT_l_obj_status(const struct sess *sp, 
 
 	assert(num >= 100 && num <= 999);
 	p = WS_Alloc(sp->obj->http->ws, 4);
-	if (p == NULL)
+	if (p == NULL) {
 		WSP(sp, SLT_LostHeader, "%s", "obj.status");
-	else
+	} else {
 		sprintf(p, "%d", num);
-	http_SetH(sp->obj->http, HTTP_HDR_STATUS, p);
-	sp->obj->http->status = num;
+		http_SetH(sp->obj->http, HTTP_HDR_STATUS, p);
+		sp->obj->http->status = num;
+	}
 }
 
 /* Add an objecthead to the saintmode list for the (hopefully) relevant
@@ -356,12 +357,13 @@ VRT_l_resp_status(const struct sess *sp,
 
 	assert(num >= 100 && num <= 999);
 	p = WS_Alloc(sp->wrk->ws, 4);
-	if (p == NULL)
+	if (p == NULL) {
 		WSP(sp, SLT_LostHeader, "%s", "resp.status");
-	else
+	} else {
 		sprintf(p, "%d", num);
-	http_SetH(sp->wrk->resp, HTTP_HDR_STATUS, p);
-	sp->wrk->resp->status = num;
+		http_SetH(sp->wrk->resp, HTTP_HDR_STATUS, p);
+		sp->wrk->resp->status = num;
+	}
 }
 
 int
@@ -461,12 +463,13 @@ VRT_l_beresp_status(const struct sess *s
 
 	assert(num >= 100 && num <= 999);
 	p = WS_Alloc(sp->wrk->beresp->ws, 4);
-	if (p == NULL)
+	if (p == NULL) {
 		WSP(sp, SLT_LostHeader, "%s", "obj.status");
-	else
+	} else {
 		sprintf(p, "%d", num);
-	http_SetH(sp->wrk->beresp, HTTP_HDR_STATUS, p);
-	sp->wrk->beresp->status = num;
+		http_SetH(sp->wrk->beresp, HTTP_HDR_STATUS, p);
+		sp->wrk->beresp->status = num;
+	}
 }
 
 int


-----------------------------------------------------------------------------
yabuki at sraoss.co.jp
SRA OSS,Inc.Japan



From ricardo at tuenti.com  Mon Aug 26 09:10:10 2013
From: ricardo at tuenti.com (Ricardo Bartolome Mendez)
Date: Mon, 26 Aug 2013 11:10:10 +0200
Subject: Varnish 3.x Expiry Thread overload
Message-ID: <CANm1ebKmUJiQTshf=HHBujETenzZQ5Hp1xpK77auJe22Ch-jCQ@mail.gmail.com>

Hi all,

I decided to share this issue with you because I was unable to confirm it's
a bug, a performance degradation over the time or a miss-configuration on
my side. So, here's the problem:


We have an scenario where we have several varnish servers that suffer this
issue at some point, always once the mmap file storage is filled up, but
not immediately after it gets filled up (in fact, weeks afterwards).

We have ~1100GB of SSD disk storage and ~64GB of RAM per node. Our content
is what industry defines it as "cold content" or "long tail". We have
objects which size is 1K-5K and others that is 50K. Our Varnish startup
configuration is [1]. Our VCL config is very simple: some Host and
Content-Encoding header homogenization and some content switching rules in
order to route the request to different backends.

Time ago (varnish 3.0.3) we suffered "Can't get storage" errors and we
decided to increase the nuke_limit to 500 (when default was 10, and
reverted to 50 by mainstream due to a unexpected behavior [5]) because we
have planned to store objects bigger than 60K and hence a 500K object may
need to evict from cache up to 500 1K objects in order to get space.
Probably this was not the smartest decision and we should deploy different
varnish instances for different type of objects.

Now we are facing a new issue, and we're quite lost about the root cause.
The timeline of the problem is the following:

   - Node is added to production and start storing data in the file storage
   (July 14th)
   - There is a point that stored objects are bigger than the active ram,
   so IOPS over the disk increase. All working as expected.
   - mmap storage gets filled up, so evictions start happening (July 18h)
      - lru_nuked_objects is 200 per sec at peak
      - cache insertions is the same rate, so apparently 1 object is
      inserted and 1 object is added. (Am I understanding incorrectly the
      nuke_limit parameter and/or lru_nuked_objects item?)
   - We detect the issue on August 8th, so we removed the affected node
   from production (10:53AM), and we have seen the following behavior:
      - Varnish keeps consuming a single core (I guess Expiry Thread, which
      is a single thread) until 11:16AM
      - Cache insertions stopped at 10:53AM, but lru_nuked_objects have
      been happening until 11:16AM

As soon as the node is out of production, varnishlog reports no client
request but prints LRU ExpKill messages [2] until 11:16AM. After this only
Backend_health messages and requests issued by monitoring system are seen.
At this point the varnishstat items "SMF.s0.c_freed" and "SMF.s0.c_fail"
increase, even "SMF.s0.c_bytes" only increments from time to time.

If we add the node back to production without restarting the process it
runs fine only for ~15 seconds, and then it happens again.  We kept this
node for experiments because it's totally reproducible.

Other remarkable fact is that some request appear to be taking tens of
hundreds of seconds [3]. Our timeouts are now the default, but apparently
threads get stuck for a while until they get space for storing the object
in cache, but when thread successfully stores the object it's too late and
client already closed connection due to timeout. We know we have thread
pile-up issues because file descriptors increase by 4x-5x until reaching 5k.

My theory is that Expiry thread is being overloaded by an incorrect
nuke_limit parameter (it tries to evict too many objects), so cache does
not freed space fast enough and threads are blocked waiting for storage. We
can't protect against it (fail fast approach) because there is no
configurable timeout for writing on storage [4]. Obviously, this is Varnish
 internals.

As the time goes on, we have seen this behavior happening again (indeed,
twice) in 4 different nodes that already suffered this issue in the past.
Fortunately we are maintaining a spreadsheet with the odd behaviors and
they happened 23, 25 and 30 days after process was started. It's just a
curious coincidence, right?

Some facts that may be useful:
- If we restart the process, problem is solved.
- We are using "critbit" hashing algorithm instead of "classic". In fact,
we haven't tried any other because we have seen it's the 3.x default
hashing and we assumed it's stable enough and has no weird bugs.
- We haven't tried branch 2.x because we are running Debian Wheezy and 3.x
was the version it provides. We upgraded to 3.0.4 from Varnish Debian
packages in order to make sure it was not a bug solved on 3.0.4.
- Our traffic per node is not really heavy (200 mbps). Indeed, the only
challenge is the big catalog of objects from different sizes we have.

Does anybody have an idea about what's going on?

Regards,





[1]
/usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -T localhost:6082 -f
/etc/varnish/default.vcl -S /etc/varnish/secret -s
file,/srv/cache/varnish_cache,1100G -p nuke_limit 500 -u varnish -g varnish -w
500,8000,30

[2]
0 ExpKill      - 2385044933 LRU

[3]
 2614 ReqEnd       - 2627025831 1375951760.351629972 1375952903.284181118
0.000064850 1142.932451248 0.000099897
 2167 ReqEnd       - 2627022893 1375951754.715034962 1375952903.284289598
0.000098705 1148.569141150 0.000113487
 1891 ReqEnd       - 2627024599 1375951758.109493732 1375952903.284304857
0.000055552 1145.174724579 0.000086546
 1469 ReqEnd       - 2627040233 1375951785.068578243 1375952903.284320831
0.000058889 1118.215640306 0.000102282
 2155 ReqEnd       - 2627022876 1375951754.677411079 1375952903.284334898
0.000061274 1148.606820107 0.000103712

[4]
https://www.varnish-software.com/static/book/Tuning.html

[5]
https://www.varnish-cache.org/trac/ticket/1012

-- 
Ricardo Bartolom? M?ndez
Systems Engineer
+34.672.194.729
tuenti.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130826/896d9c77/attachment.html>

From lkarsten at varnish-software.com  Mon Aug 26 10:17:44 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 26 Aug 2013 12:17:44 +0200
Subject: varnishstat and memory  usage
In-Reply-To: <20130807130052.0f5662e3@lobo.lobo.dom>
References: <20130807130052.0f5662e3@lobo.lobo.dom>
Message-ID: <20130826101744.GA30573@immer.varnish-software.com>

On Wed, Aug 07, 2013 at 01:00:52PM +0200, Marco Felettigh wrote:
> can someone explain please what this values mean ?

g means gauge, c means counter.

s0 means that this is the first configured storage backend. (for example one -smalloc,XX argument)

> varnish-3.0.2
> from varnishstat -1 :
> SMA.s0.c_bytes   4421961276     14330.40 Bytes allocated
> Bytes allocated from Varnish for all the things;
> cache , thread etc . Right ?
> SMA.s0.c_freed   3336892882     10813.98 Bytes freed
> freed ? that means that in the uptime of my Varnish instance it freed
> 3,3GB of bytes from the cache ?

Yes. Usually you don't need to care about these.

> So this counter can go beyond Bytes allocated ?
> SMA.s0.g_bytes   1085068394          .   Bytes outstanding
> These are the bytes used by cache ?

Yes.

If you want to graph storage use, have a look at the munin plugin for Varnish.

-- 
With regards,
Lasse Karstensen
Varnish Software AS



From Peter.Jacobs at kuleuven.be  Mon Aug 26 13:16:12 2013
From: Peter.Jacobs at kuleuven.be (Peter Jacobs)
Date: Mon, 26 Aug 2013 13:16:12 +0000
Subject: how to configure varnish to drop a connection instead of sending 503?
Message-ID: <851DE1A04402094ABD2EB2C47C31C2AD10A6294E@ICTS-S-MBX3.luna.kuleuven.be>

We have a machine with apache webserver and the apache loadbalancing module, in front of two backend machines running the same web application.
The apache loadbalancing module has very basic functionality and only drops a backend service if it doesn't respond within a certain time. This used to work fine:  if the backend service on one machine broke down, all connections would switch to the other machine

I have put Varnish on the two backend machines (it cannot be installed on the machine with apache). This helps with performance, but the loadbalancing part doesn't work anymore: if a backend service doesn't respond, varnish will return a "503 backend service not available" . The apache loadbalancer will happily accept it as a valid response and keep serving it to the client, instead of taking it out of the loadbalancer.

Is there a way to make Varnish just drop the connection if the backend is down, instead of sending a 503? This will help us until we upgrade to a better loadbalancer that can handle status codes.

Kind regards,
Peter Jacobs




From lkarsten at varnish-software.com  Mon Aug 26 13:38:41 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 26 Aug 2013 15:38:41 +0200
Subject: Varnish 3.x Expiry Thread overload
In-Reply-To: <CANm1ebKmUJiQTshf=HHBujETenzZQ5Hp1xpK77auJe22Ch-jCQ@mail.gmail.com>
References: <CANm1ebKmUJiQTshf=HHBujETenzZQ5Hp1xpK77auJe22Ch-jCQ@mail.gmail.com>
Message-ID: <20130826133840.GB30573@immer.varnish-software.com>

On Mon, Aug 26, 2013 at 11:10:10AM +0200, Ricardo Bartolome Mendez wrote:
[..]
> Does anybody have an idea about what's going on?

Hi Ricardo.

I ran this by Martin and Dag.

Just to make sure, are there a lot of bans in use? We're assuming no.

Our best guess based on the data supplied is that the tiny object handling in
the file backend is the source of this. Many small objects mixed with
larger objects makes a bad combination on -sfile.

We've seen a problem that did appear similar on a customer rig not long ago.

Our best suggestion is to reconfigure the system for swap and use -smalloc
instead, and see how that goes. This is contrary to the usual recommendations.

If you can I'd recommend adding some disk IO latency monitoring.
 
-- 
With regards,
Lasse Karstensen
Varnish Software AS



From lkarsten at varnish-software.com  Mon Aug 26 14:22:11 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 26 Aug 2013 16:22:11 +0200
Subject: how to configure varnish to drop a connection instead of sending
 503?
In-Reply-To: <851DE1A04402094ABD2EB2C47C31C2AD10A6294E@ICTS-S-MBX3.luna.kuleuven.be>
References: <851DE1A04402094ABD2EB2C47C31C2AD10A6294E@ICTS-S-MBX3.luna.kuleuven.be>
Message-ID: <20130826142210.GC30573@immer.varnish-software.com>

On Mon, Aug 26, 2013 at 01:16:12PM +0000, Peter Jacobs wrote:
[..]
> Is there a way to make Varnish just drop the connection if the backend is down, instead of sending a 503? This will help us until we upgrade to a better loadbalancer that can handle status codes.

Yes, you can use the shield vmod for this:

    https://github.com/varnish/libvmod-shield


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From ricardo at tuenti.com  Mon Aug 26 14:27:34 2013
From: ricardo at tuenti.com (Ricardo Bartolome Mendez)
Date: Mon, 26 Aug 2013 16:27:34 +0200
Subject: Varnish 3.x Expiry Thread overload
In-Reply-To: <20130826133840.GB30573@immer.varnish-software.com>
References: <CANm1ebKmUJiQTshf=HHBujETenzZQ5Hp1xpK77auJe22Ch-jCQ@mail.gmail.com>
 <20130826133840.GB30573@immer.varnish-software.com>
Message-ID: <CANm1ebJyU7_kLuRPXJP8d20gyQVh64dTyV=DYEX3rf7cvJk7OA@mail.gmail.com>

Hi Lasse,

There are no bans in use. I think we can manage to stop sending small
objects to a subset of servers and see how it behaves, even unfortunately
we'll need to wait few weeks until we check if it helps or not. We'll also
give a try to -smalloc to see how it behaves.

Shall I understand that you are actively looking into the issue because a
customer of yours is being affected or shall I open a ticket in Varnish
Trac? Is it a known issue? Is there already an existing ticket? I'm willing
to help, so don't hesitate to ask me any kind of information that can help
you to spot the issue.

Regards,


2013/8/26 Lasse Karstensen <lkarsten at varnish-software.com>

> On Mon, Aug 26, 2013 at 11:10:10AM +0200, Ricardo Bartolome Mendez wrote:
> [..]
> > Does anybody have an idea about what's going on?
>
> Hi Ricardo.
>
> I ran this by Martin and Dag.
>
> Just to make sure, are there a lot of bans in use? We're assuming no.
>
> Our best guess based on the data supplied is that the tiny object handling
> in
> the file backend is the source of this. Many small objects mixed with
> larger objects makes a bad combination on -sfile.
>
> We've seen a problem that did appear similar on a customer rig not long
> ago.
>
> Our best suggestion is to reconfigure the system for swap and use -smalloc
> instead, and see how that goes. This is contrary to the usual
> recommendations.
>
> If you can I'd recommend adding some disk IO latency monitoring.
>
> --
> With regards,
> Lasse Karstensen
> Varnish Software AS
>



-- 
Ricardo Bartolom? M?ndez
Systems Engineer
+34.672.194.729
tuenti.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130826/c3094f55/attachment.html>

From Paul.McInerney at faredge.com.au  Tue Aug 27 06:50:15 2013
From: Paul.McInerney at faredge.com.au (Paul McInerney)
Date: Tue, 27 Aug 2013 16:50:15 +1000
Subject: varnish and cookie being set by backend
Message-ID: <CCD0821AE781994EB8BAB510676B5EF5016A9C685D38@ZEUS.faredge.local>

Hi all,

I have another problem I am trying to solve.

One of the sites that sits behind a varnish service, uses an on_page_load module that sends a pop-up asking for an email address (for mail list subscription). It needs to then set a cookie that expires in 30 days (so visitors aren't constantly hit with this pop-up)

So, in vcl_fetch, I have this:


        if (!beresp.http.set-cookie ~ "modaldone"){
                unset beresp.http.set-Cookie;
        }

Where 'modaldone' is the name of the cookie being set by the backend - yet this still doesn't get set.

Here's the complete dump of default.vcl   http://pastebin.com/jzNJbs49

Have I missed something obvious?

Many thanks.

Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130827/d5388842/attachment.html>

From lkarsten at varnish-software.com  Tue Aug 27 09:15:49 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 27 Aug 2013 11:15:49 +0200
Subject: varnish and cookie being set by backend
In-Reply-To: <CCD0821AE781994EB8BAB510676B5EF5016A9C685D38@ZEUS.faredge.local>
References: <CCD0821AE781994EB8BAB510676B5EF5016A9C685D38@ZEUS.faredge.local>
Message-ID: <20130827091547.GD30573@immer.varnish-software.com>

On Tue, Aug 27, 2013 at 04:50:15PM +1000, Paul McInerney wrote:
> I have another problem I am trying to solve.
> One of the sites that sits behind a varnish service, uses an on_page_load module that sends a pop-up asking for an email address (for mail list subscription). It needs to then set a cookie that expires in 30 days (so visitors aren't constantly hit with this pop-up)

I suggest that you rething the problem and use javascript to set the cookie.
Solves everything, except that you need to filter the incoming cookie which is pretty simple.


> So, in vcl_fetch, I have this:
>         if (!beresp.http.set-cookie ~ "modaldone"){
>                 unset beresp.http.set-Cookie;
>         }
> Where 'modaldone' is the name of the cookie being set by the backend - yet this still doesn't get set.

Varnishlog will tell you.

You might want to add some printf-style debugging to understand the flow:

    import std;
    sub vcl_fetch {
	if (beresp.http.set-cookie !~ "modaldone") {
	    std.log("removing set-cookie: " + beresp.http.set-cookie);
	    unset beresp.http.set-cookie;
	}
	[..]
    }

The std.log lines will show up in varnishlog.


> Here's the complete dump of default.vcl   http://pastebin.com/jzNJbs49
> Have I missed something obvious?

I'd recommend rewriting the VCL to not use return() statements. Returning
hit_for_pass without setting the TTL can lead to request serialisation
if the object is uncacheable.

Setting req.backend in vcl_fetch is really a bit to late. We already have the
response from the backend at that point. (unless you plan to restart, with you
don't seem to do)



-- 
With regards,
Lasse Karstensen
Varnish Software AS



From lkarsten at varnish-software.com  Tue Aug 27 09:28:22 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 27 Aug 2013 11:28:22 +0200
Subject: Varnish 3.x Expiry Thread overload
In-Reply-To: <CANm1ebJyU7_kLuRPXJP8d20gyQVh64dTyV=DYEX3rf7cvJk7OA@mail.gmail.com>
References: <CANm1ebKmUJiQTshf=HHBujETenzZQ5Hp1xpK77auJe22Ch-jCQ@mail.gmail.com>
 <20130826133840.GB30573@immer.varnish-software.com>
 <CANm1ebJyU7_kLuRPXJP8d20gyQVh64dTyV=DYEX3rf7cvJk7OA@mail.gmail.com>
Message-ID: <20130827092822.GE30573@immer.varnish-software.com>

On Mon, Aug 26, 2013 at 04:27:34PM +0200, Ricardo Bartolome Mendez wrote:
> There are no bans in use. I think we can manage to stop sending small
> objects to a subset of servers and see how it behaves, even unfortunately
> we'll need to wait few weeks until we check if it helps or not. We'll also
> give a try to -smalloc to see how it behaves.

Great. Please report back if the malloc backend works as expected.


> Shall I understand that you are actively looking into the issue because a
> customer of yours is being affected or shall I open a ticket in Varnish
> Trac? Is it a known issue? Is there already an existing ticket? I'm willing
> to help, so don't hesitate to ask me any kind of information that can help
> you to spot the issue.

The customer got things working with malloc and was happy afterwards.

If our gut feelign is right, this is a situation where the assumptions about
the working set made during implementation is not entirely according to how it
is being used.

I don't believe there is a trac ticket for it. Tracking down these things are
pretty resource intensive. Create one if you have time to follow through on
in, but I'd expect that the 4.0 release gets priority the next few months.

If you need this fixed in a shorter time scale, there is commercial support
available. Ping me off-list if this is of interest.


-- 
With regards,
Lasse Karstensen
Varnish Software AS



From james at ifixit.com  Tue Aug 27 22:49:33 2013
From: james at ifixit.com (James Pearson)
Date: Tue, 27 Aug 2013 15:49:33 -0700
Subject: how to configure varnish to drop a connection instead of sending
 503?
In-Reply-To: <851DE1A04402094ABD2EB2C47C31C2AD10A6294E@ICTS-S-MBX3.luna.kuleuven.be>
References: <851DE1A04402094ABD2EB2C47C31C2AD10A6294E@ICTS-S-MBX3.luna.kuleuven.be>
Message-ID: <1377643516-sup-8208@geror.local>

Excerpts from Peter Jacobs's message of 2013-08-26 06:16:12 -0700:
> We have a machine with apache webserver and the apache loadbalancing module, in front of two backend machines running the same web application.
> The apache loadbalancing module has very basic functionality and only drops a backend service if it doesn't respond within a certain time. This used to work fine:  if the backend service on one machine broke down, all connections would switch to the other machine
> 
> I have put Varnish on the two backend machines (it cannot be installed on the machine with apache).

Why can't you install Varnish on your load balancer?

Have you considered replacing Apache with a dedicated load balancer, like
HAProxy?

libvmod-shield may do what you want, but this seems like an XY problem - the
proper solution is to change your architecture.

Of course, you may have a number of very good technical or political reasons
why you can't do that (I'm sure we've all been in those situations), but I feel
obliged to make sure that's actually the case, and we aren't just helping you
down the wrong path.
 - P




From Peter.Jacobs at kuleuven.be  Wed Aug 28 10:46:24 2013
From: Peter.Jacobs at kuleuven.be (Peter Jacobs)
Date: Wed, 28 Aug 2013 10:46:24 +0000
Subject: how to configure varnish to drop a connection instead of
 sending 503?
Message-ID: <851DE1A04402094ABD2EB2C47C31C2AD10A6FC65@ICTS-S-MBX5.luna.kuleuven.be>

Excerpts from James Pearson's message :
> Why can't you install Varnish on your load balancer?
These load balancers run other unrelated critical services, and Varnish cache is still new for us, so we find it risky to install and maintain it on those machines for now. 

> Have you considered replacing Apache with a dedicated load balancer, like HAProxy?
We will definitely replace it with a dedicated load balancer with more functionality. But that takes some more testing. 
It would be nice to have a fix now and I was hoping that I missed a simple command to drop a connection.

> libvmod-shield may do what you want, but this seems like an XY problem - the proper solution is to change your architecture.
Absolutely true! As I told in the original mail, that is our plan but that will take some time...

Thanks for the critical feedback :-)

And thanks to Lasse Karstensen for pointing me to this vmod, it may be the solution after some code change (at the moment it only works in vcl_recv, for us it should be activated in vcl_fetch after getting a timeout from the backend.)

Peter




From tharanga.abeyseela at gmail.com  Thu Aug 29 10:57:47 2013
From: tharanga.abeyseela at gmail.com (Tharanga Abeyseela)
Date: Thu, 29 Aug 2013 20:57:47 +1000
Subject: Varnish round robin using Director
Message-ID: <CAOMTrfcK09itQjH42yBZbG6=vTXz143EN1-auBrtxgJu49jAww@mail.gmail.com>

Hi Guys,

I'm trying to load-balance requests between two servers. i have the
following config.

backend website {
    .host = "xxx.com";
    .port = "8080";
   .probe = {
    .url = "/favicon.ico";
    .timeout = 60ms;
    .interval = 60s;
    .window = 5;
    .threshold = 3;
}
}
backend api {
    .host = "yyy.com";
    .port = "8080";
   .probe = {
    .url = "/favicon.ico";
    .timeout = 60ms;
    .interval = 60s;
    .window = 5;
    .threshold = 3;
}
}

sub vcl_recv {
if ( req.url ~ "/api/") {
set req.backend = api;
set req.http.host = "yyyy.com";
} else {
set req.backend = website;
}
}

director clust round-robin {
{ .backend = website; }
{ .backend = api; }
}

but this is giving the following error.

Unused backend clust, defined:
('input' Line 39 Pos 10)
director clust round-robin {
---------#####--------------

then i was tyring to add set req.backend = clust (then it is giving the
error,

Symbol not found: 'clust' (expected type BACKEND):

am i missing something here.

Cheers,
Tharanga
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130829/252e0ff8/attachment.html>

From tharanga.abeyseela at gmail.com  Thu Aug 29 11:25:42 2013
From: tharanga.abeyseela at gmail.com (Tharanga Abeyseela)
Date: Thu, 29 Aug 2013 21:25:42 +1000
Subject: Fwd: Varnish round robin using Director - FIXED
Message-ID: <CAOMTrfcSky-4z7NfxCmiX31=LcKLoLS9cOrPggZc4KDM1+Lv5w@mail.gmail.com>

Sorry my mistake, i need to define the backend before use it.

now i can start the daemon, but it is not doing the round-robin
loadbalancing. i want the first request go to the backend website and the
second should go to the api.

director clust round-robin {
{ .backend = website;
}
{ .backend = api;
}

sub vcl_recv {
if ( req.url ~ "/api/") {
set req.backend = api;
set req.http.host = "hari.com";
} else {
set req.backend = clust;
}
}
it always going to the backend -website

---------- Forwarded message ----------
From: Tharanga Abeyseela <tharanga.abeyseela at gmail.com>
Date: Thu, Aug 29, 2013 at 8:57 PM
Subject: Varnish round robin using Director
To: varnish-misc at varnish-cache.org


Hi Guys,

I'm trying to load-balance requests between two servers. i have the
following config.

backend website {
    .host = "xxx.com";
    .port = "8080";
   .probe = {
    .url = "/favicon.ico";
    .timeout = 60ms;
    .interval = 60s;
    .window = 5;
    .threshold = 3;
}
}
backend api {
    .host = "yyy.com";
    .port = "8080";
   .probe = {
    .url = "/favicon.ico";
    .timeout = 60ms;
    .interval = 60s;
    .window = 5;
    .threshold = 3;
}
}

sub vcl_recv {
if ( req.url ~ "/api/") {
set req.backend = api;
set req.http.host = "yyyy.com";
} else {
set req.backend = website;
}
}

director clust round-robin {
{ .backend = website; }
{ .backend = api; }
}

but this is giving the following error.

Unused backend clust, defined:
('input' Line 39 Pos 10)
director clust round-robin {
---------#####--------------

then i was tyring to add set req.backend = clust (then it is giving the
error,

Symbol not found: 'clust' (expected type BACKEND):

am i missing something here.

Cheers,
Tharanga
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130829/5ca11b73/attachment.html>

From tharanga.abeyseela at gmail.com  Thu Aug 29 13:17:48 2013
From: tharanga.abeyseela at gmail.com (Tharanga Abeyseela)
Date: Thu, 29 Aug 2013 23:17:48 +1000
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
Message-ID: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>

Hi,

i'm using namebased virtual hosting for 2 web servers, and i need to do the
round-robin loadbalancing
using varnish.  my config as follows

backend website {
    .host = "xxx.com";
    .port = "8080";

}
backend api {
    .host = "yyy.com";
    .port = "8080";

}

director clust round-robin {
{ .backend = api;  }
{ .backend = website;  }

}

sub vcl_recv {
set req.backend = clust;
if (req.request)
    {
    return(pass);
    }

}

But how should i send host-headers to varnish to redirect to the correct
server, This is always going to xxx.com.

Thanks in advance
Tharanga
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130829/a8f1dea5/attachment.html>

From marco at nucleus.it  Thu Aug 29 13:56:31 2013
From: marco at nucleus.it (Marco Felettigh)
Date: Thu, 29 Aug 2013 15:56:31 +0200
Subject: varnishstat and memory  usage
In-Reply-To: <20130826101744.GA30573@immer.varnish-software.com>
References: <20130807130052.0f5662e3@lobo.lobo.dom>
 <20130826101744.GA30573@immer.varnish-software.com>
Message-ID: <20130829155631.375c4f3a@lobo.lobo.dom>

Ok thanks

Marco

On Mon, 26 Aug 2013 12:17:44 +0200
Lasse Karstensen <lkarsten at varnish-software.com> wrote:

> On Wed, Aug 07, 2013 at 01:00:52PM +0200, Marco Felettigh wrote:
> > can someone explain please what this values mean ?
> 
> g means gauge, c means counter.
> 
> s0 means that this is the first configured storage backend. (for
> example one -smalloc,XX argument)
> 
> > varnish-3.0.2
> > from varnishstat -1 :
> > SMA.s0.c_bytes   4421961276     14330.40 Bytes allocated
> > Bytes allocated from Varnish for all the things;
> > cache , thread etc . Right ?
> > SMA.s0.c_freed   3336892882     10813.98 Bytes freed
> > freed ? that means that in the uptime of my Varnish instance it
> > freed 3,3GB of bytes from the cache ?
> 
> Yes. Usually you don't need to care about these.
> 
> > So this counter can go beyond Bytes allocated ?
> > SMA.s0.g_bytes   1085068394          .   Bytes outstanding
> > These are the bytes used by cache ?
> 
> Yes.
> 
> If you want to graph storage use, have a look at the munin plugin for
> Varnish.
> 




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130829/c3fe4870/attachment.bin>

From perbu at varnish-software.com  Thu Aug 29 13:58:35 2013
From: perbu at varnish-software.com (Per Buer)
Date: Thu, 29 Aug 2013 15:58:35 +0200
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
Message-ID: <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>

Hi mate,

On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
tharanga.abeyseela at gmail.com> wrote:

> (..)
> But how should i send host-headers to varnish to redirect to the correct
> server, This is always going to xxx.com.
>

You don't. :-)

You need to dispatch the request to the right backend in vcl_recv and have
the backend handle that host. You could start rewriting the host in
vcl_recv, but I would advise against it.

So, if you have two vhosts handled by "website" you just set that as the
backend in vcl_recv.

-- 
 <http://www.varnish-software.com/> *Per Buer*
CTO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130829/61180e2f/attachment.html>

From drupalnut at gmail.com  Thu Aug 29 19:34:13 2013
From: drupalnut at gmail.com (Jack Tuhman)
Date: Thu, 29 Aug 2013 15:34:13 -0400
Subject: Apache->Varnish->Apache
Message-ID: <CAN=XCeO6e++GKGW5XtKbx5aJBsvNbSVoe+avqusiUt2kcxMPRQ@mail.gmail.com>

Hi All,

I have Apache running on port 443 that forwards to varnish, that forwards
to apache.

While I know most are using nginx to respond to 443 traffic, for reasons
outside the scope of this question, I need to use apache.

My issue is, I need apache on 443, to run use a module, that will set
REMOTE_USER, I need to get this passed to varnish, then to the apache
running on port 80, so that the PHP application there and use REMOTE_USER.

Has anyone done this?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130829/7af0cf77/attachment.html>

From james at ifixit.com  Thu Aug 29 20:16:41 2013
From: james at ifixit.com (James Pearson)
Date: Thu, 29 Aug 2013 13:16:41 -0700
Subject: Apache->Varnish->Apache
In-Reply-To: <CAN=XCeO6e++GKGW5XtKbx5aJBsvNbSVoe+avqusiUt2kcxMPRQ@mail.gmail.com>
References: <CAN=XCeO6e++GKGW5XtKbx5aJBsvNbSVoe+avqusiUt2kcxMPRQ@mail.gmail.com>
Message-ID: <1377807054-sup-464@geror.local>

Excerpts from Jack Tuhman's message of 2013-08-29 12:34:13 -0700:
> Hi All,
> 
> I have Apache running on port 443 that forwards to varnish, that forwards
> to apache.
> 
> While I know most are using nginx to respond to 443 traffic, for reasons
> outside the scope of this question, I need to use apache.

We run a similar stack.

> My issue is, I need apache on 443, to run use a module, that will set
> REMOTE_USER, I need to get this passed to varnish, then to the apache
> running on port 80, so that the PHP application there and use REMOTE_USER.
> 
> Has anyone done this?

We don't use REMOTE_USER, but I imagine it wouldn't be too dissimilar from
other headers.  You may need to check the X-Forwarded-User - I'm not sure if
Apache will set that automatically.

At which point in the stack is it getting lost?




From tharanga.abeyseela at gmail.com  Fri Aug 30 00:02:36 2013
From: tharanga.abeyseela at gmail.com (Tharanga Abeyseela)
Date: Fri, 30 Aug 2013 10:02:36 +1000
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
Message-ID: <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>

Hi Per,

Thanks for the reply. Actually my set up looks like this.


cluster.com - 192.168.0.200 (varnish/port 80)
xxx.com - 192.168.0.100  (apache,namebased vhost/8080 - backendname -
website)
yyy.com - 192.168.0.100  (apache,namebased vhost/8080 -backendname - api)

cluster.com is the varnish server and front-end connections coming to this
and rewrite to other defined back-ends (round-robin based balancing)

backend website {
    .host = "xxx.com";
    .port = "8080";

}
backend api {
    .host = "yyy.com";
    .port = "8080";

}

director clust round-robin {
{ .backend = api;  }
{ .backend = website;  }

}

sub vcl_recv {
set req.backend = clust;
if (req.request)
    {
    return(pass);
    }

}


when i hit the cluster.com , it is always going to xxx.com, but what i need
to do is first request go to xxx.com second request yyy.com and so
on...when i add another server (different host/different IP say
192.168.0.111/zzz.com, and a different backend) , it goes like this

first request - xxx.com
second request - xxxx.com
third request - zzz.com


but i can change the default behavior by setting up set req.host =
yyy.comand then it will goes to
first request - yyy.com
second request - yyy.com
third request - zzz.com

this is something to do with the host-header forwarding to the correct
back-end. how should i add that functionality to the vcl_recv ?
appreciate your help on this, this is working perfectly with other servers
(different servers, not with namebased vhosts)

cheers,
Tharanga















On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com>wrote:

> Hi mate,
>
> On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
> tharanga.abeyseela at gmail.com> wrote:
>
>> (..)
>>
>> But how should i send host-headers to varnish to redirect to the correct
>> server, This is always going to xxx.com.
>>
>
> You don't. :-)
>
> You need to dispatch the request to the right backend in vcl_recv and have
> the backend handle that host. You could start rewriting the host in
> vcl_recv, but I would advise against it.
>
> So, if you have two vhosts handled by "website" you just set that as the
> backend in vcl_recv.
>
> --
>  <http://www.varnish-software.com/> *Per Buer*
> CTO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/f879800b/attachment.html>

From perbu at varnish-software.com  Fri Aug 30 07:05:13 2013
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 30 Aug 2013 09:05:13 +0200
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
 <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
Message-ID: <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>

Hi Tharanga,

You seem to be a bit confused about how the directors and backends work in
Varnish. Please read this:

https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish

Note that the .host property of the backend has NOTHING to do with the
virtual host.


On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <
tharanga.abeyseela at gmail.com> wrote:

> Hi Per,
>
> Thanks for the reply. Actually my set up looks like this.
>
>
> cluster.com - 192.168.0.200 (varnish/port 80)
> xxx.com - 192.168.0.100  (apache,namebased vhost/8080 - backendname -
> website)
> yyy.com - 192.168.0.100  (apache,namebased vhost/8080 -backendname - api)
>
> cluster.com is the varnish server and front-end connections coming to
> this and rewrite to other defined back-ends (round-robin based balancing)
>
>
> backend website {
>     .host = "xxx.com";
>     .port = "8080";
>
> }
> backend api {
>     .host = "yyy.com";
>     .port = "8080";
>
> }
>
> director clust round-robin {
> { .backend = api;  }
> { .backend = website;  }
>
> }
>
> sub vcl_recv {
> set req.backend = clust;
> if (req.request)
>     {
>     return(pass);
>     }
>
> }
>
>
> when i hit the cluster.com , it is always going to xxx.com, but what i
> need to do is first request go to xxx.com second request yyy.com and so
> on...when i add another server (different host/different IP say
> 192.168.0.111/zzz.com, and a different backend) , it goes like this
>
> first request - xxx.com
> second request - xxxx.com
> third request - zzz.com
>
>
> but i can change the default behavior by setting up set req.host = yyy.comand then it will goes to
> first request - yyy.com
> second request - yyy.com
> third request - zzz.com
>
> this is something to do with the host-header forwarding to the correct
> back-end. how should i add that functionality to the vcl_recv ?
> appreciate your help on this, this is working perfectly with other servers
> (different servers, not with namebased vhosts)
>
> cheers,
> Tharanga
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com>wrote:
>
>> Hi mate,
>>
>> On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
>> tharanga.abeyseela at gmail.com> wrote:
>>
>>> (..)
>>>
>>> But how should i send host-headers to varnish to redirect to the correct
>>> server, This is always going to xxx.com.
>>>
>>
>> You don't. :-)
>>
>> You need to dispatch the request to the right backend in vcl_recv and
>> have the backend handle that host. You could start rewriting the host in
>> vcl_recv, but I would advise against it.
>>
>> So, if you have two vhosts handled by "website" you just set that as the
>> backend in vcl_recv.
>>
>> --
>>  <http://www.varnish-software.com/> *Per Buer*
>> CTO | Varnish Software AS
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>> Winner of the Red Herring Top 100 Europe Award 2013
>>
>>
>>
>


-- 
 <http://www.varnish-software.com/> *Per Buer*
CTO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/8ca799c9/attachment.html>

From tharanga.abeyseela at gmail.com  Fri Aug 30 08:42:14 2013
From: tharanga.abeyseela at gmail.com (Tharanga Abeyseela)
Date: Fri, 30 Aug 2013 18:42:14 +1000
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
 <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
 <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>
Message-ID: <CAOMTrfcCs4oeHW+3vOg7eAqVFKKu42+bhStvfOZZtXCSvHR6bA@mail.gmail.com>

Hi,

Actually i tried that link, still it is not working. it should work for a
ip based virtual host, but namebased virtual host is not working.
becuase main request coming to varnish/80 and varnish should redirect the
traffic to the backend server based on my round-robin director. when you
resolve dns for those two hosts (xxx.com and yyy.com) it reuturns the same
IP. that is 192.168.0.100. But there should be a way to send the
host-header to apache. Not sure how it works here. But when i use different
server (192,168.0.200/ zzz.com) it works without any issue, it roundrobin
the traffic. This is an issue with namebased virtualhost.

varnish recevie the traffic (cluster.com is my  varnish server), then it
use it vcl to redirect based on my round-robin setting. but how it can send
the traffic to the correct virtualhost. because it gets only the same IP
address from the DNS. I managed to change the default behaviour by
forcefully setting the host-header as follows to my other
virtualhost(yyy..com)
set.req.host="yyy.com" ;
then all request went to yyy.com instead of xxx.com. but what i want is  a
round-robin fashio...to send traffic. first req to xxx.com/ 2nd req to
yyy.com and so on..

I was trying to find some resourses on the net, stackoverflow etc..but no
luck yet.


Thanks again for your help
Cheers,
Tharanga


On Fri, Aug 30, 2013 at 5:05 PM, Per Buer <perbu at varnish-software.com>wrote:

> Hi Tharanga,
>
> You seem to be a bit confused about how the directors and backends work in
> Varnish. Please read this:
>
>
> https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish
>
> Note that the .host property of the backend has NOTHING to do with the
> virtual host.
>
>
> On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <
> tharanga.abeyseela at gmail.com> wrote:
>
>> Hi Per,
>>
>> Thanks for the reply. Actually my set up looks like this.
>>
>>
>> cluster.com - 192.168.0.200 (varnish/port 80)
>> xxx.com - 192.168.0.100  (apache,namebased vhost/8080 - backendname -
>> website)
>> yyy.com - 192.168.0.100  (apache,namebased vhost/8080 -backendname - api)
>>
>> cluster.com is the varnish server and front-end connections coming to
>> this and rewrite to other defined back-ends (round-robin based balancing)
>>
>>
>> backend website {
>>     .host = "xxx.com";
>>     .port = "8080";
>>
>> }
>> backend api {
>>     .host = "yyy.com";
>>     .port = "8080";
>>
>> }
>>
>> director clust round-robin {
>> { .backend = api;  }
>> { .backend = website;  }
>>
>> }
>>
>> sub vcl_recv {
>> set req.backend = clust;
>> if (req.request)
>>     {
>>     return(pass);
>>     }
>>
>> }
>>
>>
>> when i hit the cluster.com , it is always going to xxx.com, but what i
>> need to do is first request go to xxx.com second request yyy.com and so
>> on...when i add another server (different host/different IP say
>> 192.168.0.111/zzz.com, and a different backend) , it goes like this
>>
>> first request - xxx.com
>> second request - xxxx.com
>> third request - zzz.com
>>
>>
>> but i can change the default behavior by setting up set req.host =
>> yyy.com and then it will goes to
>> first request - yyy.com
>> second request - yyy.com
>> third request - zzz.com
>>
>> this is something to do with the host-header forwarding to the correct
>> back-end. how should i add that functionality to the vcl_recv ?
>> appreciate your help on this, this is working perfectly with other
>> servers (different servers, not with namebased vhosts)
>>
>> cheers,
>> Tharanga
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com>wrote:
>>
>>> Hi mate,
>>>
>>> On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
>>> tharanga.abeyseela at gmail.com> wrote:
>>>
>>>> (..)
>>>>
>>>> But how should i send host-headers to varnish to redirect to the
>>>> correct server, This is always going to xxx.com.
>>>>
>>>
>>> You don't. :-)
>>>
>>> You need to dispatch the request to the right backend in vcl_recv and
>>> have the backend handle that host. You could start rewriting the host in
>>> vcl_recv, but I would advise against it.
>>>
>>> So, if you have two vhosts handled by "website" you just set that as the
>>> backend in vcl_recv.
>>>
>>> --
>>>  <http://www.varnish-software.com/> *Per Buer*
>>> CTO | Varnish Software AS
>>> Phone: +47 958 39 117 | Skype: per.buer
>>> We Make Websites Fly!
>>>
>>> Winner of the Red Herring Top 100 Europe Award 2013
>>>
>>>
>>>
>>
>
>
> --
>  <http://www.varnish-software.com/> *Per Buer*
> CTO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/2faa4102/attachment.html>

From perbu at varnish-software.com  Fri Aug 30 11:35:48 2013
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 30 Aug 2013 13:35:48 +0200
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOMTrfcCs4oeHW+3vOg7eAqVFKKu42+bhStvfOZZtXCSvHR6bA@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
 <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
 <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>
 <CAOMTrfcCs4oeHW+3vOg7eAqVFKKu42+bhStvfOZZtXCSvHR6bA@mail.gmail.com>
Message-ID: <CAOXZevAQUkzkfZ9NQ2A5nEkUrdi41RR=DvcCorjyoRBJkcjiDA@mail.gmail.com>

Hi Tharanga,

You seem not to understand. Varnish will just pass the host header in the
request to the backend you specify. Looking at the VCL you provided you put
both the backends into one director. This is probably not what you want to
do.

Try removing the director. Name the directors xxx and yyy and have the VCL
look like.

sub vcl_recv {
    if (req.http.host ~ "xxx.com") {

        req.backend = xxx;

    } else if (..


}

Also the next bit doesn't make sense:

if (req.request)
    {
    return(pass);
    }

}



On Fri, Aug 30, 2013 at 10:42 AM, Tharanga Abeyseela <
tharanga.abeyseela at gmail.com> wrote:

> Hi,
>
> Actually i tried that link, still it is not working. it should work for a
> ip based virtual host, but namebased virtual host is not working.
> becuase main request coming to varnish/80 and varnish should redirect the
> traffic to the backend server based on my round-robin director. when you
> resolve dns for those two hosts (xxx.com and yyy.com) it reuturns the
> same IP. that is 192.168.0.100. But there should be a way to send the
> host-header to apache. Not sure how it works here. But when i use different
> server (192,168.0.200/ zzz.com) it works without any issue, it roundrobin
> the traffic. This is an issue with namebased virtualhost.
>
> varnish recevie the traffic (cluster.com is my  varnish server), then it
> use it vcl to redirect based on my round-robin setting. but how it can send
> the traffic to the correct virtualhost. because it gets only the same IP
> address from the DNS. I managed to change the default behaviour by
> forcefully setting the host-header as follows to my other
> virtualhost(yyy..com)
> set.req.host="yyy.com" ;
> then all request went to yyy.com instead of xxx.com. but what i want is
> a round-robin fashio...to send traffic. first req to xxx.com/ 2nd req to
> yyy.com and so on..
>
> I was trying to find some resourses on the net, stackoverflow etc..but no
> luck yet.
>
>
> Thanks again for your help
> Cheers,
> Tharanga
>
>
> On Fri, Aug 30, 2013 at 5:05 PM, Per Buer <perbu at varnish-software.com>wrote:
>
>> Hi Tharanga,
>>
>> You seem to be a bit confused about how the directors and backends work
>> in Varnish. Please read this:
>>
>>
>> https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish
>>
>> Note that the .host property of the backend has NOTHING to do with the
>> virtual host.
>>
>>
>> On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <
>> tharanga.abeyseela at gmail.com> wrote:
>>
>>> Hi Per,
>>>
>>> Thanks for the reply. Actually my set up looks like this.
>>>
>>>
>>> cluster.com - 192.168.0.200 (varnish/port 80)
>>> xxx.com - 192.168.0.100  (apache,namebased vhost/8080 - backendname -
>>> website)
>>> yyy.com - 192.168.0.100  (apache,namebased vhost/8080 -backendname -
>>> api)
>>>
>>> cluster.com is the varnish server and front-end connections coming to
>>> this and rewrite to other defined back-ends (round-robin based balancing)
>>>
>>>
>>>
>>> backend website {
>>>     .host = "xxx.com";
>>>     .port = "8080";
>>>
>>> }
>>> backend api {
>>>     .host = "yyy.com";
>>>     .port = "8080";
>>>
>>> }
>>>
>>> director clust round-robin {
>>> { .backend = api;  }
>>> { .backend = website;  }
>>>
>>> }
>>>
>>> sub vcl_recv {
>>> set req.backend = clust;
>>> if (req.request)
>>>     {
>>>     return(pass);
>>>     }
>>>
>>> }
>>>
>>>
>>> when i hit the cluster.com , it is always going to xxx.com, but what i
>>> need to do is first request go to xxx.com second request yyy.com and so
>>> on...when i add another server (different host/different IP say
>>> 192.168.0.111/zzz.com, and a different backend) , it goes like this
>>>
>>> first request - xxx.com
>>> second request - xxxx.com
>>> third request - zzz.com
>>>
>>>
>>> but i can change the default behavior by setting up set req.host =
>>> yyy.com and then it will goes to
>>> first request - yyy.com
>>> second request - yyy.com
>>> third request - zzz.com
>>>
>>> this is something to do with the host-header forwarding to the correct
>>> back-end. how should i add that functionality to the vcl_recv ?
>>> appreciate your help on this, this is working perfectly with other
>>> servers (different servers, not with namebased vhosts)
>>>
>>> cheers,
>>> Tharanga
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com>wrote:
>>>
>>>> Hi mate,
>>>>
>>>> On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
>>>> tharanga.abeyseela at gmail.com> wrote:
>>>>
>>>>> (..)
>>>>>
>>>>> But how should i send host-headers to varnish to redirect to the
>>>>> correct server, This is always going to xxx.com.
>>>>>
>>>>
>>>> You don't. :-)
>>>>
>>>> You need to dispatch the request to the right backend in vcl_recv and
>>>> have the backend handle that host. You could start rewriting the host in
>>>> vcl_recv, but I would advise against it.
>>>>
>>>> So, if you have two vhosts handled by "website" you just set that as
>>>> the backend in vcl_recv.
>>>>
>>>> --
>>>>  <http://www.varnish-software.com/> *Per Buer*
>>>> CTO | Varnish Software AS
>>>> Phone: +47 958 39 117 | Skype: per.buer
>>>> We Make Websites Fly!
>>>>
>>>> Winner of the Red Herring Top 100 Europe Award 2013
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>>  <http://www.varnish-software.com/> *Per Buer*
>> CTO | Varnish Software AS
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>> Winner of the Red Herring Top 100 Europe Award 2013
>>
>>
>>
>


-- 
 <http://www.varnish-software.com/> *Per Buer*
CTO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/c00d1210/attachment.html>

From tharanga.abeyseela at gmail.com  Fri Aug 30 11:50:54 2013
From: tharanga.abeyseela at gmail.com (Tharanga Abeyseela)
Date: Fri, 30 Aug 2013 21:50:54 +1000
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOXZevAQUkzkfZ9NQ2A5nEkUrdi41RR=DvcCorjyoRBJkcjiDA@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
 <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
 <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>
 <CAOMTrfcCs4oeHW+3vOg7eAqVFKKu42+bhStvfOZZtXCSvHR6bA@mail.gmail.com>
 <CAOXZevAQUkzkfZ9NQ2A5nEkUrdi41RR=DvcCorjyoRBJkcjiDA@mail.gmail.com>
Message-ID: <CAOMTrffP5cTf_SZf6B6QbuWDk8mjaS5QQ-ua82K=HyDCBDdaWA@mail.gmail.com>

Thanks for the info. But in this way you can't do the round-robin
loadblancing right . actually i need to use the varnish (cluster.com/80)
and redirect requests to both back-ends in round-robin fashion.

request ---> varnish (cluster.com)  ---RR-loadblance---> xxx.com and
yyyy.com

I'm not looking to send the request to xxx.com or yyy.com directly. im
sending the reuquest to cluter.com that is my vanrish server listing on
port 80.

Thanks,
Tharanga


On Fri, Aug 30, 2013 at 9:35 PM, Per Buer <perbu at varnish-software.com>wrote:

> Hi Tharanga,
>
> You seem not to understand. Varnish will just pass the host header in the
> request to the backend you specify. Looking at the VCL you provided you put
> both the backends into one director. This is probably not what you want to
> do.
>
> Try removing the director. Name the directors xxx and yyy and have the VCL
> look like.
>
> sub vcl_recv {
>     if (req.http.host ~ "xxx.com") {
>
>         req.backend = xxx;
>
>     } else if (..
>
>
> }
>
> Also the next bit doesn't make sense:
>
> if (req.request)
>     {
>     return(pass);
>     }
>
> }
>
>
>
> On Fri, Aug 30, 2013 at 10:42 AM, Tharanga Abeyseela <
> tharanga.abeyseela at gmail.com> wrote:
>
>> Hi,
>>
>> Actually i tried that link, still it is not working. it should work for a
>> ip based virtual host, but namebased virtual host is not working.
>> becuase main request coming to varnish/80 and varnish should redirect the
>> traffic to the backend server based on my round-robin director. when you
>> resolve dns for those two hosts (xxx.com and yyy.com) it reuturns the
>> same IP. that is 192.168.0.100. But there should be a way to send the
>> host-header to apache. Not sure how it works here. But when i use different
>> server (192,168.0.200/ zzz.com) it works without any issue, it
>> roundrobin the traffic. This is an issue with namebased virtualhost.
>>
>> varnish recevie the traffic (cluster.com is my  varnish server), then it
>> use it vcl to redirect based on my round-robin setting. but how it can send
>> the traffic to the correct virtualhost. because it gets only the same IP
>> address from the DNS. I managed to change the default behaviour by
>> forcefully setting the host-header as follows to my other
>> virtualhost(yyy..com)
>> set.req.host="yyy.com" ;
>> then all request went to yyy.com instead of xxx.com. but what i want is
>> a round-robin fashio...to send traffic. first req to xxx.com/ 2nd req to
>> yyy.com and so on..
>>
>> I was trying to find some resourses on the net, stackoverflow etc..but no
>> luck yet.
>>
>>
>> Thanks again for your help
>> Cheers,
>> Tharanga
>>
>>
>> On Fri, Aug 30, 2013 at 5:05 PM, Per Buer <perbu at varnish-software.com>wrote:
>>
>>> Hi Tharanga,
>>>
>>> You seem to be a bit confused about how the directors and backends work
>>> in Varnish. Please read this:
>>>
>>>
>>> https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish
>>>
>>> Note that the .host property of the backend has NOTHING to do with the
>>> virtual host.
>>>
>>>
>>> On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <
>>> tharanga.abeyseela at gmail.com> wrote:
>>>
>>>> Hi Per,
>>>>
>>>> Thanks for the reply. Actually my set up looks like this.
>>>>
>>>>
>>>> cluster.com - 192.168.0.200 (varnish/port 80)
>>>> xxx.com - 192.168.0.100  (apache,namebased vhost/8080 - backendname -
>>>> website)
>>>> yyy.com - 192.168.0.100  (apache,namebased vhost/8080 -backendname -
>>>> api)
>>>>
>>>> cluster.com is the varnish server and front-end connections coming to
>>>> this and rewrite to other defined back-ends (round-robin based balancing)
>>>>
>>>>
>>>>
>>>> backend website {
>>>>     .host = "xxx.com";
>>>>     .port = "8080";
>>>>
>>>> }
>>>> backend api {
>>>>     .host = "yyy.com";
>>>>     .port = "8080";
>>>>
>>>> }
>>>>
>>>> director clust round-robin {
>>>> { .backend = api;  }
>>>> { .backend = website;  }
>>>>
>>>> }
>>>>
>>>> sub vcl_recv {
>>>> set req.backend = clust;
>>>> if (req.request)
>>>>     {
>>>>     return(pass);
>>>>     }
>>>>
>>>> }
>>>>
>>>>
>>>> when i hit the cluster.com , it is always going to xxx.com, but what i
>>>> need to do is first request go to xxx.com second request yyy.com and
>>>> so on...when i add another server (different host/different IP say
>>>> 192.168.0.111/zzz.com, and a different backend) , it goes like this
>>>>
>>>> first request - xxx.com
>>>> second request - xxxx.com
>>>> third request - zzz.com
>>>>
>>>>
>>>> but i can change the default behavior by setting up set req.host =
>>>> yyy.com and then it will goes to
>>>> first request - yyy.com
>>>> second request - yyy.com
>>>> third request - zzz.com
>>>>
>>>> this is something to do with the host-header forwarding to the correct
>>>> back-end. how should i add that functionality to the vcl_recv ?
>>>> appreciate your help on this, this is working perfectly with other
>>>> servers (different servers, not with namebased vhosts)
>>>>
>>>> cheers,
>>>> Tharanga
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com>wrote:
>>>>
>>>>> Hi mate,
>>>>>
>>>>> On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
>>>>> tharanga.abeyseela at gmail.com> wrote:
>>>>>
>>>>>> (..)
>>>>>>
>>>>>> But how should i send host-headers to varnish to redirect to the
>>>>>> correct server, This is always going to xxx.com.
>>>>>>
>>>>>
>>>>> You don't. :-)
>>>>>
>>>>> You need to dispatch the request to the right backend in vcl_recv and
>>>>> have the backend handle that host. You could start rewriting the host in
>>>>> vcl_recv, but I would advise against it.
>>>>>
>>>>> So, if you have two vhosts handled by "website" you just set that as
>>>>> the backend in vcl_recv.
>>>>>
>>>>> --
>>>>>  <http://www.varnish-software.com/> *Per Buer*
>>>>> CTO | Varnish Software AS
>>>>> Phone: +47 958 39 117 | Skype: per.buer
>>>>> We Make Websites Fly!
>>>>>
>>>>> Winner of the Red Herring Top 100 Europe Award 2013
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>  <http://www.varnish-software.com/> *Per Buer*
>>> CTO | Varnish Software AS
>>> Phone: +47 958 39 117 | Skype: per.buer
>>> We Make Websites Fly!
>>>
>>> Winner of the Red Herring Top 100 Europe Award 2013
>>>
>>>
>>>
>>
>
>
> --
>  <http://www.varnish-software.com/> *Per Buer*
> CTO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/1d86953f/attachment.html>

From perbu at varnish-software.com  Fri Aug 30 11:53:51 2013
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 30 Aug 2013 13:53:51 +0200
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOMTrffP5cTf_SZf6B6QbuWDk8mjaS5QQ-ua82K=HyDCBDdaWA@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
 <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
 <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>
 <CAOMTrfcCs4oeHW+3vOg7eAqVFKKu42+bhStvfOZZtXCSvHR6bA@mail.gmail.com>
 <CAOXZevAQUkzkfZ9NQ2A5nEkUrdi41RR=DvcCorjyoRBJkcjiDA@mail.gmail.com>
 <CAOMTrffP5cTf_SZf6B6QbuWDk8mjaS5QQ-ua82K=HyDCBDdaWA@mail.gmail.com>
Message-ID: <CAOXZevC5bWMaUm4ze+ebK6O5Joxxh6LTuisJPq8r3uncS7J7Eg@mail.gmail.com>

You must set up one director for each set of vhosts that you are planning
to serve. You can't just chuck all the backends into one director.


Per.


On Fri, Aug 30, 2013 at 1:50 PM, Tharanga Abeyseela <
tharanga.abeyseela at gmail.com> wrote:

> Thanks for the info. But in this way you can't do the round-robin
> loadblancing right . actually i need to use the varnish (cluster.com/80)
> and redirect requests to both back-ends in round-robin fashion.
>
> request ---> varnish (cluster.com)  ---RR-loadblance---> xxx.com and
> yyyy.com
>
> I'm not looking to send the request to xxx.com or yyy.com directly. im
> sending the reuquest to cluter.com that is my vanrish server listing on
> port 80.
>
> Thanks,
> Tharanga
>
>
> On Fri, Aug 30, 2013 at 9:35 PM, Per Buer <perbu at varnish-software.com>wrote:
>
>> Hi Tharanga,
>>
>> You seem not to understand. Varnish will just pass the host header in the
>> request to the backend you specify. Looking at the VCL you provided you put
>> both the backends into one director. This is probably not what you want to
>> do.
>>
>> Try removing the director. Name the directors xxx and yyy and have the
>> VCL look like.
>>
>> sub vcl_recv {
>>     if (req.http.host ~ "xxx.com") {
>>
>>         req.backend = xxx;
>>
>>
>>     } else if (..
>>
>>
>> }
>>
>> Also the next bit doesn't make sense:
>>
>> if (req.request)
>>     {
>>     return(pass);
>>     }
>>
>> }
>>
>>
>>
>> On Fri, Aug 30, 2013 at 10:42 AM, Tharanga Abeyseela <
>> tharanga.abeyseela at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Actually i tried that link, still it is not working. it should work for
>>> a ip based virtual host, but namebased virtual host is not working.
>>> becuase main request coming to varnish/80 and varnish should redirect
>>> the traffic to the backend server based on my round-robin director. when
>>> you resolve dns for those two hosts (xxx.com and yyy.com) it reuturns
>>> the same IP. that is 192.168.0.100. But there should be a way to send the
>>> host-header to apache. Not sure how it works here. But when i use different
>>> server (192,168.0.200/ zzz.com) it works without any issue, it
>>> roundrobin the traffic. This is an issue with namebased virtualhost.
>>>
>>> varnish recevie the traffic (cluster.com is my  varnish server), then
>>> it use it vcl to redirect based on my round-robin setting. but how it can
>>> send the traffic to the correct virtualhost. because it gets only the same
>>> IP address from the DNS. I managed to change the default behaviour by
>>> forcefully setting the host-header as follows to my other
>>> virtualhost(yyy..com)
>>> set.req.host="yyy.com" ;
>>> then all request went to yyy.com instead of xxx.com. but what i want
>>> is  a round-robin fashio...to send traffic. first req to xxx.com/ 2nd
>>> req to yyy.com and so on..
>>>
>>> I was trying to find some resourses on the net, stackoverflow etc..but
>>> no luck yet.
>>>
>>>
>>> Thanks again for your help
>>> Cheers,
>>> Tharanga
>>>
>>>
>>> On Fri, Aug 30, 2013 at 5:05 PM, Per Buer <perbu at varnish-software.com>wrote:
>>>
>>>> Hi Tharanga,
>>>>
>>>> You seem to be a bit confused about how the directors and backends work
>>>> in Varnish. Please read this:
>>>>
>>>>
>>>> https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish
>>>>
>>>> Note that the .host property of the backend has NOTHING to do with the
>>>> virtual host.
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <
>>>> tharanga.abeyseela at gmail.com> wrote:
>>>>
>>>>> Hi Per,
>>>>>
>>>>> Thanks for the reply. Actually my set up looks like this.
>>>>>
>>>>>
>>>>> cluster.com - 192.168.0.200 (varnish/port 80)
>>>>> xxx.com - 192.168.0.100  (apache,namebased vhost/8080 - backendname -
>>>>> website)
>>>>> yyy.com - 192.168.0.100  (apache,namebased vhost/8080 -backendname -
>>>>> api)
>>>>>
>>>>> cluster.com is the varnish server and front-end connections coming to
>>>>> this and rewrite to other defined back-ends (round-robin based balancing)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> backend website {
>>>>>     .host = "xxx.com";
>>>>>     .port = "8080";
>>>>>
>>>>> }
>>>>> backend api {
>>>>>     .host = "yyy.com";
>>>>>     .port = "8080";
>>>>>
>>>>> }
>>>>>
>>>>> director clust round-robin {
>>>>> { .backend = api;  }
>>>>> { .backend = website;  }
>>>>>
>>>>> }
>>>>>
>>>>> sub vcl_recv {
>>>>> set req.backend = clust;
>>>>> if (req.request)
>>>>>     {
>>>>>     return(pass);
>>>>>     }
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>> when i hit the cluster.com , it is always going to xxx.com, but what
>>>>> i need to do is first request go to xxx.com second request yyy.comand so on...when i add another server (different host/different IP say
>>>>> 192.168.0.111/zzz.com, and a different backend) , it goes like this
>>>>>
>>>>> first request - xxx.com
>>>>> second request - xxxx.com
>>>>> third request - zzz.com
>>>>>
>>>>>
>>>>> but i can change the default behavior by setting up set req.host =
>>>>> yyy.com and then it will goes to
>>>>> first request - yyy.com
>>>>> second request - yyy.com
>>>>> third request - zzz.com
>>>>>
>>>>> this is something to do with the host-header forwarding to the correct
>>>>> back-end. how should i add that functionality to the vcl_recv ?
>>>>> appreciate your help on this, this is working perfectly with other
>>>>> servers (different servers, not with namebased vhosts)
>>>>>
>>>>> cheers,
>>>>> Tharanga
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com
>>>>> > wrote:
>>>>>
>>>>>> Hi mate,
>>>>>>
>>>>>> On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
>>>>>> tharanga.abeyseela at gmail.com> wrote:
>>>>>>
>>>>>>> (..)
>>>>>>>
>>>>>>> But how should i send host-headers to varnish to redirect to the
>>>>>>> correct server, This is always going to xxx.com.
>>>>>>>
>>>>>>
>>>>>> You don't. :-)
>>>>>>
>>>>>> You need to dispatch the request to the right backend in vcl_recv and
>>>>>> have the backend handle that host. You could start rewriting the host in
>>>>>> vcl_recv, but I would advise against it.
>>>>>>
>>>>>> So, if you have two vhosts handled by "website" you just set that as
>>>>>> the backend in vcl_recv.
>>>>>>
>>>>>> --
>>>>>>  <http://www.varnish-software.com/> *Per Buer*
>>>>>> CTO | Varnish Software AS
>>>>>> Phone: +47 958 39 117 | Skype: per.buer
>>>>>> We Make Websites Fly!
>>>>>>
>>>>>> Winner of the Red Herring Top 100 Europe Award 2013
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>  <http://www.varnish-software.com/> *Per Buer*
>>>> CTO | Varnish Software AS
>>>> Phone: +47 958 39 117 | Skype: per.buer
>>>> We Make Websites Fly!
>>>>
>>>> Winner of the Red Herring Top 100 Europe Award 2013
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>>  <http://www.varnish-software.com/> *Per Buer*
>> CTO | Varnish Software AS
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>> Winner of the Red Herring Top 100 Europe Award 2013
>>
>>
>>
>


-- 
 <http://www.varnish-software.com/> *Per Buer*
CTO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/ae7d92ba/attachment.html>

From thierry.magnien at sfr.com  Fri Aug 30 12:46:22 2013
From: thierry.magnien at sfr.com (MAGNIEN, Thierry)
Date: Fri, 30 Aug 2013 12:46:22 +0000
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOXZevAQUkzkfZ9NQ2A5nEkUrdi41RR=DvcCorjyoRBJkcjiDA@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
 <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
 <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>
 <CAOMTrfcCs4oeHW+3vOg7eAqVFKKu42+bhStvfOZZtXCSvHR6bA@mail.gmail.com>
 <CAOXZevAQUkzkfZ9NQ2A5nEkUrdi41RR=DvcCorjyoRBJkcjiDA@mail.gmail.com>
Message-ID: <5D103CE839D50E4CBC62C9FD7B83287C274738FF@EXCN013.encara.local.ads>

Hi,

I think I understood the point?:
- you have 2 backends hosted under the same IP address : www.backend1.com and www.backend2.com
- your varnish server gets requests (for example) for www.frontend.com
- you want requests to be load-balanced between your 2 backends BUT this would need the bereq.host to be rewritten to either www.backend1.com or www.backend2.com 
- BUT you can't do this in your VCL because you don't know which backend will be selected.

In fact, this would need the director to rewrite the host header once the real backend has been selected, which I don't think is possible.

Anyway, I still wonder why the hell you need varnish for this, where a HAProxy would certainly do the trick ;-)

Regards,
Thierry


De?: varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org [mailto:varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la part de Per Buer
Envoy??: vendredi 30 ao?t 2013 13:36
??: Tharanga Abeyseela
Cc?: Varnish misc
Objet?: Re: Varnish round-robin loadbalancing with Virtualhost (Namebased)

Hi Tharanga,

You seem not to understand. Varnish will just pass the host header in the request to the backend you specify. Looking at the VCL you provided you put both the backends into one director. This is probably not what you want to do.

Try removing the director. Name the directors xxx and yyy and have the VCL look like.

sub vcl_recv {
    if (req.http.host ~ "xxx.com") {
        req.backend = xxx;
    } else if (..

}
Also the next bit doesn't make sense:
if (req.request)
    {
    return(pass);
    }

}

On Fri, Aug 30, 2013 at 10:42 AM, Tharanga Abeyseela <tharanga.abeyseela at gmail.com> wrote:
Hi,

Actually i tried that link, still it is not working. it should work for a ip based virtual host, but namebased virtual host is not working.
becuase main request coming to varnish/80 and varnish should redirect the traffic to the backend server based on my round-robin director. when you resolve dns for those two hosts (xxx.com and yyy.com) it reuturns the same IP. that is 192.168.0.100. But there should be a way to send the host-header to apache. Not sure how it works here. But when i use different server (192,168.0.200/ zzz.com) it works without any issue, it roundrobin the traffic. This is an issue with namebased virtualhost. 
varnish recevie the traffic (cluster.com is my? varnish server), then it use it vcl to redirect based on my round-robin setting. but how it can send the traffic to the correct virtualhost. because it gets only the same IP address from the DNS. I managed to change the default behaviour by forcefully setting the host-header as follows to my other virtualhost(yyy..com)
set.req.host="yyy.com" ;
then all request went to yyy.com instead of xxx.com. but what i want is? a round-robin fashio...to send traffic. first req to xxx.com/ 2nd req to yyy.com and so on..

I was trying to find some resourses on the net, stackoverflow etc..but no luck yet. 

Thanks again for your help 
Cheers,
Tharanga

On Fri, Aug 30, 2013 at 5:05 PM, Per Buer <perbu at varnish-software.com> wrote:
Hi Tharanga,

You seem to be a bit confused about how the directors and backends work in Varnish. Please read this:

https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish

Note that the .host property of the backend has NOTHING to do with the virtual host.

On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <tharanga.abeyseela at gmail.com> wrote:
Hi Per,
Thanks for the reply. Actually my set up looks like this.

cluster.com - 192.168.0.200 (varnish/port 80)
xxx.com - 192.168.0.100? (apache,namebased vhost/8080 - backendname - website)
yyy.com - 192.168.0.100? (apache,namebased vhost/8080 -backendname - api)
cluster.com is the varnish server and front-end connections coming to this and rewrite to other defined back-ends (round-robin based balancing)



backend website {
    .host = "xxx.com";
    .port = "8080";

}
backend api {
    .host = "yyy.com";
    .port = "8080";

}

director clust round-robin {
{ .backend = api;  }
{ .backend = website;  }

}

sub vcl_recv {
set req.backend = clust;
if (req.request)
    {
    return(pass);
    }

}

when i hit the cluster.com , it is always going to xxx.com, but what i need to do is first request go to xxx.com second request yyy.com and so on...when i add another server (different host/different IP say 192.168.0.111/zzz.com, and a different backend) , it goes like this
first request - xxx.com
second request - xxxx.com
third request - zzz.com

but i can change the default behavior by setting up set req.host = yyy.com and then it will goes to 
first request - yyy.com
second request - yyy.com
third request - zzz.com
this is something to do with the host-header forwarding to the correct back-end. how should i add that functionality to the vcl_recv ?
appreciate your help on this, this is working perfectly with other servers (different servers, not with namebased vhosts)
cheers,
Tharanga? 

? 










On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com> wrote:
Hi mate,

On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <tharanga.abeyseela at gmail.com> wrote:
(..)

But how should i send host-headers to varnish to redirect to the correct server, This is always going to xxx.com.

You don't. :-)

You need to dispatch the request to the right backend in vcl_recv and have the backend handle that host. You could start rewriting the host in vcl_recv, but I would advise against it.?

So, if you have two vhosts handled by "website" you just set that as the backend in vcl_recv.

-- 

Per Buer
CTO | Varnish Software AS
Phone:?+47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013







-- 

Per Buer
CTO | Varnish Software AS
Phone:?+47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013







-- 

Per Buer
CTO | Varnish Software AS
Phone:?+47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013






From tharanga.abeyseela at gmail.com  Fri Aug 30 12:55:37 2013
From: tharanga.abeyseela at gmail.com (Tharanga Abeyseela)
Date: Fri, 30 Aug 2013 22:55:37 +1000
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <5D103CE839D50E4CBC62C9FD7B83287C274738FF@EXCN013.encara.local.ads>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
 <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
 <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>
 <CAOMTrfcCs4oeHW+3vOg7eAqVFKKu42+bhStvfOZZtXCSvHR6bA@mail.gmail.com>
 <CAOXZevAQUkzkfZ9NQ2A5nEkUrdi41RR=DvcCorjyoRBJkcjiDA@mail.gmail.com>
 <5D103CE839D50E4CBC62C9FD7B83287C274738FF@EXCN013.encara.local.ads>
Message-ID: <CAOMTrfdtSwGpFKCf-UgDeHFXZcUa-BAvsSzjxo8hGuXp==e-Hg@mail.gmail.com>

That is my requirement !!!.  i'm using the same IP address for both hosts,
and need to do the load-balancing between two servers. thats what i wanted
to know,  Actually i'm playing with different varnish configs :) on my
virtual box.,  thought to give this config a try....but that didnt work. it
is working perfectly with different IP's, or IP based Vhosts.
Thanks Thierry for the response.
cheers,
Tharanga


On Fri, Aug 30, 2013 at 10:46 PM, MAGNIEN, Thierry
<thierry.magnien at sfr.com>wrote:

> Hi,
>
> I think I understood the point :
> - you have 2 backends hosted under the same IP address : www.backend1.comand
> www.backend2.com
> - your varnish server gets requests (for example) for www.frontend.com
> - you want requests to be load-balanced between your 2 backends BUT this
> would need the bereq.host to be rewritten to either www.backend1.com or
> www.backend2.com
> - BUT you can't do this in your VCL because you don't know which backend
> will be selected.
>
> In fact, this would need the director to rewrite the host header once the
> real backend has been selected, which I don't think is possible.
>
> Anyway, I still wonder why the hell you need varnish for this, where a
> HAProxy would certainly do the trick ;-)
>
> Regards,
> Thierry
>
>
> De : varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org[mailto:
> varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la
> part de Per Buer
> Envoy? : vendredi 30 ao?t 2013 13:36
> ? : Tharanga Abeyseela
> Cc : Varnish misc
> Objet : Re: Varnish round-robin loadbalancing with Virtualhost (Namebased)
>
> Hi Tharanga,
>
> You seem not to understand. Varnish will just pass the host header in the
> request to the backend you specify. Looking at the VCL you provided you put
> both the backends into one director. This is probably not what you want to
> do.
>
> Try removing the director. Name the directors xxx and yyy and have the VCL
> look like.
>
> sub vcl_recv {
>     if (req.http.host ~ "xxx.com") {
>         req.backend = xxx;
>     } else if (..
>
> }
> Also the next bit doesn't make sense:
> if (req.request)
>     {
>     return(pass);
>     }
>
> }
>
> On Fri, Aug 30, 2013 at 10:42 AM, Tharanga Abeyseela <
> tharanga.abeyseela at gmail.com> wrote:
> Hi,
>
> Actually i tried that link, still it is not working. it should work for a
> ip based virtual host, but namebased virtual host is not working.
> becuase main request coming to varnish/80 and varnish should redirect the
> traffic to the backend server based on my round-robin director. when you
> resolve dns for those two hosts (xxx.com and yyy.com) it reuturns the
> same IP. that is 192.168.0.100. But there should be a way to send the
> host-header to apache. Not sure how it works here. But when i use different
> server (192,168.0.200/ zzz.com) it works without any issue, it roundrobin
> the traffic. This is an issue with namebased virtualhost.
> varnish recevie the traffic (cluster.com is my  varnish server), then it
> use it vcl to redirect based on my round-robin setting. but how it can send
> the traffic to the correct virtualhost. because it gets only the same IP
> address from the DNS. I managed to change the default behaviour by
> forcefully setting the host-header as follows to my other
> virtualhost(yyy..com)
> set.req.host="yyy.com" ;
> then all request went to yyy.com instead of xxx.com. but what i want is
> a round-robin fashio...to send traffic. first req to xxx.com/ 2nd req to
> yyy.com and so on..
>
> I was trying to find some resourses on the net, stackoverflow etc..but no
> luck yet.
>
> Thanks again for your help
> Cheers,
> Tharanga
>
> On Fri, Aug 30, 2013 at 5:05 PM, Per Buer <perbu at varnish-software.com>
> wrote:
> Hi Tharanga,
>
> You seem to be a bit confused about how the directors and backends work in
> Varnish. Please read this:
>
>
> https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish
>
> Note that the .host property of the backend has NOTHING to do with the
> virtual host.
>
> On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <
> tharanga.abeyseela at gmail.com> wrote:
> Hi Per,
> Thanks for the reply. Actually my set up looks like this.
>
> cluster.com - 192.168.0.200 (varnish/port 80)
> xxx.com - 192.168.0.100  (apache,namebased vhost/8080 - backendname -
> website)
> yyy.com - 192.168.0.100  (apache,namebased vhost/8080 -backendname - api)
> cluster.com is the varnish server and front-end connections coming to
> this and rewrite to other defined back-ends (round-robin based balancing)
>
>
>
> backend website {
>     .host = "xxx.com";
>     .port = "8080";
>
> }
> backend api {
>     .host = "yyy.com";
>     .port = "8080";
>
> }
>
> director clust round-robin {
> { .backend = api;  }
> { .backend = website;  }
>
> }
>
> sub vcl_recv {
> set req.backend = clust;
> if (req.request)
>     {
>     return(pass);
>     }
>
> }
>
> when i hit the cluster.com , it is always going to xxx.com, but what i
> need to do is first request go to xxx.com second request yyy.com and so
> on...when i add another server (different host/different IP say
> 192.168.0.111/zzz.com, and a different backend) , it goes like this
> first request - xxx.com
> second request - xxxx.com
> third request - zzz.com
>
> but i can change the default behavior by setting up set req.host = yyy.comand then it will goes to
> first request - yyy.com
> second request - yyy.com
> third request - zzz.com
> this is something to do with the host-header forwarding to the correct
> back-end. how should i add that functionality to the vcl_recv ?
> appreciate your help on this, this is working perfectly with other servers
> (different servers, not with namebased vhosts)
> cheers,
> Tharanga
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com>
> wrote:
> Hi mate,
>
> On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
> tharanga.abeyseela at gmail.com> wrote:
> (..)
>
> But how should i send host-headers to varnish to redirect to the correct
> server, This is always going to xxx.com.
>
> You don't. :-)
>
> You need to dispatch the request to the right backend in vcl_recv and have
> the backend handle that host. You could start rewriting the host in
> vcl_recv, but I would advise against it.
>
> So, if you have two vhosts handled by "website" you just set that as the
> backend in vcl_recv.
>
> --
>
> Per Buer
> CTO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>
>
>
>
>
> --
>
> Per Buer
> CTO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>
>
>
>
>
> --
>
> Per Buer
> CTO | Varnish Software AS
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/8704692d/attachment.html>

From straightflush at gmail.com  Fri Aug 30 13:50:29 2013
From: straightflush at gmail.com (AD)
Date: Fri, 30 Aug 2013 09:50:29 -0400
Subject: ICP Emulation
In-Reply-To: <CAB+OxSBi=y_wdcirfue0Q988P7jQrTwAEOksCFhZoDJQGK5GUg@mail.gmail.com>
References: <CAB+OxSBi=y_wdcirfue0Q988P7jQrTwAEOksCFhZoDJQGK5GUg@mail.gmail.com>
Message-ID: <CANZWjw4f+rcet-OpqxN_Ggfov9US+n5VvV=Oi_TZM0qwZvzGEw@mail.gmail.com>

I think you need to set each varnish server as a backend, and then hash the
URL to a backend (that is a varnish server).  If the backend matches the
current varnish server serving the request then you use the real backend
(origin).  I am not sure the details of the setup, but Artur from Fastly
said they do something similar in their setup maybe one of the Fastly guys
or someone else can comment on how this is done ?

Curious myself as to how to make this work




On Wed, Aug 21, 2013 at 12:07 PM, <maillists0 at gmail.com> wrote:

> An older blog post on the Varnish site says this:
>
> "... you can emulate ICP-like behaviour by writing VCL code to check your
> neighouring cache for the request that you are looking for. Setups like
> these have been discussed on varnish-misc<http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc/>several times and implementing it should be fairly simple."
>
> I must be searching the wrong terms, because I can't find any descriptions
> of this.
>
> It looks to me like the setup would be little more than setting the peer
> as a backend and then setting req.hash_ignore_busy on one of the servers to
> avoid race conditions, as described on the wiki. Is there anything I'm
> missing, or does someone have a reference on how to set this up? Any help
> is appreciated.
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/6533d3db/attachment.html>

From jon.hauge at relationhouse.com  Fri Aug 30 17:13:46 2013
From: jon.hauge at relationhouse.com (Jon Hauge)
Date: Fri, 30 Aug 2013 19:13:46 +0200
Subject: help
Message-ID: <bea46308a6ee332009168529f5525d5a@mail.gmail.com>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/0d51ae5d/attachment.html>

From roberto.fernandezcrisial at gmail.com  Fri Aug 30 18:26:32 2013
From: roberto.fernandezcrisial at gmail.com (=?ISO-8859-1?Q?Roberto_O=2E_Fern=E1ndez_Crisial?=)
Date: Fri, 30 Aug 2013 15:26:32 -0300
Subject: help
In-Reply-To: <bea46308a6ee332009168529f5525d5a@mail.gmail.com>
References: <bea46308a6ee332009168529f5525d5a@mail.gmail.com>
Message-ID: <CAAyZgSgsG2JrXbEU6T_oAbpEr3H9Tw1RqfXheHwCQYuP-YMbkw@mail.gmail.com>

I need somebody..


On Fri, Aug 30, 2013 at 2:13 PM, Jon Hauge <jon.hauge at relationhouse.com>wrote:

>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/51214030/attachment.html>

From david.birdsong at gmail.com  Fri Aug 30 18:49:04 2013
From: david.birdsong at gmail.com (David Birdsong)
Date: Fri, 30 Aug 2013 11:49:04 -0700
Subject: help
In-Reply-To: <CAAyZgSgsG2JrXbEU6T_oAbpEr3H9Tw1RqfXheHwCQYuP-YMbkw@mail.gmail.com>
References: <bea46308a6ee332009168529f5525d5a@mail.gmail.com>
 <CAAyZgSgsG2JrXbEU6T_oAbpEr3H9Tw1RqfXheHwCQYuP-YMbkw@mail.gmail.com>
Message-ID: <CABJfL5jqYgeXEsaQVUSPJJZqyz7HAniQOBJDQbczNf8g23wa+w@mail.gmail.com>

...not just anybody


On Fri, Aug 30, 2013 at 11:26 AM, Roberto O. Fern?ndez Crisial <
roberto.fernandezcrisial at gmail.com> wrote:

> I need somebody..
>
>
> On Fri, Aug 30, 2013 at 2:13 PM, Jon Hauge <jon.hauge at relationhouse.com>wrote:
>
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/17293994/attachment.html>

From Travis.Crowder at penton.com  Fri Aug 30 19:45:56 2013
From: Travis.Crowder at penton.com (Crowder, Travis)
Date: Fri, 30 Aug 2013 19:45:56 +0000
Subject: help
In-Reply-To: <CABJfL5jqYgeXEsaQVUSPJJZqyz7HAniQOBJDQbczNf8g23wa+w@mail.gmail.com>
Message-ID: <CE46608A.1B03D%travis.crowder@penton.com>

? you know I need someone

From: David Birdsong <david.birdsong at gmail.com<mailto:david.birdsong at gmail.com>>
Date: Friday, August 30, 2013 1:49 PM
To: "Roberto O. Fern?ndez Crisial" <roberto.fernandezcrisial at gmail.com<mailto:roberto.fernandezcrisial at gmail.com>>
Cc: Varnish misc <varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>>
Subject: Re: help

...not just anybody


On Fri, Aug 30, 2013 at 11:26 AM, Roberto O. Fern?ndez Crisial <roberto.fernandezcrisial at gmail.com<mailto:roberto.fernandezcrisial at gmail.com>> wrote:
I need somebody..


On Fri, Aug 30, 2013 at 2:13 PM, Jon Hauge <jon.hauge at relationhouse.com<mailto:jon.hauge at relationhouse.com>> wrote:


_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc


_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/f72211f4/attachment.html>

From lee at leetrout.com  Fri Aug 30 19:58:07 2013
From: lee at leetrout.com (Lee Trout)
Date: Fri, 30 Aug 2013 15:58:07 -0400
Subject: help
In-Reply-To: <CABJfL5jqYgeXEsaQVUSPJJZqyz7HAniQOBJDQbczNf8g23wa+w@mail.gmail.com>
References: <bea46308a6ee332009168529f5525d5a@mail.gmail.com>
 <CAAyZgSgsG2JrXbEU6T_oAbpEr3H9Tw1RqfXheHwCQYuP-YMbkw@mail.gmail.com>
 <CABJfL5jqYgeXEsaQVUSPJJZqyz7HAniQOBJDQbczNf8g23wa+w@mail.gmail.com>
Message-ID: <CAMN4RjKJQnsk9MNBLk_3cuQ6=dox1QzPwc7JgXY2nRke0UcEOQ@mail.gmail.com>

you know, I need someone


On Fri, Aug 30, 2013 at 2:49 PM, David Birdsong <david.birdsong at gmail.com>wrote:

> ...not just anybody
>
>
> On Fri, Aug 30, 2013 at 11:26 AM, Roberto O. Fern?ndez Crisial <
> roberto.fernandezcrisial at gmail.com> wrote:
>
>> I need somebody..
>>
>>
>> On Fri, Aug 30, 2013 at 2:13 PM, Jon Hauge <jon.hauge at relationhouse.com>wrote:
>>
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130830/4ced2e99/attachment.html>

From n.j.saunders at gmail.com  Fri Aug 30 23:09:36 2013
From: n.j.saunders at gmail.com (Neil Saunders)
Date: Sat, 31 Aug 2013 00:09:36 +0100
Subject: help
In-Reply-To: <CAMN4RjKJQnsk9MNBLk_3cuQ6=dox1QzPwc7JgXY2nRke0UcEOQ@mail.gmail.com>
References: <bea46308a6ee332009168529f5525d5a@mail.gmail.com>
 <CAAyZgSgsG2JrXbEU6T_oAbpEr3H9Tw1RqfXheHwCQYuP-YMbkw@mail.gmail.com>
 <CABJfL5jqYgeXEsaQVUSPJJZqyz7HAniQOBJDQbczNf8g23wa+w@mail.gmail.com>
 <CAMN4RjKJQnsk9MNBLk_3cuQ6=dox1QzPwc7JgXY2nRke0UcEOQ@mail.gmail.com>
Message-ID: <04BFB182-C081-4950-A8E4-3D025F2C004D@gmail.com>

Heeeeeeeeelllllp

On 30 Aug 2013, at 20:58, Lee Trout <lee at leetrout.com> wrote:

> you know, I need someone
> 
> 
> On Fri, Aug 30, 2013 at 2:49 PM, David Birdsong <david.birdsong at gmail.com> wrote:
>> ...not just anybody
>> 
>> 
>> On Fri, Aug 30, 2013 at 11:26 AM, Roberto O. Fern?ndez Crisial <roberto.fernandezcrisial at gmail.com> wrote:
>>> I need somebody..
>>> 
>>> 
>>> On Fri, Aug 30, 2013 at 2:13 PM, Jon Hauge <jon.hauge at relationhouse.com> wrote:
>>>>  
>>>> 
>>>> 
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>> 
>>> 
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>> 
>> 
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130831/b0ef2477/attachment.html>

From numard at gmail.com  Sat Aug 31 13:49:48 2013
From: numard at gmail.com (Norberto Meijome)
Date: Sat, 31 Aug 2013 23:49:48 +1000
Subject: Varnish round-robin loadbalancing with Virtualhost (Namebased)
In-Reply-To: <CAOMTrfdtSwGpFKCf-UgDeHFXZcUa-BAvsSzjxo8hGuXp==e-Hg@mail.gmail.com>
References: <CAOMTrfcRK2kYjxYw0-bvQsp3t93O6LP5GCZDEmU9d4j_YquerA@mail.gmail.com>
 <CAOXZevCw3H4yfRfB9rFZp5-tOuCD7oxgEiCxtyGGvD4QFx_hfw@mail.gmail.com>
 <CAOMTrfdOfzwtCP38haLkEy5MrroYwzp_eVvgsVeiTJQnG1KaTA@mail.gmail.com>
 <CAOXZevA=+2imHa13D1=mNXdq8HNc=woNxoQu=3wRaZzc4nbG_w@mail.gmail.com>
 <CAOMTrfcCs4oeHW+3vOg7eAqVFKKu42+bhStvfOZZtXCSvHR6bA@mail.gmail.com>
 <CAOXZevAQUkzkfZ9NQ2A5nEkUrdi41RR=DvcCorjyoRBJkcjiDA@mail.gmail.com>
 <5D103CE839D50E4CBC62C9FD7B83287C274738FF@EXCN013.encara.local.ads>
 <CAOMTrfdtSwGpFKCf-UgDeHFXZcUa-BAvsSzjxo8hGuXp==e-Hg@mail.gmail.com>
Message-ID: <CACj2-4LRnwWA8wLexv7oOenR_1x3Hzqc-r9p152utfB5fRCWZw@mail.gmail.com>

Bind to different ports?
On 31/08/2013 12:00 AM, "Tharanga Abeyseela" <tharanga.abeyseela at gmail.com>
wrote:

> That is my requirement !!!.  i'm using the same IP address for both hosts,
> and need to do the load-balancing between two servers. thats what i wanted
> to know,  Actually i'm playing with different varnish configs :) on my
> virtual box.,  thought to give this config a try....but that didnt work. it
> is working perfectly with different IP's, or IP based Vhosts.
> Thanks Thierry for the response.
> cheers,
> Tharanga
>
>
> On Fri, Aug 30, 2013 at 10:46 PM, MAGNIEN, Thierry <
> thierry.magnien at sfr.com> wrote:
>
>> Hi,
>>
>> I think I understood the point :
>> - you have 2 backends hosted under the same IP address : www.backend1.comand
>> www.backend2.com
>> - your varnish server gets requests (for example) for www.frontend.com
>> - you want requests to be load-balanced between your 2 backends BUT this
>> would need the bereq.host to be rewritten to either www.backend1.com or
>> www.backend2.com
>> - BUT you can't do this in your VCL because you don't know which backend
>> will be selected.
>>
>> In fact, this would need the director to rewrite the host header once the
>> real backend has been selected, which I don't think is possible.
>>
>> Anyway, I still wonder why the hell you need varnish for this, where a
>> HAProxy would certainly do the trick ;-)
>>
>> Regards,
>> Thierry
>>
>>
>> De : varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org[mailto:
>> varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la
>> part de Per Buer
>> Envoy? : vendredi 30 ao?t 2013 13:36
>> ? : Tharanga Abeyseela
>> Cc : Varnish misc
>> Objet : Re: Varnish round-robin loadbalancing with Virtualhost (Namebased)
>>
>> Hi Tharanga,
>>
>> You seem not to understand. Varnish will just pass the host header in the
>> request to the backend you specify. Looking at the VCL you provided you put
>> both the backends into one director. This is probably not what you want to
>> do.
>>
>> Try removing the director. Name the directors xxx and yyy and have the
>> VCL look like.
>>
>> sub vcl_recv {
>>     if (req.http.host ~ "xxx.com") {
>>         req.backend = xxx;
>>     } else if (..
>>
>> }
>> Also the next bit doesn't make sense:
>> if (req.request)
>>     {
>>     return(pass);
>>     }
>>
>> }
>>
>> On Fri, Aug 30, 2013 at 10:42 AM, Tharanga Abeyseela <
>> tharanga.abeyseela at gmail.com> wrote:
>> Hi,
>>
>> Actually i tried that link, still it is not working. it should work for a
>> ip based virtual host, but namebased virtual host is not working.
>> becuase main request coming to varnish/80 and varnish should redirect the
>> traffic to the backend server based on my round-robin director. when you
>> resolve dns for those two hosts (xxx.com and yyy.com) it reuturns the
>> same IP. that is 192.168.0.100. But there should be a way to send the
>> host-header to apache. Not sure how it works here. But when i use different
>> server (192,168.0.200/ zzz.com) it works without any issue, it
>> roundrobin the traffic. This is an issue with namebased virtualhost.
>> varnish recevie the traffic (cluster.com is my  varnish server), then it
>> use it vcl to redirect based on my round-robin setting. but how it can send
>> the traffic to the correct virtualhost. because it gets only the same IP
>> address from the DNS. I managed to change the default behaviour by
>> forcefully setting the host-header as follows to my other
>> virtualhost(yyy..com)
>> set.req.host="yyy.com" ;
>> then all request went to yyy.com instead of xxx.com. but what i want is
>> a round-robin fashio...to send traffic. first req to xxx.com/ 2nd req to
>> yyy.com and so on..
>>
>> I was trying to find some resourses on the net, stackoverflow etc..but no
>> luck yet.
>>
>> Thanks again for your help
>> Cheers,
>> Tharanga
>>
>> On Fri, Aug 30, 2013 at 5:05 PM, Per Buer <perbu at varnish-software.com>
>> wrote:
>> Hi Tharanga,
>>
>> You seem to be a bit confused about how the directors and backends work
>> in Varnish. Please read this:
>>
>>
>> https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish
>>
>> Note that the .host property of the backend has NOTHING to do with the
>> virtual host.
>>
>> On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <
>> tharanga.abeyseela at gmail.com> wrote:
>> Hi Per,
>> Thanks for the reply. Actually my set up looks like this.
>>
>> cluster.com - 192.168.0.200 (varnish/port 80)
>> xxx.com - 192.168.0.100  (apache,namebased vhost/8080 - backendname -
>> website)
>> yyy.com - 192.168.0.100  (apache,namebased vhost/8080 -backendname - api)
>> cluster.com is the varnish server and front-end connections coming to
>> this and rewrite to other defined back-ends (round-robin based balancing)
>>
>>
>>
>> backend website {
>>     .host = "xxx.com";
>>     .port = "8080";
>>
>> }
>> backend api {
>>     .host = "yyy.com";
>>     .port = "8080";
>>
>> }
>>
>> director clust round-robin {
>> { .backend = api;  }
>> { .backend = website;  }
>>
>> }
>>
>> sub vcl_recv {
>> set req.backend = clust;
>> if (req.request)
>>     {
>>     return(pass);
>>     }
>>
>> }
>>
>> when i hit the cluster.com , it is always going to xxx.com, but what i
>> need to do is first request go to xxx.com second request yyy.com and so
>> on...when i add another server (different host/different IP say
>> 192.168.0.111/zzz.com, and a different backend) , it goes like this
>> first request - xxx.com
>> second request - xxxx.com
>> third request - zzz.com
>>
>> but i can change the default behavior by setting up set req.host =
>> yyy.com and then it will goes to
>> first request - yyy.com
>> second request - yyy.com
>> third request - zzz.com
>> this is something to do with the host-header forwarding to the correct
>> back-end. how should i add that functionality to the vcl_recv ?
>> appreciate your help on this, this is working perfectly with other
>> servers (different servers, not with namebased vhosts)
>> cheers,
>> Tharanga
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com>
>> wrote:
>> Hi mate,
>>
>> On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <
>> tharanga.abeyseela at gmail.com> wrote:
>> (..)
>>
>> But how should i send host-headers to varnish to redirect to the correct
>> server, This is always going to xxx.com.
>>
>> You don't. :-)
>>
>> You need to dispatch the request to the right backend in vcl_recv and
>> have the backend handle that host. You could start rewriting the host in
>> vcl_recv, but I would advise against it.
>>
>> So, if you have two vhosts handled by "website" you just set that as the
>> backend in vcl_recv.
>>
>> --
>>
>> Per Buer
>> CTO | Varnish Software AS
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>> Winner of the Red Herring Top 100 Europe Award 2013
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Per Buer
>> CTO | Varnish Software AS
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>> Winner of the Red Herring Top 100 Europe Award 2013
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Per Buer
>> CTO | Varnish Software AS
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>> Winner of the Red Herring Top 100 Europe Award 2013
>>
>>
>>
>>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130831/7d7b5552/attachment.html>

From pug at felsing.net  Sat Aug 31 21:04:52 2013
From: pug at felsing.net (Christian Felsing)
Date: Sat, 31 Aug 2013 23:04:52 +0200
Subject: Rewrite With Varnish
Message-ID: <52225A74.5010308@felsing.net>

Hello,

is there a rewrite method in Varnish which has same effect like following rule in Apache 2.4?

RewriteRule ^/(.*)$ http://plone:8080/VirtualHostBase/https/plone.example.net:443/cms/VirtualHostRoot/$1
[L,P,E=remoteUser:%{LA-U:SSL_CLIENT_S_DN_CN}]

This rule reads CN from a client certificate and set remoteUser to CN.

best regards
Christian