From andrew at rocketnumbernine.com  Mon Jul  1 08:15:55 2013
From: andrew at rocketnumbernine.com (Andrew Smallbone)
Date: Mon, 1 Jul 2013 11:15:55 +0300
Subject: unexpected pass messages in log files
Message-ID: <CADpKiocWXcQO2Gs8dRciV2Fg-mLC1E3p3fcV=ssm8CTcsVmhww@mail.gmail.com>

I'm using varnish in a simple system, everything works fine when I
test using curl and web browsers and load testing systems, but when
the content is accessed by our iOS/unity front end, I'm getting "pass"
messages in the log files. ?Presumably the HTTP request is subtly
different (It's being logged as HTTP1.1, all requests both working and
not are coming through Amazon ELB - which is terminating the SSL).

Any hints if there's anything I can do with Varnish to see what the
issue is or am I missing something really basic?

varnished (varnish-3.0.3 revision 9e6a70f)
My default.vcl is below - only weirdness is if a specific header isn't
present the request is forwarded to another website, and that header
value is also used in the hash key

sub vcl_recv {
? if (req.http.x-forwarded-for) {
? ? set req.http.X-Forwarded-For =
? ? req.http.X-Forwarded-For + ", " + client.ip;
? } else {
? ? set req.http.X-Forwarded-For = client.ip;
? }
? # ignore AWS/ELB health messages
? if (req.url ~ ".*/health/.*") {
? ? return (pass);
? }
? // if no header error
? if (!req.http.xxx) {
? ? error 750 "Moved";
? }
}
sub vcl_error {
? if (obj.status == 750) {
? ? set obj.response = "Moved Temporarily";
? ? set obj.http.Location = "http://www.badrequest.com/";
? }
? set obj.status = 302;
? return (deliver);
}
# store results for each header code
sub vcl_hash {
? ? hash_data(req.url);
? ? hash_data(req.http.xxx);
? ? return (hash);
}
sub vcl_fetch {
? ? set beresp.ttl = 60s;
}
# remove http headers to confuse bad people
sub vcl_deliver {
? ?remove resp.http.X-Varnish;
? ?remove resp.http.Via;
? ?set resp.http.Server = "xxx";
}


regards



From andrew at rocketnumbernine.com  Mon Jul  1 08:57:53 2013
From: andrew at rocketnumbernine.com (Andrew Smallbone)
Date: Mon, 1 Jul 2013 11:57:53 +0300
Subject: unexpected pass messages in log files
In-Reply-To: <CADpKiocWXcQO2Gs8dRciV2Fg-mLC1E3p3fcV=ssm8CTcsVmhww@mail.gmail.com>
References: <CADpKiocWXcQO2Gs8dRciV2Fg-mLC1E3p3fcV=ssm8CTcsVmhww@mail.gmail.com>
Message-ID: <CADpKioc4LZKtx_kZ77V=TXDQY_kagDaGVxXgqq_YUz_D-bbowA@mail.gmail.com>

Should have looked harder myself - front end is adding an unused Cookie.
Adding "unset req.http.cookie;" to vcl_recv results in caching

regards
Andrew


On 1 July 2013 11:15, Andrew Smallbone <andrew at rocketnumbernine.com> wrote:
> I'm using varnish in a simple system, everything works fine when I
> test using curl and web browsers and load testing systems, but when
> the content is accessed by our iOS/unity front end, I'm getting "pass"
> messages in the log files.  Presumably the HTTP request is subtly
> different (It's being logged as HTTP1.1, all requests both working and
> not are coming through Amazon ELB - which is terminating the SSL).
>
> Any hints if there's anything I can do with Varnish to see what the
> issue is or am I missing something really basic?
>
> varnished (varnish-3.0.3 revision 9e6a70f)
> My default.vcl is below - only weirdness is if a specific header isn't
> present the request is forwarded to another website, and that header
> value is also used in the hash key
>
> sub vcl_recv {
>   if (req.http.x-forwarded-for) {
>     set req.http.X-Forwarded-For =
>     req.http.X-Forwarded-For + ", " + client.ip;
>   } else {
>     set req.http.X-Forwarded-For = client.ip;
>   }
>   # ignore AWS/ELB health messages
>   if (req.url ~ ".*/health/.*") {
>     return (pass);
>   }
>   // if no header error
>   if (!req.http.xxx) {
>     error 750 "Moved";
>   }
> }
> sub vcl_error {
>   if (obj.status == 750) {
>     set obj.response = "Moved Temporarily";
>     set obj.http.Location = "http://www.badrequest.com/";
>   }
>   set obj.status = 302;
>   return (deliver);
> }
> # store results for each header code
> sub vcl_hash {
>     hash_data(req.url);
>     hash_data(req.http.xxx);
>     return (hash);
> }
> sub vcl_fetch {
>     set beresp.ttl = 60s;
> }
> # remove http headers to confuse bad people
> sub vcl_deliver {
>    remove resp.http.X-Varnish;
>    remove resp.http.Via;
>    set resp.http.Server = "xxx";
> }
>
>
> regards



From hans at moc.net  Tue Jul  2 11:45:28 2013
From: hans at moc.net (Hans Schou)
Date: Tue, 02 Jul 2013 13:45:28 +0200
Subject: #1320 Varnish returns a 503 when being given a 302 by the webapp
Message-ID: <51D2BD58.2030405@moc.net>

Hi

https://www.varnish-cache.org/trac/ticket/1320

The error occur when requesting a sub-page in Typo3 like 
http://example.org/foo and it is being redirected to 
http://example.org/foo/ by typo3 backend. Then Content-Encoding is set 
to "gzip" and "Content-Length" is 0.

I got this error with varnish 3.0.4 and solved it with this (if it could 
be to any help).

if (beresp.http.Content-Encoding ~ "gzip" ) {
   if (beresp.http.Content-Length == "0") {
        unset beresp.http.Content-Encoding;
   }
}

...it might be possible to write it more "correct".

-- 
Venlig hilsen
Hans Schou
tel:46923438




From daghf at varnish-software.com  Tue Jul  2 12:30:37 2013
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 2 Jul 2013 14:30:37 +0200
Subject: #1320 Varnish returns a 503 when being given a 302 by the webapp
In-Reply-To: <51D2BD58.2030405@moc.net>
References: <51D2BD58.2030405@moc.net>
Message-ID: <CAEh05VaUxb4ZohnAbL73e8+dRzspdacWB08Vup1=+M8fcRbdJg@mail.gmail.com>

Hi Hans - I added a reference to this workaround in the bug report. Thanks

On Tue, Jul 2, 2013 at 1:45 PM, Hans Schou <hans at moc.net> wrote:
> Hi
>
> https://www.varnish-cache.org/trac/ticket/1320
>
> The error occur when requesting a sub-page in Typo3 like
> http://example.org/foo and it is being redirected to http://example.org/foo/
> by typo3 backend. Then Content-Encoding is set to "gzip" and
> "Content-Length" is 0.
>
> I got this error with varnish 3.0.4 and solved it with this (if it could be
> to any help).
>
> if (beresp.http.Content-Encoding ~ "gzip" ) {
>   if (beresp.http.Content-Length == "0") {
>        unset beresp.http.Content-Encoding;
>   }
> }
>
> ...it might be possible to write it more "correct".
>
> --
> Venlig hilsen
> Hans Schou
> tel:46923438
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



-- 
Dag Haavi Finstad
Software Developer | Varnish Software AS
Mobile: +47 476 64 134 | Office: +47 21 98 92 60
We Make Websites Fly!



From hans at moc.net  Wed Jul  3 07:09:29 2013
From: hans at moc.net (Hans Schou)
Date: Wed, 03 Jul 2013 09:09:29 +0200
Subject: Swap exceed & Child not responding to CLI, killing it
In-Reply-To: <0423F797BEB04005A87E7E5669927081@MezhRoze>
References: <51C2AD5C.5010005@moc.net>
 <0423F797BEB04005A87E7E5669927081@MezhRoze>
Message-ID: <51D3CE29.7030501@moc.net>

Den 06/20/2013 04:51 PM, Reinis Rozitis skrev:
>> Any hint or help much appreciated.
>
> Try setting sysctl vm.swappiness=0
Well, after trying that a couple of days, and upgrading to 3.0.4, I 
think I have hit a memory leak. Varnish keeps up eating mem and when 
there is no more mem I got a restart. Last one was:

Child (6477) died signal=6
Child (6477) Panic message: Assert error in VGZ_NewGzip(), cache_gzip.c 
line 209:#012  Condition(Z_OK == i) not true.#012errno = 12 (Cannot 
allocate memory)...

Is there anything in 'varnishstat -1' which can give a clue?

client_conn             626603         9.22 Client connections accepted
client_drop                  0         0.00 Connection dropped, no sess/wrk
client_req             2501923        36.83 Client requests received
cache_hit              2135392        31.43 Cache hits
cache_hitpass             2486         0.04 Cache hits for pass
cache_miss              347931         5.12 Cache misses
backend_conn             53645         0.79 Backend conn. success
backend_unhealthy            0         0.00 Backend conn. not attempted
backend_busy                 0         0.00 Backend conn. too many
backend_fail                 1         0.00 Backend conn. failures
backend_reuse           319326         4.70 Backend conn. reuses
backend_toolate          17374         0.26 Backend conn. was closed
backend_recycle         336723         4.96 Backend conn. recycles
backend_retry                5         0.00 Backend conn. retry
fetch_head                 535         0.01 Fetch head
fetch_length            357385         5.26 Fetch with Length
fetch_chunked            11490         0.17 Fetch chunked
fetch_eof                    0         0.00 Fetch EOF
fetch_bad                    0         0.00 Fetch had bad headers
fetch_close                702         0.01 Fetch wanted close
fetch_oldhttp                0         0.00 Fetch pre HTTP/1.1 closed
fetch_zero                   0         0.00 Fetch zero len
fetch_failed                 0         0.00 Fetch failed
fetch_1xx                    0         0.00 Fetch no body (1xx)
fetch_204                    0         0.00 Fetch no body (204)
fetch_304                 1243         0.02 Fetch no body (304)
n_sess_mem                 207          .   N struct sess_mem
n_sess                      70          .   N struct sess
n_object                252984          .   N struct object
n_vampireobject              0          .   N unresurrected objects
n_objectcore            253053          .   N struct objectcore
n_objecthead             58090          .   N struct objecthead
n_waitinglist               94          .   N struct waitinglist
n_vbc                       19          .   N struct vbc
n_wrk                      400          .   N worker threads
n_wrk_create               400         0.01 N worker threads created
n_wrk_failed                 0         0.00 N worker threads not created
n_wrk_max                    0         0.00 N worker threads limited
n_wrk_lqueue                 0         0.00 work request queue length
n_wrk_queued                 6         0.00 N queued work requests
n_wrk_drop                   0         0.00 N dropped work requests
n_backend                   17          .   N backends
n_expired                94945          .   N expired objects
n_lru_nuked                  0          .   N LRU nuked objects
n_lru_moved            1415668          .   N LRU moved objects
losthdr                      1         0.00 HTTP header overflows
n_objsendfile                0         0.00 Objects sent with sendfile
n_objwrite             2436792        35.87 Objects sent with write
n_objoverflow                0         0.00 Objects overflowing workspace
s_sess                  626604         9.22 Total Sessions
s_req                  2501923        36.83 Total Requests
s_pipe                    1610         0.02 Total pipe
s_pass                   23428         0.34 Total pass
s_fetch                 371355         5.47 Total fetch
s_hdrbytes           907219967     13354.04 Total header bytes
s_bodybytes        21866125041    321863.59 Total body bytes
sess_closed             107908         1.59 Session Closed
sess_pipeline             4939         0.07 Session Pipeline
sess_readahead            1677         0.02 Session Read Ahead
sess_linger            2426224        35.71 Session Linger
sess_herd              1424318        20.97 Session herd
shm_records          118481149      1744.01 SHM records
shm_writes             6661958        98.06 SHM writes
shm_flushes                  6         0.00 SHM flushes due to overflow
shm_cont                  6534         0.10 SHM MTX contention
shm_cycles                  53         0.00 SHM cycles through buffer
sms_nreq                   302         0.00 SMS allocator requests
sms_nobj                     0          .   SMS outstanding allocations
sms_nbytes                   0          .   SMS outstanding bytes
sms_balloc              401612          .   SMS bytes allocated
sms_bfree               401612          .   SMS bytes freed
backend_req             371356         5.47 Backend requests made
n_vcl                        1         0.00 N vcl total
n_vcl_avail                  1         0.00 N vcl available
n_vcl_discard                0         0.00 N vcl discarded
n_ban                      298          .   N total active bans
n_ban_gone                 277          .   N total gone bans
n_ban_add                  298         0.00 N new bans added
n_ban_retire                 0         0.00 N old bans deleted
n_ban_obj_test           32976         0.49 N objects tested
n_ban_re_test           290209         4.27 N regexps tested against
n_ban_dups                 276         0.00 N duplicate bans removed
hcb_nolock             2485461        36.59 HCB Lookups without lock
hcb_lock                139934         2.06 HCB Lookups with lock
hcb_insert              139934         2.06 HCB Inserts
esi_errors                   0         0.00 ESI parse errors (unlock)
esi_warnings                 0         0.00 ESI parse warnings (unlock)
accept_fail                  0         0.00 Accept failures
client_drop_late             0         0.00 Connection dropped late
uptime                   67936         1.00 Client uptime
dir_dns_lookups              0         0.00 DNS director lookups
dir_dns_failed               0         0.00 DNS director failed lookups
dir_dns_hit                  0         0.00 DNS director cached lookups hit
dir_dns_cache_full           0         0.00 DNS director full dnscache
vmods                        0          .   Loaded VMODs
n_gzip                  131411         1.93 Gzip operations
n_gunzip                295578         4.35 Gunzip operations
sess_pipe_overflow           0          .   Dropped sessions due to 
session pipe overflow
LCK.sms.creat                1         0.00 Created locks
LCK.sms.destroy              0         0.00 Destroyed locks
LCK.sms.locks              906         0.01 Lock Operations
LCK.sms.colls                0         0.00 Collisions
LCK.smp.creat                0         0.00 Created locks
LCK.smp.destroy              0         0.00 Destroyed locks
LCK.smp.locks                0         0.00 Lock Operations
LCK.smp.colls                0         0.00 Collisions
LCK.sma.creat                1         0.00 Created locks
LCK.sma.destroy              0         0.00 Destroyed locks
LCK.sma.locks           994791        14.64 Lock Operations
LCK.sma.colls                0         0.00 Collisions
LCK.smf.creat                1         0.00 Created locks
LCK.smf.destroy              0         0.00 Destroyed locks
LCK.smf.locks           341171         5.02 Lock Operations
LCK.smf.colls                0         0.00 Collisions
LCK.hsl.creat                0         0.00 Created locks
LCK.hsl.destroy              0         0.00 Destroyed locks
LCK.hsl.locks                0         0.00 Lock Operations
LCK.hsl.colls                0         0.00 Collisions
LCK.hcb.creat                1         0.00 Created locks
LCK.hcb.destroy              0         0.00 Destroyed locks
LCK.hcb.locks           222504         3.28 Lock Operations
LCK.hcb.colls                0         0.00 Collisions
LCK.hcl.creat                0         0.00 Created locks
LCK.hcl.destroy              0         0.00 Destroyed locks
LCK.hcl.locks                0         0.00 Lock Operations
LCK.hcl.colls                0         0.00 Collisions
LCK.vcl.creat                1         0.00 Created locks
LCK.vcl.destroy              0         0.00 Destroyed locks
LCK.vcl.locks             3172         0.05 Lock Operations
LCK.vcl.colls                0         0.00 Collisions
LCK.stat.creat               1         0.00 Created locks
LCK.stat.destroy             0         0.00 Destroyed locks
LCK.stat.locks          626744         9.23 Lock Operations
LCK.stat.colls               0         0.00 Collisions
LCK.sessmem.creat            1         0.00 Created locks
LCK.sessmem.destroy            0         0.00 Destroyed locks
LCK.sessmem.locks         631760         9.30 Lock Operations
LCK.sessmem.colls              0         0.00 Collisions
LCK.wstat.creat                1         0.00 Created locks
LCK.wstat.destroy              0         0.00 Destroyed locks
LCK.wstat.locks           168421         2.48 Lock Operations
LCK.wstat.colls                0         0.00 Collisions
LCK.herder.creat               1         0.00 Created locks
LCK.herder.destroy             0         0.00 Destroyed locks
LCK.herder.locks               1         0.00 Lock Operations
LCK.herder.colls               0         0.00 Collisions
LCK.wq.creat                   2         0.00 Created locks
LCK.wq.destroy                 0         0.00 Destroyed locks
LCK.wq.locks             3206044        47.19 Lock Operations
LCK.wq.colls                   0         0.00 Collisions
LCK.objhdr.creat          140007         2.06 Created locks
LCK.objhdr.destroy         81917         1.21 Destroyed locks
LCK.objhdr.locks        10343620       152.26 Lock Operations
LCK.objhdr.colls               0         0.00 Collisions
LCK.exp.creat                  1         0.00 Created locks
LCK.exp.destroy                0         0.00 Destroyed locks
LCK.exp.locks             517536         7.62 Lock Operations
LCK.exp.colls                  0         0.00 Collisions
LCK.lru.creat                  2         0.00 Created locks
LCK.lru.destroy                0         0.00 Destroyed locks
LCK.lru.locks             354670         5.22 Lock Operations
LCK.lru.colls                  0         0.00 Collisions
LCK.cli.creat                  1         0.00 Created locks
LCK.cli.destroy                0         0.00 Destroyed locks
LCK.cli.locks              22642         0.33 Lock Operations
LCK.cli.colls                  0         0.00 Collisions
LCK.ban.creat                  1         0.00 Created locks
LCK.ban.destroy                0         0.00 Destroyed locks
LCK.ban.locks             544373         8.01 Lock Operations
LCK.ban.colls                  0         0.00 Collisions
LCK.vbp.creat                  1         0.00 Created locks
LCK.vbp.destroy                0         0.00 Destroyed locks
LCK.vbp.locks              13583         0.20 Lock Operations
LCK.vbp.colls                  0         0.00 Collisions
LCK.vbe.creat                  1         0.00 Created locks
LCK.vbe.destroy                0         0.00 Destroyed locks
LCK.vbe.locks             107279         1.58 Lock Operations
LCK.vbe.colls                  0         0.00 Collisions
LCK.backend.creat             17         0.00 Created locks
LCK.backend.destroy            0         0.00 Destroyed locks
LCK.backend.locks         834359        12.28 Lock Operations
LCK.backend.colls              0         0.00 Collisions
SMF.s0.c_req              194123         2.86 Allocator requests
SMF.s0.c_fail                  0         0.00 Allocator failures
SMF.s0.c_bytes        4399304704     64756.60 Bytes allocated
SMF.s0.c_freed        3393925120     49957.68 Bytes freed
SMF.s0.g_alloc             58453          .   Allocations outstanding
SMF.s0.g_bytes        1005379584          .   Bytes outstanding
SMF.s0.g_space       63419129856          .   Bytes available
SMF.s0.g_smf               58533          .   N struct smf
SMF.s0.g_smf_frag              3          .   N small free smf
SMF.s0.g_smf_large            77          .   N large free smf
SMA.Transient.c_req       550339         8.10 Allocator requests
SMA.Transient.c_fail           0         0.00 Allocator failures
SMA.Transient.c_bytes  40394887294    594602.09 Bytes allocated
SMA.Transient.c_freed  30467874668    448479.08 Bytes freed
SMA.Transient.g_alloc       340120          .   Allocations outstanding
SMA.Transient.g_bytes   9927012626          .   Bytes outstanding
SMA.Transient.g_space            0          .   Bytes available
VBE.web15(192.168.4.15,,80).vcls            1          .   VCL references
VBE.web15(192.168.4.15,,80).happy           0          .   Happy health 
probes
VBE.web25(192.168.4.25,,80).vcls              1          .   VCL references
VBE.web25(192.168.4.25,,80).happy             0          .   Happy 
health probes
VBE.web78(192.168.4.78,,80).vcls            1          .   VCL references
VBE.web78(192.168.4.78,,80).happy           0          .   Happy health 
probes
VBE.web78(192.168.4.78,,8081).vcls            1          .   VCL references
VBE.web78(192.168.4.78,,8081).happy           0          .   Happy 
health probes
VBE.web32(192.168.4.32,,80).vcls                    1          . VCL 
references
VBE.web32(192.168.4.32,,80).happy                   0          . Happy 
health probes
VBE.web53(192.168.4.53,,80).vcls                      1          . VCL 
references
VBE.web53(192.168.4.53,,80).happy                     0          . Happy 
health probes
VBE.web48(192.168.4.48,,80).vcls                    1          . VCL 
references
VBE.web48(192.168.4.48,,80).happy                   0          . Happy 
health probes
VBE.web42(192.168.4.42,,80).vcls                      1          . VCL 
references
VBE.web42(192.168.4.42,,80).happy                     0          . Happy 
health probes
VBE.web43(192.168.4.43,,80).vcls                      1          . VCL 
references
VBE.web43(192.168.4.43,,80).happy                     0          . Happy 
health probes
VBE.web45(192.168.4.45,,80).vcls                     1          . VCL 
references
VBE.web45(192.168.4.45,,80).happy                    0          . Happy 
health probes
VBE.web21(192.168.4.21,,80).vcls                     1          . VCL 
references
VBE.web21(192.168.4.21,,80).happy                    0          . Happy 
health probes
VBE.web62(192.168.4.62,,80).vcls                   1          . VCL 
references
VBE.web62(192.168.4.62,,80).happy 18446744073709551615          .   
Happy health probes
VBE.web56(192.168.4.56,,80).vcls                     1          . VCL 
references
VBE.web56(192.168.4.56,,80).happy                    0          . Happy 
health probes
VBE.web30(192.168.4.30,,80).vcls                    1          . VCL 
references
VBE.web30(192.168.4.30,,80).happy                   0          . Happy 
health probes
VBE.web55(192.168.4.55,,80).vcls                     1          . VCL 
references
VBE.web55(192.168.4.55,,80).happy                    0          . Happy 
health probes
VBE.web75(192.168.4.75,,80).vcls                    1          . VCL 
references
VBE.web75(192.168.4.75,,80).happy                   0          . Happy 
health probes
VBE.web88(192.168.4.88,,80).vcls                   1          . VCL 
references
VBE.web88(192.168.4.88,,80).happy                  0          . Happy 
health probes


-- 
Venlig hilsen
Hans Schou
tel:46923438




From apj at mutt.dk  Wed Jul  3 07:59:04 2013
From: apj at mutt.dk (Andreas Plesner Jacobsen)
Date: Wed, 3 Jul 2013 09:59:04 +0200
Subject: Swap exceed & Child not responding to CLI, killing it
In-Reply-To: <51D3CE29.7030501@moc.net>
References: <51C2AD5C.5010005@moc.net>
 <0423F797BEB04005A87E7E5669927081@MezhRoze>
 <51D3CE29.7030501@moc.net>
Message-ID: <20130703075904.GR19694@nerd.dk>

On Wed, Jul 03, 2013 at 09:09:29AM +0200, Hans Schou wrote:

> SMA.Transient.c_req       550339         8.10 Allocator requests
> SMA.Transient.c_fail           0         0.00 Allocator failures
> SMA.Transient.c_bytes  40394887294    594602.09 Bytes allocated
> SMA.Transient.c_freed  30467874668    448479.08 Bytes freed
> SMA.Transient.g_alloc       340120          .   Allocations outstanding
> SMA.Transient.g_bytes   9927012626          .   Bytes outstanding
> SMA.Transient.g_space            0          .   Bytes available

You're using transient storage, which is unbounded. Do you use very low ttl?

-- 
Andreas



From hans at moc.net  Wed Jul  3 11:29:23 2013
From: hans at moc.net (Hans Schou)
Date: Wed, 03 Jul 2013 13:29:23 +0200
Subject: Swap exceed & Child not responding to CLI, killing it
In-Reply-To: <20130703075904.GR19694@nerd.dk>
References: <51C2AD5C.5010005@moc.net>
 <0423F797BEB04005A87E7E5669927081@MezhRoze> <51D3CE29.7030501@moc.net>
 <20130703075904.GR19694@nerd.dk>
Message-ID: <51D40B13.4090300@moc.net>

Den 07/03/2013 09:59 AM, Andreas Plesner Jacobsen skrev:
> On Wed, Jul 03, 2013 at 09:09:29AM +0200, Hans Schou wrote:
>
>> SMA.Transient.c_req       550339         8.10 Allocator requests
>> SMA.Transient.c_fail           0         0.00 Allocator failures
>> SMA.Transient.c_bytes  40394887294    594602.09 Bytes allocated
>> SMA.Transient.c_freed  30467874668    448479.08 Bytes freed
>> SMA.Transient.g_alloc       340120          .   Allocations outstanding
>> SMA.Transient.g_bytes   9927012626          .   Bytes outstanding
>> SMA.Transient.g_space            0          .   Bytes available
> You're using transient storage, which is unbounded. Do you use very low ttl?
>
Thanks for pointing that out about transient.

Our default ttl is 0 seconds and is controlled by the backend (Typo3). 
Typically pages has a ttl in 10 minutes. Images and the like are always 
with ttl 1 hour.

I did not expected the non-cachable to go into transient. Is that what 
happens?

I wonder if it could help to add and specify transient like this:

  # egrep "^[^#]*-s" /etc/default/varnish
  -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,50G \
  -s Transient=malloc,2G \


-- 
Venlig hilsen
Hans Schou
tel:46923438




From dridi.boukelmoune at zenika.com  Wed Jul  3 11:50:24 2013
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Wed, 3 Jul 2013 13:50:24 +0200
Subject: Passing cookies to ESI includes
Message-ID: <CABtDKm4JQgdAkHQcNGfC07T1NFX14nt0tRKFm1SUv1me67z58w@mail.gmail.com>

Hi all,

I hope someone can help me with this ESI problem I have with one of my clients.

One of the applications here use Varnish in front of two backends, and
we see a lot of session garbage allocated because of ESI not passing
cookies back-and-forth.

The two backends in a nutshell:
- front office (drupal cms)
- back office (java webapp, connected to other services of the company)

For the sake of simplicity, assume the front and back offices send
respectively PHPSESSID and JSESSIONID cookies.

The setup is also twisted, since you can actually access to both front
and back offices (varnish does the routing) and each can esi:include
fragments from the other. I haven't got through the 5 whys with the
team yet...

The worst case scenario:

- Access to the front office without any cookie
=> drupal adds a set-cookie header for PHPSESSID
=> varnish finds N esi:include tags
==> the back office creates N different JSESSIONID set-cookie headers
(one per request)

It means N wasted sessions on the back office since none of the
cookies will be retrieved by the user agent. And this will happen
until you actually hit a page directly on the back office (with a
shiny JSESSIONID).

The ideal solution:

- Access to the front office without any cookie
=> drupal adds a set-cookie header for PHPSESSID
=> varnish finds N esi:include tags
==> varnish fires the first esi request
===> the backoffice adds a set-cookie header for JSESSIONID
===> varnish adds the JSESSIONID cookie to the existing set-cookie header
===> varnish adds the JSESSIONID cookie header to the next requests
==> varnish fires the other N-1 esi requests with the JSESSIONID cookie
===> the back office doesn't create additional sessions
==> varnish answers with both PHPSESSID and JSESSIONID cookies
=> champagne \o/

I'm open to any kind of solution, including (no pun intended) inline C
or open source vmods. Any help appreciated :)

One more thing:
I'm aware of libvmod-header, which solves the (implicit) multiple
Set-Cookie headers issue, but doesn't help cross the ESI boundaries.

Best Regards,
Dridi



From apj at mutt.dk  Wed Jul  3 13:03:19 2013
From: apj at mutt.dk (Andreas Plesner Jacobsen)
Date: Wed, 3 Jul 2013 15:03:19 +0200
Subject: Swap exceed & Child not responding to CLI, killing it
In-Reply-To: <51D40B13.4090300@moc.net>
References: <51C2AD5C.5010005@moc.net>
 <0423F797BEB04005A87E7E5669927081@MezhRoze>
 <51D3CE29.7030501@moc.net> <20130703075904.GR19694@nerd.dk>
 <51D40B13.4090300@moc.net>
Message-ID: <20130703130319.GS19694@nerd.dk>

On Wed, Jul 03, 2013 at 01:29:23PM +0200, Hans Schou wrote:
> 
> Our default ttl is 0 seconds and is controlled by the backend
> (Typo3). Typically pages has a ttl in 10 minutes. Images and the
> like are always with ttl 1 hour.
> 
> I did not expected the non-cachable to go into transient. Is that
> what happens?

Transient will be used in three cases:

1) Any content with a TTL lower than the "shortlived" parameter, which defaults
to 10 secs.
2) Any content that we can't make room in the storage backend (usually due to a
too low "nuke_limit"). If this happens the TTL should be set to "shortlived",
but that was broken in pre-3.0.3 (bug #1140).
3) I believe your 0 ttl content would hit transient, and probably stay there
for $beresp.grace seconds

So you're probably hitting 2. Try upping the nuke_limit.
It may also be 3 if your grace is high.

> I wonder if it could help to add and specify transient like this:

That would probably just end up 503ing, since Transient would get filled, and
due to the low nuke_limit, we'd be unable to free space there, and then there
would be nowhere to go for new objects.

-- 
Andreas



From gabster at lelutin.ca  Thu Jul  4 01:55:39 2013
From: gabster at lelutin.ca (Gabriel Filion)
Date: Wed, 03 Jul 2013 21:55:39 -0400
Subject: Monitoring cache file usage with varnish 3
Message-ID: <51D4D61B.1060304@lelutin.ca>

Hi there,

I've recently upgraded a cluster's caching nodes to varnish 3, and saw
that our graphs for the cache file usage were now giving no info anymore.

from what I could see, that's because the value "sm_balloc" is just not
there anymore in the output of varnishstat -1. unfortunately, I couldn't
find a replacement for it for disk-based cache files.

Is there a way to monitor usage and free space of the cache file with
varnish 3?

-- 
Gabriel Filion

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 291 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130703/bfa43645/attachment.bin>

From sambati at corp.untd.com  Thu Jul  4 05:43:53 2013
From: sambati at corp.untd.com (Ambati, Sugunakar)
Date: Thu, 4 Jul 2013 05:43:53 +0000
Subject: reg: varnish config
Message-ID: <31D094CE71D4194AAE5CFFE9426DB9345039EC21@HYDMBX01.hyd.corp.int.untd.com>

Hi,

Good Morning.

We have several mobile clients and for few mobile users, we wanted to serve from server instead of cache.

There are 30 such agents and I would like to automate the way so that every time a new mobile comes into market, we don't have to do anything manual.

Could someone suggest on this?

Thanks
Sugunakar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130704/001dbbdc/attachment.html>

From hans at moc.net  Thu Jul  4 06:26:19 2013
From: hans at moc.net (Hans Schou)
Date: Thu, 04 Jul 2013 08:26:19 +0200
Subject: Swap exceed & Child not responding to CLI, killing it
In-Reply-To: <20130703130319.GS19694@nerd.dk>
References: <51C2AD5C.5010005@moc.net>
 <0423F797BEB04005A87E7E5669927081@MezhRoze> <51D3CE29.7030501@moc.net>
 <20130703075904.GR19694@nerd.dk> <51D40B13.4090300@moc.net>
 <20130703130319.GS19694@nerd.dk>
Message-ID: <51D5158B.9070908@moc.net>

Den 07/03/2013 03:03 PM, Andreas Plesner Jacobsen skrev:

> 3) I believe your 0 ttl content would hit transient, and probably stay there
> for $beresp.grace seconds

> It may also be 3 if your grace is high.

Well, my was for some reason set to beresp.grace = 24h. It worked well 
with varnish v2.

If have now set it to 1h. I will se in a few hours if the problem is solved.

>
>> I wonder if it could help to add and specify transient like this:
> That would probably just end up 503ing,
OK, I'll skip that.


-- 
Venlig hilsen
Hans Schou
tel:46923438




From perbu at varnish-software.com  Thu Jul  4 07:51:54 2013
From: perbu at varnish-software.com (Per Buer)
Date: Thu, 4 Jul 2013 09:51:54 +0200
Subject: reg: varnish config
In-Reply-To: <31D094CE71D4194AAE5CFFE9426DB9345039EC21@HYDMBX01.hyd.corp.int.untd.com>
References: <31D094CE71D4194AAE5CFFE9426DB9345039EC21@HYDMBX01.hyd.corp.int.untd.com>
Message-ID: <CAOXZevC2O6yarJnOBDjPhAkk6eBgsGp0qnA=yHoHJCtOrKNi-w@mail.gmail.com>

Hi,

On Thu, Jul 4, 2013 at 7:43 AM, Ambati, Sugunakar <sambati at corp.untd.com>wrote:

>  Hi,****
>
> ** **
>
> Good Morning.****
>
> ** **
>
> We have several mobile clients and for few mobile users, we wanted to
> serve from server instead of cache.****
>
> ** **
>
> There are 30 such agents and I would like to automate the way so that
> every time a new mobile comes into market, we don?t have to do anything
> manual.****
>
> ** **
>
> Could someone suggest on this?
>

There are four different ways to do this. The simple is the devicedetect
VCL, available at Github. This uses regular expressions to match.

Then there are three different VMODs that handle this. dClass is open
source and uses the Apache OpenDDR to look up devices. Then there are two
proprietary VMODs as well.
Varnish Software has written a VMOD that incorporates the DeviceAtlas
library from Dotmobi (
https://www.varnish-cache.org/vmod/deviceatlas-mobile-detection). Scientia
Mobile has also written a VMOD that uses their now proprietary WURFL
library.

Cheers,

Per.
-- 
 <http://www.varnish-software.com/> *Per Buer*
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130704/4a7050e1/attachment.html>

From celene at sandydproductions.com  Sat Jul  6 19:22:15 2013
From: celene at sandydproductions.com (Celene)
Date: Sat, 06 Jul 2013 12:22:15 -0700
Subject: varnishstat not reporting correct values
Message-ID: <51D86E67.3080300@sandydproductions.com>

Hi, I have been using varnish to accelerate some of my sites.

Interestingly enough, I added a rule in the VCL to show hits in the
header (below)
-------------
if (obj.hits > 0) {
                set resp.http.X-Cache = "HIT";
        } else {
                set resp.http.X-Cache = "MISS";
        }
-------------

The headers show up as a hit in my browser, with a cleared cache.

However, the stats in varnishstat never change, even if I put load on
varnish using loadimpact.

Anyone have any idea where I should check first?

Thanks,

Celene



From james at ifixit.com  Mon Jul  8 22:20:18 2013
From: james at ifixit.com (James Pearson)
Date: Mon, 08 Jul 2013 15:20:18 -0700
Subject: varnishstat not reporting correct values
In-Reply-To: <51D86E67.3080300@sandydproductions.com>
References: <51D86E67.3080300@sandydproductions.com>
Message-ID: <1373321948-sup-5230@geror.local>

Excerpts from Celene's message of 2013-07-06 12:22:15 -0700:
> Hi, I have been using varnish to accelerate some of my sites.
> 
> Interestingly enough, I added a rule in the VCL to show hits in the
> header (below)
> -------------
> if (obj.hits > 0) {
>                 set resp.http.X-Cache = "HIT";
>         } else {
>                 set resp.http.X-Cache = "MISS";
>         }
> -------------
> 
> The headers show up as a hit in my browser, with a cleared cache.
> 
> However, the stats in varnishstat never change, even if I put load on
> varnish using loadimpact.
> 
> Anyone have any idea where I should check first?

Do *none* of the stats in varnishstat change, or just the hitrate?

How about the other tools - varnishhist, varnishlog, etc.?

 - P



From hans at moc.net  Wed Jul 10 12:20:46 2013
From: hans at moc.net (Hans Schou)
Date: Wed, 10 Jul 2013 14:20:46 +0200
Subject: Swap exceed & Child not responding to CLI, killing it
In-Reply-To: <51D5158B.9070908@moc.net>
References: <51C2AD5C.5010005@moc.net>
 <0423F797BEB04005A87E7E5669927081@MezhRoze> <51D3CE29.7030501@moc.net>
 <20130703075904.GR19694@nerd.dk> <51D40B13.4090300@moc.net>
 <20130703130319.GS19694@nerd.dk> <51D5158B.9070908@moc.net>
Message-ID: <51DD519E.6030409@moc.net>

Den 07/04/2013 08:26 AM, Hans Schou skrev:
> Well, my was for some reason set to beresp.grace = 24h. It worked well 
> with varnish v2.
>
> If have now set it to 1h. I will se in a few hours if the problem is 
> solved. 
Just to close this, setting beresp.grace to 1h solved the problem.

Thanks for the hint.

Lesson learned: Extreme misconfiguration can end with a crash.

-- 
Venlig hilsen
Hans Schou
tel:46923438




From gabster at lelutin.ca  Wed Jul 10 22:01:48 2013
From: gabster at lelutin.ca (Gabriel Filion)
Date: Wed, 10 Jul 2013 18:01:48 -0400
Subject: Monitoring cache file usage with varnish 3
In-Reply-To: <51D4D61B.1060304@lelutin.ca>
References: <51D4D61B.1060304@lelutin.ca>
Message-ID: <51DDD9CC.1080807@lelutin.ca>

Hello,

bumping this to see if anyone can help.

any idea what I can use instead of sm_balloc for monitoring cache file?

if my message is confusing, I can point to more detailed information.

On 03/07/13 09:55 PM, Gabriel Filion wrote:
> I've recently upgraded a cluster's caching nodes to varnish 3, and saw
> that our graphs for the cache file usage were now giving no info anymore.
> 
> from what I could see, that's because the value "sm_balloc" is just not
> there anymore in the output of varnishstat -1. unfortunately, I couldn't
> find a replacement for it for disk-based cache files.
> 
> Is there a way to monitor usage and free space of the cache file with
> varnish 3?


-- 
Gabriel Filion

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 291 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130710/a5424018/attachment.bin>

From gabster at lelutin.ca  Thu Jul 11 02:40:30 2013
From: gabster at lelutin.ca (Gabriel Filion)
Date: Wed, 10 Jul 2013 22:40:30 -0400
Subject: [225-19709AAD-074D] Re: Monitoring cache file usage with varnish 3
In-Reply-To: <8004562df6da4beb9cd165224e7bb608@redehost.com.br>
References: <8004562df6da4beb9cd165224e7bb608@redehost.com.br>
Message-ID: <51DE1B1E.6030906@lelutin.ca>

Hi there, [bringing this back to the list since it doesn't seem like
your message was purposefully aimed at just me]

On 10/07/13 06:33 PM, Suporte Cloud - RedeHost wrote:
> Found in research on the internet the following link, I believe it has
> enough information about Varnish and its tools.
> 
> http://www.devin.com.br/varnish-logs/

ah, thanks. I think I was able to make out enough parts of the
portuguese to understand :)

> Hope this helps, if you have more questions or have understood your
> question earlier bad, ask you to return your difficulty explaining in
> more detail.

hmm so according to this page, the values:

SMF.s0.g_bytes and SMF.s0.g_space should represent the cache usage /
total, have I read this out correctly?

If I calculate with what I have:

SMF.s0.g_bytes         256258048          .   Bytes outstanding
SMF.s0.g_space         817483776          .   Bytes available

it gives roughly 245Mb used over roughly 780Mb total for the cache.

but in /etc/default/varnish I have the following as an argument to the
daemon:

INSTANCE=$(uname -n)
DAEMON_OPTS="#[...irrelevant] \
             -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"

# ls -l /var/lib/varnish/cache01/varnish_storage.bin
-rw------- 1 root root 1073741824 Jul 10 22:35
/var/lib/varnish/cache01/varnish_storage.bin

so I'm wondering whether the cache file is not used entirely or else the
values don't correspond to the cache file's usage and total size..

> ------------------------------------------------------------------------
> *De*: Gabriel Filion <gabster at lelutin.ca>
> *Enviado*: qua, 10 jul 2013 19:20:30 -0300
> *Para*: "varnish-misc at varnish-cache.org" <varnish-misc at varnish-cache.org>
> *Assunto*: Re: Monitoring cache file usage with varnish 3
> 
> Hello,
> 
> bumping this to see if anyone can help.
> 
> any idea what I can use instead of sm_balloc for monitoring cache file?
> 
> if my message is confusing, I can point to more detailed information.
> 
> On 03/07/13 09:55 PM, Gabriel Filion wrote:
>> I've recently upgraded a cluster's caching nodes to varnish 3, and saw
>> that our graphs for the cache file usage were now giving no info anymore.
>>
>> from what I could see, that's because the value "sm_balloc" is just not
>> there anymore in the output of varnishstat -1. unfortunately, I couldn't
>> find a replacement for it for disk-based cache files.
>>
>> Is there a way to monitor usage and free space of the cache file with
>> varnish 3?
> 
> --
> Gabriel Filion
> 


-- 
Gabriel Filion

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 291 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130710/a0bb9ee3/attachment.bin>

From mattias at nucleus.be  Thu Jul 11 07:06:17 2013
From: mattias at nucleus.be (Mattias Geniar)
Date: Thu, 11 Jul 2013 07:06:17 +0000
Subject: Monitoring cache file usage with varnish 3
In-Reply-To: <51DDD9CC.1080807@lelutin.ca>
Message-ID: <CE0425D2.6C431%mattias@nucleus.be>

>
>any idea what I can use instead of sm_balloc for monitoring cache file?
>
>if my message is confusing, I can point to more detailed information.

Hi Gabriel,

You should be able to use SMF.s0.g_bytes (bytes allocated) and
SMF.s0.g_space (bytes free). To total amount of possible memory would be
the sum of both.

Regards,
Mattias




From gabster at lelutin.ca  Thu Jul 11 12:27:34 2013
From: gabster at lelutin.ca (Gabriel Filion)
Date: Thu, 11 Jul 2013 08:27:34 -0400
Subject: Monitoring cache file usage with varnish 3
In-Reply-To: <CE0425D2.6C431%mattias@nucleus.be>
References: <CE0425D2.6C431%mattias@nucleus.be>
Message-ID: <51DEA4B6.7010903@lelutin.ca>

On 11/07/13 03:06 AM, Mattias Geniar wrote:
>>
>> any idea what I can use instead of sm_balloc for monitoring cache file?
>>
>> if my message is confusing, I can point to more detailed information.

> You should be able to use SMF.s0.g_bytes (bytes allocated) and
> SMF.s0.g_space (bytes free). To total amount of possible memory would be
> the sum of both.

ah, well this would make more sense.


thanks to both of you.

-- 
Gabriel Filion

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 291 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130711/421ba3c0/attachment.bin>

From arkharov at gmail.com  Thu Jul 11 17:35:49 2013
From: arkharov at gmail.com (Roman Arkharov)
Date: Thu, 11 Jul 2013 21:35:49 +0400
Subject: Handle ESI-includes on cached page
Message-ID: <CAFxzbanhdTwXSXNqCKeQ+Zr=j8_wRzrPuYr2k3eY+HZdzDm7Nw@mail.gmail.com>

Hi Everyone.

I have a question about ESI includes in Varnish 3. I have 2 types of cache
which stored in Varnish:
1) page cache,
2) blocks cache.

Every page contains several blocks. Pages are generated by script index.php
and have TTL 15 minutes (set by HTTP header max-age). Blocks are generated
by script blocks.php and have TTL from 1 to 5 minutes. Blocks are included
to page by ESI instructions like:

<esi:include src="/blocks.php?block=some-name" />

I've read manual on varnish-cache.org (
https://www.varnish-cache.org/docs/3.0/tutorial/esi.html) and different
tutorials, but I've not found: is it possible to handle ESI instructions on
every request to cached page? In cause of page has TTL more than blocks,
blocks are updated only when page cache flushed.

As far as I understand from this presentation:
http://www.slideshare.net/neorey/varnish-the-high-performance-valhalla-9321198(slides
33-34) ESI-includes can be handle only if request served from
backend, because only in this case vcl_fetch called, am I right?

Thank you.
Roman Arkharov
arkharov at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130711/9383a6a3/attachment.html>

From perbu at varnish-software.com  Thu Jul 11 19:49:52 2013
From: perbu at varnish-software.com (Per Buer)
Date: Thu, 11 Jul 2013 21:49:52 +0200
Subject: Handle ESI-includes on cached page
In-Reply-To: <CAFxzbanhdTwXSXNqCKeQ+Zr=j8_wRzrPuYr2k3eY+HZdzDm7Nw@mail.gmail.com>
References: <CAFxzbanhdTwXSXNqCKeQ+Zr=j8_wRzrPuYr2k3eY+HZdzDm7Nw@mail.gmail.com>
Message-ID: <CAOXZevBAfv17HDNrkiPk8bkG5d_6i8rppxNkrkXL4xqqX7U2fg@mail.gmail.com>

Hi Roman,

On Thu, Jul 11, 2013 at 7:35 PM, Roman Arkharov <arkharov at gmail.com> wrote:
 (..)

> I've read manual on varnish-cache.org (
> https://www.varnish-cache.org/docs/3.0/tutorial/esi.html) and different
> tutorials, but I've not found: is it possible to handle ESI instructions on
> every request to cached page? In cause of page has TTL more than blocks,
> blocks are updated only when page cache flushed.
>

Yes. See below.

As far as I understand from this presentation:
> http://www.slideshare.net/neorey/varnish-the-high-performance-valhalla-9321198(slides 33-34) ESI-includes can be handle only if request served from
> backend, because only in this case vcl_fetch called, am I right?
>

Not quite. The parser is only run when the document is fetched.  When the
document is parses it is stored more or less like a list of pointers to
"esi fragments" in memory. When it is delivered these are more or less just
spit out. If one of these fragments time out they are refetched and
replaced.

So, you can have a static parent page with at TTL of 24h and within it
dynamic fragments.

I'll make a note out of going through the docs and see if I can make it a
bit more clear.

-- 
 <http://www.varnish-software.com/> *Per Buer*
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130711/28f0df14/attachment.html>

From supergrilo at gmail.com  Thu Jul 11 20:11:39 2013
From: supergrilo at gmail.com (Fabio Gomes dos Santos)
Date: Thu, 11 Jul 2013 17:11:39 -0300
Subject: Varnish return 503 (no backend connection) and never can back
Message-ID: <CAN0DB6N689GfDrHS=kad7wRt03K28BLPf2pSn44ex7sR==y_Jw@mail.gmail.com>

i have a problem using varnish 3.0 and debian wheezy. When i use backend on
command line, everything works. If i use init script, after some
connections varnish return 503 and never can back. Backend is not a sick
mode, health check still return 200, varnishlog only retorns  '4 FetchError
  c no backend connection'.


Full varnishlog request:

    0 Backend_health - foo Still healthy 4--X-RH 5 3 5 0.735348 0.733736
HTTP/1.1 200 OK

    0 Backend_health - bar Still healthy 4--X-RH 5 3 5 0.735955 0.730752
HTTP/1.1 200 OK

    0 CLI          - Rd ping

    0 CLI          - Wr 200 19 PONG 1373573348 1.0

    0 CLI          - Rd ping

    0 CLI          - Wr 200 19 PONG 1373573351 1.0

    0 Backend_health - foo Still healthy 4--X-RH 5 3 5 0.749470 0.737669
HTTP/1.1 200 OK

    0 Backend_health - bar Still healthy 4--X-RH 5 3 5 0.714554 0.726702
HTTP/1.1 200 OK

    0 CLI          - Rd ping

    0 CLI          - Wr 200 19 PONG 1373573354 1.0

    0 CLI          - Rd ping

    0 CLI          - Wr 200 19 PONG 1373573357 1.0

    0 Backend_health - foo Still healthy 4--X-RH 5 3 5 0.748910 0.740479
HTTP/1.1 200 OK

    0 Backend_health - bar Still healthy 4--X-RH 5 3 5 0.763746 0.735963
HTTP/1.1 200 OK

    0 CLI          - Rd ping

    0 CLI          - Wr 200 19 PONG 1373573360 1.0

    0 CLI          - Rd ping

    0 CLI          - Wr 200 19 PONG 1373573363 1.0

    0 Backend_health - foo Still healthy 4--X-RH 5 3 5 0.733996 0.738859
HTTP/1.1 200 OK

    0 Backend_health - bar Still healthy 4--X-RH 5 3 5 0.708820 0.729177
HTTP/1.1 200 OK

   12 SessionOpen  c 177.126.186.3 38253 :80

   12 ReqStart     c 177.126.186.3 38253 834855100

   12 RxRequest    c HEAD

   12 RxURL        c
/local/sp/bariri/bancos/C4037652224C3E4C3E/banco_santander_banespa_sa.html

   12 RxProtocol   c HTTP/1.1

   12 RxHeader     c User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu)
libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6

   12 RxHeader     c Host: ec2-54-224-160-79.compute-1.amazonaws.com

   12 RxHeader     c Accept: */*

   12 VCL_call     c recv lookup

   12 VCL_call     c hash

   12 Hash         c
/local/sp/bariri/bancos/C4037652224C3E4C3E/banco_santander_banespa_sa.html

   12 Hash         c ec2-54-224-160-79.compute-1.amazonaws.com

   12 VCL_return   c hash

   12 VCL_call     c miss fetch

   12 FetchError   c no backend connection

   12 VCL_call     c error deliver

   12 VCL_call     c deliver deliver

   12 TxProtocol   c HTTP/1.1

   12 TxStatus     c 503

   12 TxResponse   c Service Unavailable

   12 TxHeader     c Server: Varnish

   12 TxHeader     c Content-Type: text/html; charset=utf-8

   12 TxHeader     c Retry-After: 5

   12 TxHeader     c Content-Length: 418

   12 TxHeader     c Accept-Ranges: bytes

   12 TxHeader     c Date: Thu, 11 Jul 2013 20:09:24 GMT

   12 TxHeader     c Age: 0

   12 TxHeader     c Connection: close

   12 Length       c 418

   12 ReqEnd       c 834855100 1373573364.686753750 1373573364.686913252
0.000081301 0.000107527 0.000051975

   12 SessionClose c error

   12 StatSess     c 177.126.186.3 38253 0 1 1 0 0 0 216 418

    0 CLI          - Rd ping

    0 CLI          - Wr 200 19 PONG 1373573366 1.0



-- 
F?bio Santos
supergrilo at gmail.com
<http://br.linkedin.com/pub/f%C3%A1bio-santos/1b/20/422>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130711/e77b7616/attachment.html>

From mattias at nucleus.be  Fri Jul 12 07:05:32 2013
From: mattias at nucleus.be (Mattias Geniar)
Date: Fri, 12 Jul 2013 07:05:32 +0000
Subject: Varnish return 503 (no backend connection) and never can back
In-Reply-To: <CAN0DB6N689GfDrHS=kad7wRt03K28BLPf2pSn44ex7sR==y_Jw@mail.gmail.com>
Message-ID: <CE05771E.6CC08%mattias@nucleus.be>


> If i use init script, after some connections varnish return 503 and
>never can back. 
> Backend is not a sick mode, health check still return
> 200, varnishlog only retorns  '4 FetchError   c no backend connection'.

What does your backend definition look like?

M.




From perbu at varnish-software.com  Fri Jul 12 08:06:36 2013
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 12 Jul 2013 10:06:36 +0200
Subject: Varnish return 503 (no backend connection) and never can back
In-Reply-To: <CAN0DB6N689GfDrHS=kad7wRt03K28BLPf2pSn44ex7sR==y_Jw@mail.gmail.com>
References: <CAN0DB6N689GfDrHS=kad7wRt03K28BLPf2pSn44ex7sR==y_Jw@mail.gmail.com>
Message-ID: <CAOXZevB685ZVTj-7MDy484gdfJcHbUVejU4uS-W5Tyt2iytfTg@mail.gmail.com>

On Thu, Jul 11, 2013 at 10:11 PM, Fabio Gomes dos Santos <
supergrilo at gmail.com> wrote:

> i have a problem using varnish 3.0 and debian wheezy. When i use backend
> on command line, everything works. If i use init script, after some
> connections varnish return 503 and never can back. Backend is not a sick
> mode, health check still return 200, varnishlog only retorns  '4 FetchError
>   c no backend connection'.
>
Are you using saintmode? If so, you might want to try to set
saintmode_threshold to 0. Saintmode will kick in and make a backend sort of
sick. And if I recall correctly it does this without logging it.

-- 
 <http://www.varnish-software.com/> *Per Buer*
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130712/e21ac17b/attachment.html>

From arkharov at gmail.com  Fri Jul 12 10:38:49 2013
From: arkharov at gmail.com (Roman Arkharov)
Date: Fri, 12 Jul 2013 14:38:49 +0400
Subject: Handle ESI-includes on cached page
In-Reply-To: <CAOXZevBAfv17HDNrkiPk8bkG5d_6i8rppxNkrkXL4xqqX7U2fg@mail.gmail.com>
References: <CAFxzbanhdTwXSXNqCKeQ+Zr=j8_wRzrPuYr2k3eY+HZdzDm7Nw@mail.gmail.com>
 <CAOXZevBAfv17HDNrkiPk8bkG5d_6i8rppxNkrkXL4xqqX7U2fg@mail.gmail.com>
Message-ID: <CAFxzba=XkzhnzDwTqL4e07M50f7RZtVNOTQn7AaVAtd3qak=UA@mail.gmail.com>

Per, thank you for your answer.

I found the reason of my problem with blocks cache, it was in my
application. My app sets HTTP header: "Vary: Cookie". When I added line:

unset beresp.http.Vary;

to my vcl_fetch (or removed this header from app) problem gone.

Thank you again for your help.

Roman Arkharov,
arkharov at gmail.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130712/bd732b56/attachment.html>

From marcel at globaltopmedia.com  Fri Jul 12 11:09:22 2013
From: marcel at globaltopmedia.com (Marcel Dumont)
Date: Fri, 12 Jul 2013 11:09:22 +0000
Subject: mod_pagespeed with varnish
Message-ID: <CE05B081.C7B3%marcel@globaltopmedia.com>

Trying to get varnish to work efficient with a mod_pagespeed enabled web farm and getting poor hit ratios even when running using 1 web server.
Only relevant information on how to efficiently combine varnish with mod_pagespeed seems to be this thread:
https://groups.google.com/forum/#!msg/mod-pagespeed-discuss/j7X4uSckXlg/yl938D2e9yAJ

Is there any other best practices/recommendations out there?

thanks

Marcel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130712/07a13d90/attachment.html>

From lury at critical.pl  Fri Jul 12 12:02:27 2013
From: lury at critical.pl (=?UTF-8?Q?=C5=81ukasz_Rysiak?=)
Date: Fri, 12 Jul 2013 14:02:27 +0200
Subject: mod_pagespeed with varnish
In-Reply-To: <CE05B081.C7B3%marcel@globaltopmedia.com>
References: <CE05B081.C7B3%marcel@globaltopmedia.com>
Message-ID: <CAGeq0-etfPiEP_n1Kzy0o7u_9wuWR_3MkEbQ7HuZAvZDrf6pqw@mail.gmail.com>

Hi Marcel,

Keep in mind, that you have to overwrite expires, because MPS disables
user-side cache for optimized pages, and stores it's own cache for 5
minutes (on disk by default, but you can move it to memcache).



Regards,
?ukasz Rysiak



2013/7/12 Marcel Dumont <marcel at globaltopmedia.com>

>  Trying to get varnish to work efficient with a mod_pagespeed enabled web
> farm and getting poor hit ratios even when running using 1 web server.
> Only relevant information on how to efficiently combine varnish with
> mod_pagespeed seems to be this thread:
>
> https://groups.google.com/forum/#!msg/mod-pagespeed-discuss/j7X4uSckXlg/yl938D2e9yAJ
>
>  Is there any other best practices/recommendations out there?
>
>  thanks
>
>  Marcel
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130712/04dcecd0/attachment.html>

From supergrilo at gmail.com  Fri Jul 12 12:30:44 2013
From: supergrilo at gmail.com (Fabio Gomes dos Santos)
Date: Fri, 12 Jul 2013 09:30:44 -0300
Subject: Varnish return 503 (no backend connection) and never can back
In-Reply-To: <CAOXZevB685ZVTj-7MDy484gdfJcHbUVejU4uS-W5Tyt2iytfTg@mail.gmail.com>
References: <CAN0DB6N689GfDrHS=kad7wRt03K28BLPf2pSn44ex7sR==y_Jw@mail.gmail.com>
 <CAOXZevB685ZVTj-7MDy484gdfJcHbUVejU4uS-W5Tyt2iytfTg@mail.gmail.com>
Message-ID: <CAN0DB6Mv3TB4jF1nUBs4y4gig4F8=_Q3AkE3fh9yNvR9LfUNcw@mail.gmail.com>

Yes! I used saint mode.
This solve my problem. Thank you!


2013/7/12 Per Buer <perbu at varnish-software.com>

> On Thu, Jul 11, 2013 at 10:11 PM, Fabio Gomes dos Santos <
> supergrilo at gmail.com> wrote:
>
>> i have a problem using varnish 3.0 and debian wheezy. When i use backend
>> on command line, everything works. If i use init script, after some
>> connections varnish return 503 and never can back. Backend is not a sick
>> mode, health check still return 200, varnishlog only retorns  '4 FetchError
>>   c no backend connection'.
>>
> Are you using saintmode? If so, you might want to try to set
> saintmode_threshold to 0. Saintmode will kick in and make a backend sort of
> sick. And if I recall correctly it does this without logging it.
>
> --
>  <http://www.varnish-software.com/> *Per Buer*
> Phone: +47 958 39 117 | Skype: per.buer
> We Make Websites Fly!
>
> Winner of the Red Herring Top 100 Europe Award 2013
>
>
>


-- 
F?bio Santos
supergrilo at gmail.com
<http://br.linkedin.com/pub/f%C3%A1bio-santos/1b/20/422>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130712/39e5e5f7/attachment.html>

From perbu at varnish-software.com  Fri Jul 12 12:41:52 2013
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 12 Jul 2013 14:41:52 +0200
Subject: Varnish return 503 (no backend connection) and never can back
In-Reply-To: <CAN0DB6Mv3TB4jF1nUBs4y4gig4F8=_Q3AkE3fh9yNvR9LfUNcw@mail.gmail.com>
References: <CAN0DB6N689GfDrHS=kad7wRt03K28BLPf2pSn44ex7sR==y_Jw@mail.gmail.com>
 <CAOXZevB685ZVTj-7MDy484gdfJcHbUVejU4uS-W5Tyt2iytfTg@mail.gmail.com>
 <CAN0DB6Mv3TB4jF1nUBs4y4gig4F8=_Q3AkE3fh9yNvR9LfUNcw@mail.gmail.com>
Message-ID: <CAOXZevAoHob-84GEwZ4SwUkSU1q3a59-bu-xppfdV8BrXB=+RQ@mail.gmail.com>

I thought so. I have a patch somewhere that enables logging whenever
saintmode denies a backend connection.


On Fri, Jul 12, 2013 at 2:30 PM, Fabio Gomes dos Santos <
supergrilo at gmail.com> wrote:

> Yes! I used saint mode.
> This solve my problem. Thank you!
>
>
> 2013/7/12 Per Buer <perbu at varnish-software.com>
>
>> On Thu, Jul 11, 2013 at 10:11 PM, Fabio Gomes dos Santos <
>> supergrilo at gmail.com> wrote:
>>
>>> i have a problem using varnish 3.0 and debian wheezy. When i use backend
>>> on command line, everything works. If i use init script, after some
>>> connections varnish return 503 and never can back. Backend is not a sick
>>> mode, health check still return 200, varnishlog only retorns  '4 FetchError
>>>   c no backend connection'.
>>>
>> Are you using saintmode? If so, you might want to try to set
>> saintmode_threshold to 0. Saintmode will kick in and make a backend sort of
>> sick. And if I recall correctly it does this without logging it.
>>
>> --
>>  <http://www.varnish-software.com/> *Per Buer*
>> Phone: +47 958 39 117 | Skype: per.buer
>> We Make Websites Fly!
>>
>> Winner of the Red Herring Top 100 Europe Award 2013
>>
>>
>>
>
>
> --
> F?bio Santos
> supergrilo at gmail.com
>  <http://br.linkedin.com/pub/f%C3%A1bio-santos/1b/20/422>
>



-- 
 <http://www.varnish-software.com/> *Per Buer*
CEO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130712/11d82b07/attachment.html>

From john at startupgiraffe.com  Thu Jul 18 04:55:47 2013
From: john at startupgiraffe.com (John Cihocki)
Date: Thu, 18 Jul 2013 00:55:47 -0400
Subject: Hi everyone
Message-ID: <CAAd7LO=d30_Vo51xtxh22rquxwLpTa9ii5Ww4e4jXNS2RJ=yrQ@mail.gmail.com>

I am very curious about the oft quoted 5000 worker thread limit for varnish
instances. Particularly:

1) Is this recommendation still best practice despite having been made
years ago? Is it still relevant for -- now readily available -- beefier
server configurations like 16 or 32 cpu monsters with hundreds of GB RAM
and 10gb ethernet? Part of the original recommendation mentions bumping up
against file descriptor limits, but a cr1.8xlarge ec2 cluster instance
allows over 20m open file descriptors.
2) Is it the recommended limit for a single varnish instance, or a
recommended limit machine-wide. For instance, if I'm running 5 varnishd
instances on a single host, can each instance safely flex up to 5000 worker
threads concurrently or should the sum of worker threads across all
instances not exceed 5000?

Very interested to hear -- thanks!

Johnny
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130718/541657e0/attachment.html>

From hugo.cisneiros at gmail.com  Fri Jul 19 23:29:03 2013
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Fri, 19 Jul 2013 20:29:03 -0300
Subject: How client.identity works?
Message-ID: <CA+KACLnvG2H6bdnFxBJMWNazzMqe9=RJ9bVwWkJCcZGJduhncQ@mail.gmail.com>

Hau,

I'm testing some configurations with the client director and cookie-based
sticky sessions and want to know:

How client.identity decides which backend to use?

For example, if I have two servers with exactly the *same* backends, *same*
VCLs, and a request is sent with client.identity = "ABCDEF1234567890" on
BOTH servers, does both requests on both servers go to the same backend
server? It could be true if using some sort of hash table.

Or the client director first picks a random backend and associates its
value to the client.identity, remembering for the future? This could be
true if using some sort of a key:value table. This way, first scenario
won't work reliably.

Many thanks,

-- 
[]'s
Hugo
www.devin.com.br
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130719/786da76f/attachment.html>

From ksorensen at nordija.com  Sat Jul 20 12:10:39 2013
From: ksorensen at nordija.com (Kristian =?ISO-8859-1?Q?Gr=F8nfeldt_S=F8rensen?=)
Date: Sat, 20 Jul 2013 14:10:39 +0200
Subject: How client.identity works?
In-Reply-To: <CA+KACLnvG2H6bdnFxBJMWNazzMqe9=RJ9bVwWkJCcZGJduhncQ@mail.gmail.com>
References: <CA+KACLnvG2H6bdnFxBJMWNazzMqe9=RJ9bVwWkJCcZGJduhncQ@mail.gmail.com>
Message-ID: <1374322239.16821.21.camel@kriller.nordija.dk>

On Fri, 2013-07-19 at 20:29 -0300, Hugo Cisneiros (Eitch) wrote:
> Hau,
> 
> 
> I'm testing some configurations with the client director and
> cookie-based sticky sessions and want to know:
> 
> 
> How client.identity decides which backend to use?
> 
> 
> For example, if I have two servers with exactly the *same* backends,
> *same* VCLs, and a request is sent with client.identity =
> "ABCDEF1234567890" on BOTH servers, does both requests on both servers
> go to the same backend server? It could be true if using some sort of
> hash table.

It is my experience that this is the case, and I'm relying on this
behaviour in a couple of installations. IIRC, the director is an array
of backends, and the hashing ultimately returns an array index, so as
long as your director 

      * has the same number of backends 
      * has its backends defined in the same order
      * has the same weights
      * has the same view of backend health

on each Varnish instance, the same client.identity string will cause the
same backend to be chosen. So if one of your backends become sick as
seen only from the one of your Varnishes, then requests for that backend
will be routed differently between the two Varnishes.

> 
> Or the client director first picks a random backend and associates its
> value to the client.identity, remembering for the future? This could
> be true if using some sort of a key:value table. This way, first
> scenario won't work reliably.

The client director does not save any state between requests.

/Kristian





From imanandshah at gmail.com  Mon Jul 22 09:49:18 2013
From: imanandshah at gmail.com (Anand Shah)
Date: Mon, 22 Jul 2013 09:49:18 +0000
Subject: Enable Client Cache control Headers
Message-ID: <CAGUf+MxQwpBH0yJQ3+-VuO2y9NDR4KVA6zt3ey7Lbqab4uBaWg@mail.gmail.com>

Hi,

I am trying to enable client side cache handling but not working for some
unknown reasons. I will get two parameters to read from origin's response

.i.e


1. Edge-control: downstream-ttl=200s

2. Edge-control: cache-maxage=200s





RULES:-


This goes in vcl_fetch

  if ( beresp.http.Edge-control ) {
                if (beresp.http.Edge-control ~ "cache-maxage") {
                        set beresp.ttl =
std.duration(regsub(beresp.http.Edge-control, "cache-maxage=([0-9]+).*",
"\1s"), 0s);
                        #set beresp.http.Cache-Control =
regsub(beresp.http.Edge-control, ".*cache-maxage=([0-9a-z]+).*", "\1");
                        #set beresp.http.X-AGEBACKEND = "yes";
                } elseif ( beresp.http.Edge-control ~
"downstream-ttl=([0-9]+)s")  {
                        set beresp.http.Cache-Control =
regsub(beresp.http.Edge-control, "downstream-ttl=([0-9]+).*", "maxage=\1" +
"s");
                }
                unset beresp.http.Edge-control;
                return (deliver);
        }



It exists as soon as it matches any of the criteria and does not do a
lookup for other Cache parameter and delivers.


 1. It should enter the loop only if the Response header has Edge-Control.

2. check for matching parameters and set else follow the vcl.

3. If only cache-maxage contains it should check for Cache Control headers
below in the vcl (this does not work; want it to work)



No Idea what is going is wrong here.











 Regards,

   Anand Shah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130722/52903847/attachment.html>

From yarishima42 at googlemail.com  Wed Jul 24 23:31:29 2013
From: yarishima42 at googlemail.com (Yari Shima)
Date: Thu, 25 Jul 2013 01:31:29 +0200
Subject: Varnish pipe through for SSL requests
Message-ID: <2691037.EI24baZqZ7@hierro>

Hello,

We have the following server setting:

First a Magento shop on a managed server. It serves the non-SSL and SSL 
traffic. Now we want to setup a varnish caching server. Because it is a 
managed server we cannot install any extra software on it, so we need to do it 
on another root server.

I setup varnish on this root server and for non-SSL requests everything works 
fine. Varnish should only cache non-SSL pages, everything regarding the 
checkout or account should not be cached by varnish. My question is, how do I 
have to setup varnish, that the SSL requests pipe through it and go directly 
to the managed server? And were do I have to install the SSL certificate?

Thanx in advance
YS



From smwood4 at gmail.com  Thu Jul 25 01:39:28 2013
From: smwood4 at gmail.com (Stephen Wood)
Date: Wed, 24 Jul 2013 18:39:28 -0700
Subject: Varnish pipe through for SSL requests
In-Reply-To: <2691037.EI24baZqZ7@hierro>
References: <2691037.EI24baZqZ7@hierro>
Message-ID: <CA+pzux-TO=hHOF07vhVftBP7Sez4_uKKd72_buLJQnWq_F6nhQ@mail.gmail.com>

Unfortunately varnish only supports HTTP and not HTTPS, but you may find
this other users experience helpful:

https://www.varnish-cache.org/lists/pipermail/varnish-misc/2011-June/020695.html


On Wed, Jul 24, 2013 at 4:31 PM, Yari Shima <yarishima42 at googlemail.com>wrote:

> Hello,
>
> We have the following server setting:
>
> First a Magento shop on a managed server. It serves the non-SSL and SSL
> traffic. Now we want to setup a varnish caching server. Because it is a
> managed server we cannot install any extra software on it, so we need to
> do it
> on another root server.
>
> I setup varnish on this root server and for non-SSL requests everything
> works
> fine. Varnish should only cache non-SSL pages, everything regarding the
> checkout or account should not be cached by varnish. My question is, how
> do I
> have to setup varnish, that the SSL requests pipe through it and go
> directly
> to the managed server? And were do I have to install the SSL certificate?
>
> Thanx in advance
> YS
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
Stephen Wood
Dev/Ops Engineer
Moz, Inc.
Website: www.heystephenwood.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130724/70da4f00/attachment.html>

From r at roze.lv  Thu Jul 25 02:12:43 2013
From: r at roze.lv (Reinis Rozitis)
Date: Thu, 25 Jul 2013 05:12:43 +0300
Subject: Varnish pipe through for SSL requests
In-Reply-To: <2691037.EI24baZqZ7@hierro>
References: <2691037.EI24baZqZ7@hierro>
Message-ID: <192D29F2D6C34655A14B0C23F87CD1F7@NeiRoze>

> My question is, how do I have to setup varnish, that the SSL requests pipe 
> through it and go directly to the managed server? And were do I have to 
> install the SSL certificate?

Varnish doesnt play with SSL in any fashion ( 
https://www.varnish-cache.org/docs/trunk/phk/ssl.html ), so you have to use 
other tools or different approaches for serving/piping the SSL traffic.

Depending on the software you are familiar with you can either directly 
forward the 443 port to your backend using the OS tools like 
iptables/ipfw/xinetd/etc (or any other "firewall/portfoward type" 
software) - then you need to install the certificate on the backend 
webserver.
The drawback of this method is (unless you are using something like TPROXY 
for the iptable rules) the backend won't see the original client ip.


Or use something like haproxy / nginx / stud to offload the SSL.
Then you have to install the certificate on the proxy (unless it works in 
"tcp mode" -  haproxy (and nginx with third party module) can operate like 
that).

Usually this is more easy to setup and the client ip can be passed with 
additional http headers (X-Forwarded-For) and most webservers have modules 
to transparently convert the ip for the application (nginx - realip / 
apache - mod_rpaf)


For a single instance of varnish I personally use Stud ( 
https://github.com/bumptech/stud ).
Haproxy ( http://haproxy.1wt.eu/ ) on the other hand is more suitable for 
more complex setups (multiple backends / loadbalancing and more).

rr 




From yarishima42 at googlemail.com  Thu Jul 25 19:22:40 2013
From: yarishima42 at googlemail.com (Yari Shima)
Date: Thu, 25 Jul 2013 21:22:40 +0200
Subject: Varnish pipe through for SSL requests
In-Reply-To: <192D29F2D6C34655A14B0C23F87CD1F7@NeiRoze>
References: <2691037.EI24baZqZ7@hierro>
 <192D29F2D6C34655A14B0C23F87CD1F7@NeiRoze>
Message-ID: <3996460.HSvvyf2783@hierro>

Hi Reinis,

Thanks for your awnser.
But can't I use apache to listen on port 443 on my root server and with 
mod_proxy pipr the traffic through to my managed server?

Best
YS



From smwood4 at gmail.com  Thu Jul 25 20:31:58 2013
From: smwood4 at gmail.com (Stephen Wood)
Date: Thu, 25 Jul 2013 13:31:58 -0700
Subject: Varnish pipe through for SSL requests
In-Reply-To: <3996460.HSvvyf2783@hierro>
References: <2691037.EI24baZqZ7@hierro>
 <192D29F2D6C34655A14B0C23F87CD1F7@NeiRoze>
 <3996460.HSvvyf2783@hierro>
Message-ID: <CA+pzux-+Mai4kcsvkWvGS+__GSFzoajxS5gepnw4pJnRUGRCgw@mail.gmail.com>

I'm not familiar with mod_proxy, but the point being is that traffic being
received by varnish must be plain http traffic. Any SSL encryption
*must*be terminated before it reaches the port Varnish is running.

The are many different ways to do this and mod_proxy and apache might be
one as long as you fulfill the above requirement and Varnish is receiving
only http traffic. I find that the easiest solution is to do SSL
termination on your load balancer.


On Thu, Jul 25, 2013 at 12:22 PM, Yari Shima <yarishima42 at googlemail.com>wrote:

> Hi Reinis,
>
> Thanks for your awnser.
> But can't I use apache to listen on port 443 on my root server and with
> mod_proxy pipr the traffic through to my managed server?
>
> Best
> YS
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
Stephen Wood
Dev/Ops Engineer
Moz, Inc.
Website: www.heystephenwood.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130725/bc5f2324/attachment.html>

From numard at gmail.com  Fri Jul 26 01:22:24 2013
From: numard at gmail.com (Norberto Meijome)
Date: Fri, 26 Jul 2013 11:22:24 +1000
Subject: Varnish pipe through for SSL requests
In-Reply-To: <3996460.HSvvyf2783@hierro>
References: <2691037.EI24baZqZ7@hierro>
 <192D29F2D6C34655A14B0C23F87CD1F7@NeiRoze>
 <3996460.HSvvyf2783@hierro>
Message-ID: <CACj2-4KxHpWFSEuAOGYogbQw7zc0Nxt-Ng08UG2F9f6xb3+Y6Q@mail.gmail.com>

You should be able to with modproxy.. We terminate on nginx which acts as
proxy for clusters of app servers and varnishes...just tell nginx to
connect to varnish over http.
On 26/07/2013 5:27 AM, "Yari Shima" <yarishima42 at googlemail.com> wrote:

> Hi Reinis,
>
> Thanks for your awnser.
> But can't I use apache to listen on port 443 on my root server and with
> mod_proxy pipr the traffic through to my managed server?
>
> Best
> YS
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130726/b40aea4b/attachment.html>

From puneet.arora at insticator.com  Sat Jul 27 20:48:31 2013
From: puneet.arora at insticator.com (Puneet)
Date: Sat, 27 Jul 2013 16:48:31 -0400
Subject: Stop users accessing website via IP address
Message-ID: <000f01ce8b0a$a872c230$f9584690$@insticator.com>

Hi all,


I want to stop the users accessing my website via IP address.
I am using varnish as cache.
I have the following code in place but it is not working.

In vcl_recv() {
     if(req.url ~ "XX.XX.XXX.XXX") {
     error 750 "Moved Permanently";
  } } 

And in vcl_error()
sub vcl_error {
    if (obj.status == 750) {
    set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");
    set obj.http.Location = req.http.X-REDIRURL;
    set obj.status = 301;
    unset req.http.X-REDIRURL;
    return(deliver);
}

But this does not redirect the user to the website, instead it delivers the
page.
Can anyone tell what I am missing?

 

Thanks

Puneet

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/98eea45c/attachment.html>

From smwood4 at gmail.com  Sat Jul 27 21:04:43 2013
From: smwood4 at gmail.com (Stephen Wood)
Date: Sat, 27 Jul 2013 14:04:43 -0700
Subject: Stop users accessing website via IP address
In-Reply-To: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
Message-ID: <CA+pzux8bJ2CY6iCciYpLR8-FURdoR=eEE06Cgxt_Zj6OCcN3qA@mail.gmail.com>

Hello, Puneet. You'll need to do the blocking on the sub vcl_recv and not
sub vcl_error.

Here's an easy VCL config example:

acl forbidden {
  "xxx.xxx.xxx.xxx";
  "xxx.xxx.xxx.xxx";
}

sub vcl_recv {
  # Prevent access from these hosts
  if (client.ip ~ forbidden) {
    error 403 "Forbidden";
  }
}

Let me know if you have any other questions.

On Sat, Jul 27, 2013 at 1:48 PM, Puneet <puneet.arora at insticator.com> wrote:

> Hi all,****
>
>
> I want to stop the users accessing my website via IP address.
> I am using varnish as cache.
> I have the following code in place but it is not working.
>
> In vcl_recv() {
>      if(req.url ~ "XX.XX.XXX.XXX") {
>      error 750 "Moved Permanently";
>   } }
>
> And in vcl_error()
> sub vcl_error {
>     if (obj.status == 750) {
>     set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
> http://mywebsite.com<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5qRF&_t=tracking_anet>
> ");
>     set obj.http.Location = req.http.X-REDIRURL;
>     set obj.status = 301;
>     unset req.http.X-REDIRURL;
>     return(deliver);
> }
>
> But this does not redirect the user to the website, instead it delivers
> the page.
> Can anyone tell what I am missing?****
>
> ** **
>
> Thanks****
>
> Puneet****
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
Stephen Wood
Dev/Ops Engineer
Moz, Inc.
Website: www.heystephenwood.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/9c351fd5/attachment.html>

From puneet.arora at insticator.com  Sat Jul 27 21:15:31 2013
From: puneet.arora at insticator.com (Puneet)
Date: Sat, 27 Jul 2013 17:15:31 -0400
Subject: Stop users accessing website via IP address
In-Reply-To: <CA+pzux8bJ2CY6iCciYpLR8-FURdoR=eEE06Cgxt_Zj6OCcN3qA@mail.gmail.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
 <CA+pzux8bJ2CY6iCciYpLR8-FURdoR=eEE06Cgxt_Zj6OCcN3qA@mail.gmail.com>
Message-ID: <002301ce8b0e$6e5f37a0$4b1da6e0$@insticator.com>

Hi Stephen,

 

I think I did not stated the problem correctly.

 

Let me explain the issue:

Say, my website www.mywebsite.com <http://www.mywebsite.com>  is hosted on
IP : 167.123.98.20

Now the Users could type in the IP address i.e. 167.123.98.20 and access my
website as the server is listening at port 80.

 

Now, if any user does that, I want to redirect the Users to
www.mywebsite.com <http://www.mywebsite.com> .

I don't want any User transaction to be done using 167.123.98.20.

 

Therefore I had the following code in place.

 

//If the User access my website using the IP address

vcl_recv() {
     if(req.url ~ "167.123.98.20") {
     error 750 "Moved Permanently";
  } } 

 

Redirect the user to www.mywebsite.com.

sub vcl_error {
    if (obj.status == 750) {
    set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");
    set obj.http.Location = req.http.X-REDIRURL;
    set obj.status = 301;
    unset req.http.X-REDIRURL;
    return(deliver);
}

 

I hope I clarified the issue.

 

Thanks

Puneet

 

From: Stephen Wood [mailto:smwood4 at gmail.com] 
Sent: Saturday, July 27, 2013 5:05 PM
To: Puneet
Cc: varnish-misc at varnish-cache.org
Subject: Re: Stop users accessing website via IP address

 

Hello, Puneet. You'll need to do the blocking on the sub vcl_recv and not
sub vcl_error.

 

Here's an easy VCL config example:

 

acl forbidden {
  "xxx.xxx.xxx.xxx";
  "xxx.xxx.xxx.xxx";
}

sub vcl_recv {
  # Prevent access from these hosts
  if (client.ip ~ forbidden) {
    error 403 "Forbidden";
  }
}


Let me know if you have any other questions.

 

On Sat, Jul 27, 2013 at 1:48 PM, Puneet <puneet.arora at insticator.com
<mailto:puneet.arora at insticator.com> > wrote:

Hi all,


I want to stop the users accessing my website via IP address.
I am using varnish as cache.
I have the following code in place but it is not working.

In vcl_recv() {
     if(req.url ~ "XX.XX.XXX.XXX") {
     error 750 "Moved Permanently";
  } } 

And in vcl_error()
sub vcl_error {
    if (obj.status == 750) {
    set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");
    set obj.http.Location = req.http.X-REDIRURL;
    set obj.status = 301;
    unset req.http.X-REDIRURL;
    return(deliver);
}

But this does not redirect the user to the website, instead it delivers the
page.
Can anyone tell what I am missing?

 

Thanks

Puneet


_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org> 
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc





 

-- 

Stephen Wood

Dev/Ops Engineer

Moz, Inc.

Website: www.heystephenwood.com <http://www.heystephenwood.com> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/b8c5def5/attachment.html>

From hugo.cisneiros at gmail.com  Sat Jul 27 21:28:57 2013
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Sat, 27 Jul 2013 18:28:57 -0300
Subject: Stop users accessing website via IP address
In-Reply-To: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
Message-ID: <CA+KACLkCCfpK2XmM4-vaxOW2b=XBL-nbgadgt7FE3XejfWA6cw@mail.gmail.com>

On Sat, Jul 27, 2013 at 5:48 PM, Puneet <puneet.arora at insticator.com> wrote:

> I want to stop the users accessing my website via IP address.
>
> I am using varnish as cache.
> I have the following code in place but it is not working.
>
> In vcl_recv() {
>      if(req.url ~ "XX.XX.XXX.XXX") {
>      error 750 "Moved Permanently";
>   } }
>

In vcl_recv, you're comparting the IP address with the request URL
(req.url), which is wrong. You should compare with client.ip, as it
represents the user's IP address.

Anyway, a much better approach in my opinion is the code:

# list of forbidden ips
acl forbidden {
  "192.168.0.1",
  "192.168.0.2",
  "XXX.XXX.XXX.XXX"
}

sub vcl_recv {
  if (client.ip ~ forbidden) {
    error 301 "http://mywebsite.com";
  }
}

sub vcl_error {
  set obj.http.Content-Type = "text/html; charset=utf-8";
  set obj.http.Retry-After = "5";

  # we deal with redirects here
  if (obj.status == 301) {
    set obj.http.Location = obj.response;
    set obj.response = "Moved Temporarily";
    return (deliver);
  }

  if (obj.status == 301){
    set obj.http.Location = obj.response;
    set obj.response = "Moved Permanently";
    return (deliver);
  }
}

This way you can update the ACL to multiple IP addresses and they'll be all
redirected to mywebsite.com.

-- 
[]'s
Hugo
www.devin.com.br
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/8ceb19b3/attachment.html>

From travis.crowder at spechal.com  Sat Jul 27 23:53:11 2013
From: travis.crowder at spechal.com (Travis Crowder)
Date: Sat, 27 Jul 2013 18:53:11 -0500
Subject: Stop users accessing website via IP address
In-Reply-To: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
Message-ID: <A89915AA-A7FE-432D-90CD-E8B6BF12A105@spechal.com>

Check against req.http.Host

In vcl_recv:

     if(req.http.Host ~ "8.8.8.8") {
         error 750;
     }

-Travis Crowder

On Jul 27, 2013, at 3:48 PM, Puneet <puneet.arora at insticator.com> wrote:

> Hi all,
> 
> I want to stop the users accessing my website via IP address.
> I am using varnish as cache.
> I have the following code in place but it is not working.
> 
> In vcl_recv() {
>      if(req.url ~ "XX.XX.XXX.XXX") {
>      error 750 "Moved Permanently";
>   } } 
> 
> And in vcl_error()
> sub vcl_error {
>     if (obj.status == 750) {
>     set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "http://mywebsite.com");
>     set obj.http.Location = req.http.X-REDIRURL;
>     set obj.status = 301;
>     unset req.http.X-REDIRURL;
>     return(deliver);
> }
> 
> But this does not redirect the user to the website, instead it delivers the page.
> Can anyone tell what I am missing?
>  
> Thanks
> Puneet
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/da02d31d/attachment.html>

From puneet.arora at insticator.com  Sun Jul 28 02:38:24 2013
From: puneet.arora at insticator.com (Puneet)
Date: Sat, 27 Jul 2013 22:38:24 -0400
Subject: Stop users accessing website via IP address
In-Reply-To: <A89915AA-A7FE-432D-90CD-E8B6BF12A105@spechal.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
 <A89915AA-A7FE-432D-90CD-E8B6BF12A105@spechal.com>
Message-ID: <005701ce8b3b$88e6ee10$9ab4ca30$@insticator.com>

HI Travis,

 

Thanks for the reply.

I think that should work.

 

Just one question.

In sub vcl_error() should I also change the 

 

set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");

TO : -->

set req.http.X-REDIRURL = regsub(req.http.host,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");

?

 

Because when replace req.url with req.http.host, It again stops working.

And If I don't do it, the bowser gives an error "Too many redirects"

 

Thanks

Puneet

 

From: Travis Crowder [mailto:travis.crowder at spechal.com] 
Sent: Saturday, July 27, 2013 7:53 PM
To: Puneet
Cc: varnish-misc at varnish-cache.org
Subject: Re: Stop users accessing website via IP address

 

Check against req.http.Host

 

In vcl_recv:

 

     if(req.http.Host ~ "8.8.8.8") {

         error 750;

     }

 

-Travis Crowder

 

On Jul 27, 2013, at 3:48 PM, Puneet <puneet.arora at insticator.com
<mailto:puneet.arora at insticator.com> > wrote:





Hi all,


I want to stop the users accessing my website via IP address.
I am using varnish as cache.
I have the following code in place but it is not working.

In vcl_recv() {
     if(req.url ~ "XX.XX.XXX.XXX") {
     error 750 "Moved Permanently";
  } } 

And in vcl_error()
sub vcl_error {
    if (obj.status == 750) {
    set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");
    set obj.http.Location = req.http.X-REDIRURL;
    set obj.status = 301;
    unset req.http.X-REDIRURL;
    return(deliver);
}

But this does not redirect the user to the website, instead it delivers the
page.
Can anyone tell what I am missing?

 

Thanks

Puneet

_______________________________________________
varnish-misc mailing list
 <mailto:varnish-misc at varnish-cache.org> varnish-misc at varnish-cache.org
 <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/21c9f677/attachment.html>

From travis.crowder at spechal.com  Sun Jul 28 03:03:44 2013
From: travis.crowder at spechal.com (Travis Crowder)
Date: Sat, 27 Jul 2013 22:03:44 -0500
Subject: Stop users accessing website via IP address
In-Reply-To: <005701ce8b3b$88e6ee10$9ab4ca30$@insticator.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
 <A89915AA-A7FE-432D-90CD-E8B6BF12A105@spechal.com>
 <005701ce8b3b$88e6ee10$9ab4ca30$@insticator.com>
Message-ID: <DA483569-D63F-4DE5-ADD5-855DD87872FC@spechal.com>

Your regular expression is malformed and you don't do any capturing or replacement in the regsub function so I am not sure what you're trying to accomplish there.

Can you elaborate on what you're trying set this header's value to?

-Travis Crowder


On Jul 27, 2013, at 9:38 PM, Puneet <puneet.arora at insticator.com> wrote:

> HI Travis,
>  
> Thanks for the reply.
> I think that should work.
>  
> Just one question.
> In sub vcl_error() should I also change the
>  
> set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "http://mywebsite.com");
> TO : ?
> set req.http.X-REDIRURL = regsub(req.http.host,"https?://[^/$]+", "http://mywebsite.com");
> ?
>  
> Because when replace req.url with req.http.host, It again stops working.
> And If I don?t do it, the bowser gives an error ?Too many redirects?
>  
> Thanks
> Puneet
>  
> From: Travis Crowder [mailto:travis.crowder at spechal.com] 
> Sent: Saturday, July 27, 2013 7:53 PM
> To: Puneet
> Cc: varnish-misc at varnish-cache.org
> Subject: Re: Stop users accessing website via IP address
>  
> Check against req.http.Host
>  
> In vcl_recv:
>  
>      if(req.http.Host ~ "8.8.8.8") {
>          error 750;
>      }
>  
> -Travis Crowder
>  
> On Jul 27, 2013, at 3:48 PM, Puneet <puneet.arora at insticator.com> wrote:
> 
> 
> Hi all,
> 
> I want to stop the users accessing my website via IP address.
> I am using varnish as cache.
> I have the following code in place but it is not working.
> 
> In vcl_recv() {
>      if(req.url ~ "XX.XX.XXX.XXX") {
>      error 750 "Moved Permanently";
>   } } 
> 
> And in vcl_error()
> sub vcl_error {
>     if (obj.status == 750) {
>     set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "http://mywebsite.com");
>     set obj.http.Location = req.http.X-REDIRURL;
>     set obj.status = 301;
>     unset req.http.X-REDIRURL;
>     return(deliver);
> }
> 
> But this does not redirect the user to the website, instead it delivers the page.
> Can anyone tell what I am missing?
>  
> Thanks
> Puneet
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/d5a8f122/attachment.html>

From puneet.arora at insticator.com  Sun Jul 28 03:13:12 2013
From: puneet.arora at insticator.com (Puneet)
Date: Sat, 27 Jul 2013 23:13:12 -0400
Subject: Stop users accessing website via IP address
In-Reply-To: <DA483569-D63F-4DE5-ADD5-855DD87872FC@spechal.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
 <A89915AA-A7FE-432D-90CD-E8B6BF12A105@spechal.com>
 <005701ce8b3b$88e6ee10$9ab4ca30$@insticator.com>
 <DA483569-D63F-4DE5-ADD5-855DD87872FC@spechal.com>
Message-ID: <006a01ce8b40$65984990$30c8dcb0$@insticator.com>

HI Travis,

 

I am trying to redirect the user to the location of my website.

 

For eg. The user enters http://X.X.X.X/login

I want to redirect the user to http://www.mywebsite.com/login

 

In the regular expression I am replacing the HOST NAME i.e. http://X.X.X.X
with http://www.mywebsite.com.

Therefore, I am just setting the header value to
http://www.mywebsite.com/login

And then in the next line

Set , obj.location to : http://www.mywebsite.com/login

 

This was what I intended to accomplish.

 

Thanks

Puneet

From: Travis Crowder [mailto:travis.crowder at spechal.com] 
Sent: Saturday, July 27, 2013 11:04 PM
To: Puneet
Cc: varnish-misc at varnish-cache.org
Subject: Re: Stop users accessing website via IP address

 

Your regular expression is malformed and you don't do any capturing or
replacement in the regsub function so I am not sure what you're trying to
accomplish there.

 

Can you elaborate on what you're trying set this header's value to?

 

-Travis Crowder

 

 

On Jul 27, 2013, at 9:38 PM, Puneet <puneet.arora at insticator.com
<mailto:puneet.arora at insticator.com> > wrote:





HI Travis,

 

Thanks for the reply.

I think that should work.

 

Just one question.

In sub vcl_error() should I also change the

 

set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");

TO : -->

set req.http.X-REDIRURL = regsub(req.http.host,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");

?

 

Because when replace req.url with req.http.host, It again stops working.

And If I don't do it, the bowser gives an error "Too many redirects"

 

Thanks

Puneet

 

From: Travis Crowder [mailto:travis.crowder at spechal.com <http://spechal.com>
] 
Sent: Saturday, July 27, 2013 7:53 PM
To: Puneet
Cc: varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org> 
Subject: Re: Stop users accessing website via IP address

 

Check against req.http.Host

 

In vcl_recv:

 

     if(req.http.Host ~ "8.8.8.8") {

         error 750;

     }

 

-Travis Crowder

 

On Jul 27, 2013, at 3:48 PM, Puneet < <mailto:puneet.arora at insticator.com>
puneet.arora at insticator.com> wrote:






Hi all,


I want to stop the users accessing my website via IP address.
I am using varnish as cache.
I have the following code in place but it is not working.

In vcl_recv() {
     if(req.url ~ "XX.XX.XXX.XXX") {
     error 750 "Moved Permanently";
  } } 

And in vcl_error()
sub vcl_error {
    if (obj.status == 750) {
    set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");
    set obj.http.Location = req.http.X-REDIRURL;
    set obj.status = 301;
    unset req.http.X-REDIRURL;
    return(deliver);
}

But this does not redirect the user to the website, instead it delivers the
page.
Can anyone tell what I am missing?

 

Thanks

Puneet

_______________________________________________
varnish-misc mailing list
 <mailto:varnish-misc at varnish-cache.org> varnish-misc at varnish-cache.org
 <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/0f680cff/attachment.html>

From travis.crowder at spechal.com  Sun Jul 28 04:15:42 2013
From: travis.crowder at spechal.com (Travis Crowder)
Date: Sat, 27 Jul 2013 23:15:42 -0500
Subject: Stop users accessing website via IP address
In-Reply-To: <DA483569-D63F-4DE5-ADD5-855DD87872FC@spechal.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
 <A89915AA-A7FE-432D-90CD-E8B6BF12A105@spechal.com>
 <005701ce8b3b$88e6ee10$9ab4ca30$@insticator.com>
 <DA483569-D63F-4DE5-ADD5-855DD87872FC@spechal.com>
Message-ID: <184F346E-E119-4734-A882-FCF9BDA84D34@spechal.com>

I guess I am confused as to why you need the rewrite.

sub vcl_recv {
	if(req.http.Host ~ "8.8.8.8") { error 750; }
}

sub vcl_error {
	if(obj.status == 750) { 
		 set obj.http.Location = "http://google.com/login"; 
		 set obj.status = 302; 
		 return (deliver); 
	}
}

On Jul 27, 2013, at 10:03 PM, Travis Crowder <travis.crowder at spechal.com> wrote:

> Your regular expression is malformed and you don't do any capturing or replacement in the regsub function so I am not sure what you're trying to accomplish there.
> 
> Can you elaborate on what you're trying set this header's value to?
> 
> -Travis Crowder
> 
> 
> On Jul 27, 2013, at 9:38 PM, Puneet <puneet.arora at insticator.com> wrote:
> 
>> HI Travis,
>>  
>> Thanks for the reply.
>> I think that should work.
>>  
>> Just one question.
>> In sub vcl_error() should I also change the
>>  
>> set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "http://mywebsite.com");
>> TO : ?
>> set req.http.X-REDIRURL = regsub(req.http.host,"https?://[^/$]+", "http://mywebsite.com");
>> ?
>>  
>> Because when replace req.url with req.http.host, It again stops working.
>> And If I don?t do it, the bowser gives an error ?Too many redirects?
>>  
>> Thanks
>> Puneet
>>  
>> From: Travis Crowder [mailto:travis.crowder at spechal.com] 
>> Sent: Saturday, July 27, 2013 7:53 PM
>> To: Puneet
>> Cc: varnish-misc at varnish-cache.org
>> Subject: Re: Stop users accessing website via IP address
>>  
>> Check against req.http.Host
>>  
>> In vcl_recv:
>>  
>>      if(req.http.Host ~ "8.8.8.8") {
>>          error 750;
>>      }
>>  
>> -Travis Crowder
>>  
>> On Jul 27, 2013, at 3:48 PM, Puneet <puneet.arora at insticator.com> wrote:
>> 
>> 
>> Hi all,
>> 
>> I want to stop the users accessing my website via IP address.
>> I am using varnish as cache.
>> I have the following code in place but it is not working.
>> 
>> In vcl_recv() {
>>      if(req.url ~ "XX.XX.XXX.XXX") {
>>      error 750 "Moved Permanently";
>>   } } 
>> 
>> And in vcl_error()
>> sub vcl_error {
>>     if (obj.status == 750) {
>>     set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "http://mywebsite.com");
>>     set obj.http.Location = req.http.X-REDIRURL;
>>     set obj.status = 301;
>>     unset req.http.X-REDIRURL;
>>     return(deliver);
>> }
>> 
>> But this does not redirect the user to the website, instead it delivers the page.
>> Can anyone tell what I am missing?
>>  
>> Thanks
>> Puneet
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130727/1d718d5d/attachment.html>

From pprocacci at datapipe.com  Sun Jul 28 05:57:39 2013
From: pprocacci at datapipe.com (Paul A. Procacci)
Date: Sun, 28 Jul 2013 00:57:39 -0500
Subject: Stop users accessing website via IP address
In-Reply-To: <184F346E-E119-4734-A882-FCF9BDA84D34@spechal.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
 <A89915AA-A7FE-432D-90CD-E8B6BF12A105@spechal.com>
 <005701ce8b3b$88e6ee10$9ab4ca30$@insticator.com>
 <DA483569-D63F-4DE5-ADD5-855DD87872FC@spechal.com>
 <184F346E-E119-4734-A882-FCF9BDA84D34@spechal.com>
Message-ID: <20130728055739.GL45203@nat.myhome>

> I guess I am confused as to why you need the rewrite.
>
> sub vcl_recv {
>       if(req.http.Host ~ "8.8.8.8") { error 750; }
> }

Not to step on any toes.....

The above can have unwanted side effects.  And not konwing
exactly how Puneet's machine is setup one should make it
explicit that the above will match a lot more then you
possibly intend.

8.8.8.8[0-9]

This being just an example.  The first octet having the same
natural problem though that's less likely to happen.
You can fix the above problem be either anchor'ing your
match or using the appropriate operator '=='.

~Paul

________________________________

This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.



From phk at phk.freebsd.dk  Sun Jul 28 07:12:02 2013
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sun, 28 Jul 2013 07:12:02 +0000
Subject: Stop users accessing website via IP address
In-Reply-To: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
Message-ID: <4258.1374995522@critter.freebsd.dk>

In message <000f01ce8b0a$a872c230$f9584690$@insticator.com>, Puneet writes:

>I want to stop the users accessing my website via IP address.
>I am using varnish as cache.
>I have the following code in place but it is not working.
>
>In vcl_recv() {
>     if(req.url ~ "XX.XX.XXX.XXX") {

You should check the req.http.host header, that's where you will
find the IP# or hostname the user used.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From puneet.arora at insticator.com  Sun Jul 28 14:20:36 2013
From: puneet.arora at insticator.com (Puneet)
Date: Sun, 28 Jul 2013 10:20:36 -0400
Subject: Stop users accessing website via IP address
In-Reply-To: <4258.1374995522@critter.freebsd.dk>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
 <4258.1374995522@critter.freebsd.dk>
Message-ID: <000a01ce8b9d$a1dc9fc0$e595df40$@insticator.com>

Hi All,

Apologies for all this confusion and chaos.

I just don't want my website to be accessible by using the IP address on
which the website is hosted. Even if the Users use the IP address, I want
them to be redirected to my website.

It is the same way as www.facebook .com is hosted at "173.252.110.27".
When a user types "173.252.110.27" in the URL he is redirected to
"www.facebook.com"

I want to have the same functionality for mywebsite also.

To have this functionality I was trying Varnish to redirect the Users to
www.mywebsite.com .

Thanks
Puneet

-----Original Message-----
From: Poul-Henning Kamp [mailto:phk at phk.freebsd.dk] 
Sent: Sunday, July 28, 2013 3:12 AM
To: Puneet
Cc: varnish-misc at varnish-cache.org
Subject: Re: Stop users accessing website via IP address

In message <000f01ce8b0a$a872c230$f9584690$@insticator.com>, Puneet writes:

>I want to stop the users accessing my website via IP address.
>I am using varnish as cache.
>I have the following code in place but it is not working.
>
>In vcl_recv() {
>     if(req.url ~ "XX.XX.XXX.XXX") {

You should check the req.http.host header, that's where you will find the
IP# or hostname the user used.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.




From puneet.arora at insticator.com  Sun Jul 28 16:58:17 2013
From: puneet.arora at insticator.com (Puneet)
Date: Sun, 28 Jul 2013 12:58:17 -0400
Subject: Stop users accessing website via IP address
In-Reply-To: <000a01ce8b9d$a1dc9fc0$e595df40$@insticator.com>
References: <000f01ce8b0a$a872c230$f9584690$@insticator.com>
 <4258.1374995522@critter.freebsd.dk>
 <000a01ce8b9d$a1dc9fc0$e595df40$@insticator.com>
Message-ID: <001f01ce8bb3$a951a4b0$fbf4ee10$@insticator.com>

Hi All,

Thanks for all the time.
I am able to get that working.
I was just missing :
	set obj.http.Location = "http://mywebsite.com"+req.url;

Thanks
Puneet 


-----Original Message-----
From: Puneet [mailto:puneet.arora at insticator.com] 
Sent: Sunday, July 28, 2013 10:21 AM
To: 'Poul-Henning Kamp'; 'Travis Crowder'; 'Paul A. Procacci'
Cc: varnish-misc at varnish-cache.org
Subject: RE: Stop users accessing website via IP address

Hi All,

Apologies for all this confusion and chaos.

I just don't want my website to be accessible by using the IP address on
which the website is hosted. Even if the Users use the IP address, I want
them to be redirected to my website.

It is the same way as www.facebook .com is hosted at "173.252.110.27".
When a user types "173.252.110.27" in the URL he is redirected to
"www.facebook.com"

I want to have the same functionality for mywebsite also.

To have this functionality I was trying Varnish to redirect the Users to
www.mywebsite.com .

Thanks
Puneet

-----Original Message-----
From: Poul-Henning Kamp [mailto:phk at phk.freebsd.dk]
Sent: Sunday, July 28, 2013 3:12 AM
To: Puneet
Cc: varnish-misc at varnish-cache.org
Subject: Re: Stop users accessing website via IP address

In message <000f01ce8b0a$a872c230$f9584690$@insticator.com>, Puneet writes:

>I want to stop the users accessing my website via IP address.
>I am using varnish as cache.
>I have the following code in place but it is not working.
>
>In vcl_recv() {
>     if(req.url ~ "XX.XX.XXX.XXX") {

You should check the req.http.host header, that's where you will find the
IP# or hostname the user used.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.





From rlane at ahbelo.com  Mon Jul 29 12:30:21 2013
From: rlane at ahbelo.com (Lane, Richard)
Date: Mon, 29 Jul 2013 07:30:21 -0500
Subject: varnish-misc Digest, Vol 88, Issue 17
In-Reply-To: <mailman.4115.1374979112.2301.varnish-misc@varnish-cache.org>
References: <mailman.4115.1374979112.2301.varnish-misc@varnish-cache.org>
Message-ID: <CADt-aG2jZX=VUvwkgooeUJ_HLqBpSE8m2ucguXcbcM4f_Lb50w@mail.gmail.com>

I am not sure if it is my email but it looks like you are passing HTML in
the REDIRECT URL.

The REDIRECT URL should be just a URL. Try a simple URL like
http://www.yahoo.com first. Then work on your regsub statement to set the
right URL.



On Sat, Jul 27, 2013 at 9:38 PM, <varnish-misc-request at varnish-cache.org>wrote:

> Send varnish-misc mailing list submissions to
>         varnish-misc at varnish-cache.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> or, via email, send a message with subject or body 'help' to
>         varnish-misc-request at varnish-cache.org
>
> You can reach the person managing the list at
>         varnish-misc-owner at varnish-cache.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of varnish-misc digest..."
>
>
> Today's Topics:
>
>    1. Re: Stop users accessing website via IP address
>       (Hugo Cisneiros (Eitch))
>    2. Re: Stop users accessing website via IP address (Travis Crowder)
>    3. RE: Stop users accessing website via IP address (Puneet)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 27 Jul 2013 18:28:57 -0300
> From: "Hugo Cisneiros (Eitch)" <hugo.cisneiros at gmail.com>
> To: "varnish-misc at varnish-cache.org" <varnish-misc at varnish-cache.org>
> Subject: Re: Stop users accessing website via IP address
> Message-ID:
>         <CA+KACLkCCfpK2XmM4-vaxOW2b=
> XBL-nbgadgt7FE3XejfWA6cw at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Sat, Jul 27, 2013 at 5:48 PM, Puneet <puneet.arora at insticator.com>
> wrote:
>
> > I want to stop the users accessing my website via IP address.
> >
> > I am using varnish as cache.
> > I have the following code in place but it is not working.
> >
> > In vcl_recv() {
> >      if(req.url ~ "XX.XX.XXX.XXX") {
> >      error 750 "Moved Permanently";
> >   } }
> >
>
> In vcl_recv, you're comparting the IP address with the request URL
> (req.url), which is wrong. You should compare with client.ip, as it
> represents the user's IP address.
>
> Anyway, a much better approach in my opinion is the code:
>
> # list of forbidden ips
> acl forbidden {
>   "192.168.0.1",
>   "192.168.0.2",
>   "XXX.XXX.XXX.XXX"
> }
>
> sub vcl_recv {
>   if (client.ip ~ forbidden) {
>     error 301 "http://mywebsite.com";
>   }
> }
>
> sub vcl_error {
>   set obj.http.Content-Type = "text/html; charset=utf-8";
>   set obj.http.Retry-After = "5";
>
>   # we deal with redirects here
>   if (obj.status == 301) {
>     set obj.http.Location = obj.response;
>     set obj.response = "Moved Temporarily";
>     return (deliver);
>   }
>
>   if (obj.status == 301){
>     set obj.http.Location = obj.response;
>     set obj.response = "Moved Permanently";
>     return (deliver);
>   }
> }
>
> This way you can update the ACL to multiple IP addresses and they'll be all
> redirected to mywebsite.com.
>
> --
> []'s
> Hugo
> www.devin.com.br
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20130727/8ceb19b3/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Sat, 27 Jul 2013 18:53:11 -0500
> From: Travis Crowder <travis.crowder at spechal.com>
> To: Puneet <puneet.arora at insticator.com>
> Cc: varnish-misc at varnish-cache.org
> Subject: Re: Stop users accessing website via IP address
> Message-ID: <A89915AA-A7FE-432D-90CD-E8B6BF12A105 at spechal.com>
> Content-Type: text/plain; charset="us-ascii"
>
> Check against req.http.Host
>
> In vcl_recv:
>
>      if(req.http.Host ~ "8.8.8.8") {
>          error 750;
>      }
>
> -Travis Crowder
>
> On Jul 27, 2013, at 3:48 PM, Puneet <puneet.arora at insticator.com> wrote:
>
> > Hi all,
> >
> > I want to stop the users accessing my website via IP address.
> > I am using varnish as cache.
> > I have the following code in place but it is not working.
> >
> > In vcl_recv() {
> >      if(req.url ~ "XX.XX.XXX.XXX") {
> >      error 750 "Moved Permanently";
> >   } }
> >
> > And in vcl_error()
> > sub vcl_error {
> >     if (obj.status == 750) {
> >     set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
> http://mywebsite.com");
> >     set obj.http.Location = req.http.X-REDIRURL;
> >     set obj.status = 301;
> >     unset req.http.X-REDIRURL;
> >     return(deliver);
> > }
> >
> > But this does not redirect the user to the website, instead it delivers
> the page.
> > Can anyone tell what I am missing?
> >
> > Thanks
> > Puneet
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20130727/da02d31d/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Sat, 27 Jul 2013 22:38:24 -0400
> From: Puneet <puneet.arora at insticator.com>
> To: "'Travis Crowder'" <travis.crowder at spechal.com>
> Cc: varnish-misc at varnish-cache.org
> Subject: RE: Stop users accessing website via IP address
> Message-ID: <005701ce8b3b$88e6ee10$9ab4ca30$@insticator.com>
> Content-Type: text/plain; charset="us-ascii"
>
> HI Travis,
>
>
>
> Thanks for the reply.
>
> I think that should work.
>
>
>
> Just one question.
>
> In sub vcl_error() should I also change the
>
>
>
> set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
> <
> http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
> qRF&_t=tracking_anet> http://mywebsite.com");
>
> TO : -->
>
> set req.http.X-REDIRURL = regsub(req.http.host,"https?://[^/$]+", "
> <
> http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
> qRF&_t=tracking_anet> http://mywebsite.com");
>
> ?
>
>
>
> Because when replace req.url with req.http.host, It again stops working.
>
> And If I don't do it, the bowser gives an error "Too many redirects"
>
>
>
> Thanks
>
> Puneet
>
>
>
> From: Travis Crowder [mailto:travis.crowder at spechal.com]
> Sent: Saturday, July 27, 2013 7:53 PM
> To: Puneet
> Cc: varnish-misc at varnish-cache.org
> Subject: Re: Stop users accessing website via IP address
>
>
>
> Check against req.http.Host
>
>
>
> In vcl_recv:
>
>
>
>      if(req.http.Host ~ "8.8.8.8") {
>
>          error 750;
>
>      }
>
>
>
> -Travis Crowder
>
>
>
> On Jul 27, 2013, at 3:48 PM, Puneet <puneet.arora at insticator.com
> <mailto:puneet.arora at insticator.com> > wrote:
>
>
>
>
>
> Hi all,
>
>
> I want to stop the users accessing my website via IP address.
> I am using varnish as cache.
> I have the following code in place but it is not working.
>
> In vcl_recv() {
>      if(req.url ~ "XX.XX.XXX.XXX") {
>      error 750 "Moved Permanently";
>   } }
>
> And in vcl_error()
> sub vcl_error {
>     if (obj.status == 750) {
>     set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
> <
> http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
> qRF&_t=tracking_anet> http://mywebsite.com");
>     set obj.http.Location = req.http.X-REDIRURL;
>     set obj.status = 301;
>     unset req.http.X-REDIRURL;
>     return(deliver);
> }
>
> But this does not redirect the user to the website, instead it delivers the
> page.
> Can anyone tell what I am missing?
>
>
>
> Thanks
>
> Puneet
>
> _______________________________________________
> varnish-misc mailing list
>  <mailto:varnish-misc at varnish-cache.org> varnish-misc at varnish-cache.org
>  <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20130727/21c9f677/attachment.html
> >
>
> ------------------------------
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
> End of varnish-misc Digest, Vol 88, Issue 17
> ********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130729/ea504e3d/attachment.html>

From dharrigan at gmail.com  Mon Jul 29 13:24:43 2013
From: dharrigan at gmail.com (David Harrigan)
Date: Mon, 29 Jul 2013 14:24:43 +0100
Subject: Varnish pipe through for SSL requests
In-Reply-To: <CACj2-4KxHpWFSEuAOGYogbQw7zc0Nxt-Ng08UG2F9f6xb3+Y6Q@mail.gmail.com>
References: <2691037.EI24baZqZ7@hierro>
 <192D29F2D6C34655A14B0C23F87CD1F7@NeiRoze>
 <3996460.HSvvyf2783@hierro>
 <CACj2-4KxHpWFSEuAOGYogbQw7zc0Nxt-Ng08UG2F9f6xb3+Y6Q@mail.gmail.com>
Message-ID: <CAKdGheMrDDcRUVXrAafCJOqea=swanWoRMfw6gbME0PchMrOoA@mail.gmail.com>

Hi,

Our approach is to terminate using Pound (http://www.apsis.ch/pound), then
to pass on to Varnish. It works *wonderfully* well and is super easy to
configure.

-=david=-


On 26 July 2013 02:22, Norberto Meijome <numard at gmail.com> wrote:

> You should be able to with modproxy.. We terminate on nginx which acts as
> proxy for clusters of app servers and varnishes...just tell nginx to
> connect to varnish over http.
> On 26/07/2013 5:27 AM, "Yari Shima" <yarishima42 at googlemail.com> wrote:
>
>> Hi Reinis,
>>
>> Thanks for your awnser.
>> But can't I use apache to listen on port 443 on my root server and with
>> mod_proxy pipr the traffic through to my managed server?
>>
>> Best
>> YS
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
I prefer encrypted and signed messages. KeyID: B20A22F9
Fingerprint: 110A F423 3647 54E2 880F ADAD 1C52 85BF B20A 22F9

"It is not usually until you've built and used a version of the program
that you understand the issues well enough to get the design right." - Rob
Pike, Brian Kernighan.

No trees were harmed in the sending of this message, however, a number of
electrons were inconvenienced.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130729/7bbd1a7b/attachment.html>

From puneet.arora at insticator.com  Mon Jul 29 14:13:00 2013
From: puneet.arora at insticator.com (Puneet)
Date: Mon, 29 Jul 2013 10:13:00 -0400
Subject: varnish-misc Digest, Vol 88, Issue 17
In-Reply-To: <CADt-aG2jZX=VUvwkgooeUJ_HLqBpSE8m2ucguXcbcM4f_Lb50w@mail.gmail.com>
References: <mailman.4115.1374979112.2301.varnish-misc@varnish-cache.org>
 <CADt-aG2jZX=VUvwkgooeUJ_HLqBpSE8m2ucguXcbcM4f_Lb50w@mail.gmail.com>
Message-ID: <002b01ce8c65$bc5e3840$351aa8c0$@insticator.com>

Hi Richard,

 

I am able to do have the desired functionality.

I was missing a piece. The issue is resolved.

 

Thanks

Puneet

 

From: varnish-misc-bounces+puneet.arora=insticator.com at varnish-cache.org
[mailto:varnish-misc-bounces+puneet.arora=insticator.com at varnish-cache.org]
On Behalf Of Lane, Richard
Sent: Monday, July 29, 2013 8:30 AM
To: varnish-misc at varnish-cache.org
Subject: Re: varnish-misc Digest, Vol 88, Issue 17

 

I am not sure if it is my email but it looks like you are passing HTML in
the REDIRECT URL.

 

The REDIRECT URL should be just a URL. Try a simple URL like
http://www.yahoo.com first. Then work on your regsub statement to set the
right URL.

 

 

On Sat, Jul 27, 2013 at 9:38 PM, <varnish-misc-request at varnish-cache.org
<mailto:varnish-misc-request at varnish-cache.org> > wrote:

Send varnish-misc mailing list submissions to
        varnish-misc at varnish-cache.org
<mailto:varnish-misc at varnish-cache.org> 

To subscribe or unsubscribe via the World Wide Web, visit
        https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
or, via email, send a message with subject or body 'help' to
        varnish-misc-request at varnish-cache.org
<mailto:varnish-misc-request at varnish-cache.org> 

You can reach the person managing the list at
        varnish-misc-owner at varnish-cache.org
<mailto:varnish-misc-owner at varnish-cache.org> 

When replying, please edit your Subject line so it is more specific
than "Re: Contents of varnish-misc digest..."


Today's Topics:

   1. Re: Stop users accessing website via IP address
      (Hugo Cisneiros (Eitch))
   2. Re: Stop users accessing website via IP address (Travis Crowder)
   3. RE: Stop users accessing website via IP address (Puneet)


----------------------------------------------------------------------

Message: 1
Date: Sat, 27 Jul 2013 18:28:57 -0300
From: "Hugo Cisneiros (Eitch)" <hugo.cisneiros at gmail.com
<mailto:hugo.cisneiros at gmail.com> >
To: "varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org>
" <varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org> >
Subject: Re: Stop users accessing website via IP address
Message-ID:
        <CA+KACLkCCfpK2XmM4-vaxOW2b=XBL-nbgadgt7FE3XejfWA6cw at mail.gmail.com
<mailto:XBL-nbgadgt7FE3XejfWA6cw at mail.gmail.com> >
Content-Type: text/plain; charset="iso-8859-1"

On Sat, Jul 27, 2013 at 5:48 PM, Puneet <puneet.arora at insticator.com
<mailto:puneet.arora at insticator.com> > wrote:

> I want to stop the users accessing my website via IP address.
>
> I am using varnish as cache.
> I have the following code in place but it is not working.
>
> In vcl_recv() {
>      if(req.url ~ "XX.XX.XXX.XXX") {
>      error 750 "Moved Permanently";
>   } }
>

In vcl_recv, you're comparting the IP address with the request URL
(req.url), which is wrong. You should compare with client.ip, as it
represents the user's IP address.

Anyway, a much better approach in my opinion is the code:

# list of forbidden ips
acl forbidden {
  "192.168.0.1",
  "192.168.0.2",
  "XXX.XXX.XXX.XXX"
}

sub vcl_recv {
  if (client.ip ~ forbidden) {
    error 301 "http://mywebsite.com";
  }
}

sub vcl_error {
  set obj.http.Content-Type = "text/html; charset=utf-8";
  set obj.http.Retry-After = "5";

  # we deal with redirects here
  if (obj.status == 301) {
    set obj.http.Location = obj.response;
    set obj.response = "Moved Temporarily";
    return (deliver);
  }

  if (obj.status == 301){
    set obj.http.Location = obj.response;
    set obj.response = "Moved Permanently";
    return (deliver);
  }
}

This way you can update the ACL to multiple IP addresses and they'll be all
redirected to mywebsite.com <http://mywebsite.com> .

--
[]'s
Hugo
www.devin.com.br <http://www.devin.com.br> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/2013
0727/8ceb19b3/attachment-0001.html>

------------------------------

Message: 2
Date: Sat, 27 Jul 2013 18:53:11 -0500
From: Travis Crowder <travis.crowder at spechal.com
<mailto:travis.crowder at spechal.com> >
To: Puneet <puneet.arora at insticator.com <mailto:puneet.arora at insticator.com>
>
Cc: varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org> 
Subject: Re: Stop users accessing website via IP address
Message-ID: <A89915AA-A7FE-432D-90CD-E8B6BF12A105 at spechal.com
<mailto:A89915AA-A7FE-432D-90CD-E8B6BF12A105 at spechal.com> >
Content-Type: text/plain; charset="us-ascii"

Check against req.http.Host

In vcl_recv:

     if(req.http.Host ~ "8.8.8.8") {
         error 750;
     }

-Travis Crowder

On Jul 27, 2013, at 3:48 PM, Puneet <puneet.arora at insticator.com
<mailto:puneet.arora at insticator.com> > wrote:

> Hi all,
>
> I want to stop the users accessing my website via IP address.
> I am using varnish as cache.
> I have the following code in place but it is not working.
>
> In vcl_recv() {
>      if(req.url ~ "XX.XX.XXX.XXX") {
>      error 750 "Moved Permanently";
>   } }
>
> And in vcl_error()
> sub vcl_error {
>     if (obj.status == 750) {
>     set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+",
"http://mywebsite.com");
>     set obj.http.Location = req.http.X-REDIRURL;
>     set obj.status = 301;
>     unset req.http.X-REDIRURL;
>     return(deliver);
> }
>
> But this does not redirect the user to the website, instead it delivers
the page.
> Can anyone tell what I am missing?
>
> Thanks
> Puneet
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org> 
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/2013
0727/da02d31d/attachment-0001.html>

------------------------------

Message: 3
Date: Sat, 27 Jul 2013 22:38:24 -0400
From: Puneet <puneet.arora at insticator.com
<mailto:puneet.arora at insticator.com> >
To: "'Travis Crowder'" <travis.crowder at spechal.com
<mailto:travis.crowder at spechal.com> >
Cc: varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org> 
Subject: RE: Stop users accessing website via IP address
Message-ID: <005701ce8b3b$88e6ee10$9ab4ca30$@insticator.com
<http://insticator.com> >
Content-Type: text/plain; charset="us-ascii"

HI Travis,



Thanks for the reply.

I think that should work.



Just one question.

In sub vcl_error() should I also change the



set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> &urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");

TO : -->

set req.http.X-REDIRURL = regsub(req.http.host,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> &urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");

?



Because when replace req.url with req.http.host, It again stops working.

And If I don't do it, the bowser gives an error "Too many redirects"



Thanks

Puneet



From: Travis Crowder [mailto:travis.crowder at spechal.com
<mailto:travis.crowder at spechal.com> ]
Sent: Saturday, July 27, 2013 7:53 PM
To: Puneet
Cc: varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org> 
Subject: Re: Stop users accessing website via IP address



Check against req.http.Host



In vcl_recv:



     if(req.http.Host ~ "8.8.8.8") {

         error 750;

     }



-Travis Crowder



On Jul 27, 2013, at 3:48 PM, Puneet <puneet.arora at insticator.com
<mailto:puneet.arora at insticator.com> 
<mailto:puneet.arora at insticator.com <mailto:puneet.arora at insticator.com> > >
wrote:





Hi all,


I want to stop the users accessing my website via IP address.
I am using varnish as cache.
I have the following code in place but it is not working.

In vcl_recv() {
     if(req.url ~ "XX.XX.XXX.XXX") {
     error 750 "Moved Permanently";
  } }

And in vcl_error()
sub vcl_error {
    if (obj.status == 750) {
    set req.http.X-REDIRURL = regsub(req.url,"https?://[^/$]+", "
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom
<http://www.linkedin.com/redirect?url=http%3A%2F%2Fmywebsite%2Ecom&urlhash=5
qRF&_t=tracking_anet> &urlhash=5
qRF&_t=tracking_anet> http://mywebsite.com");
    set obj.http.Location = req.http.X-REDIRURL;
    set obj.status = 301;
    unset req.http.X-REDIRURL;
    return(deliver);
}

But this does not redirect the user to the website, instead it delivers the
page.
Can anyone tell what I am missing?



Thanks

Puneet

_______________________________________________
varnish-misc mailing list
 <mailto:varnish-misc at varnish-cache.org
<mailto:varnish-misc at varnish-cache.org> > varnish-misc at varnish-cache.org
<mailto:varnish-misc at varnish-cache.org> 
 <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/2013
0727/21c9f677/attachment.html>

------------------------------

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org> 
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

End of varnish-misc Digest, Vol 88, Issue 17
********************************************

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130729/58693394/attachment.html>

From gabster at lelutin.ca  Mon Jul 29 18:12:23 2013
From: gabster at lelutin.ca (Gabriel Filion)
Date: Mon, 29 Jul 2013 14:12:23 -0400
Subject: Varnish pipe through for SSL requests
In-Reply-To: <CAKdGheMrDDcRUVXrAafCJOqea=swanWoRMfw6gbME0PchMrOoA@mail.gmail.com>
References: <2691037.EI24baZqZ7@hierro>
 <192D29F2D6C34655A14B0C23F87CD1F7@NeiRoze> <3996460.HSvvyf2783@hierro>
 <CACj2-4KxHpWFSEuAOGYogbQw7zc0Nxt-Ng08UG2F9f6xb3+Y6Q@mail.gmail.com>
 <CAKdGheMrDDcRUVXrAafCJOqea=swanWoRMfw6gbME0PchMrOoA@mail.gmail.com>
Message-ID: <51F6B087.7040607@lelutin.ca>

Hi there,

On 29/07/13 09:24 AM, David Harrigan wrote:
> Our approach is to terminate using Pound (http://www.apsis.ch/pound),
> then to pass on to Varnish. It works *wonderfully* well and is super
> easy to configure.

Please note that if it is setup that way with the infrastructure that
the OP described (e.g. caching needs to be on another server than the
web server), then it means that your clients who are using an encrypted
connection to your site will have their traffic pass over the internet
unencrypted between the caching node and the web server.

that's usually very bad security-wise because as a client if you use
encryption, you expect that any sensitive data passed to a site stays
encrypted over the network and that only that website can gain access to
the sensitive data. if traffic goes through the net unencrypted, then
that assumption is completely false.


in that case, you can either:

 * consider moving your web hosting to your other server that hosts
varnish, if you feel up to the challenge of managing your own web server.
 * or find some way to reencrypt traffic between the caching and the web
server.


for the 2nd option, the easiest would be to setup an encryption tunnel
(like a VPN) between both servers and use the tunnel exclusively to
communicate between varnish and the web server.

> On 26 July 2013 02:22, Norberto Meijome <numard at gmail.com
> <mailto:numard at gmail.com>> wrote:
> 
>     You should be able to with modproxy.. We terminate on nginx which
>     acts as proxy for clusters of app servers and varnishes...just tell
>     nginx to connect to varnish over http.
> 
>     On 26/07/2013 5:27 AM, "Yari Shima" <yarishima42 at googlemail.com
>     <mailto:yarishima42 at googlemail.com>> wrote:
> 
>         Hi Reinis,
> 
>         Thanks for your awnser.
>         But can't I use apache to listen on port 443 on my root server
>         and with
>         mod_proxy pipr the traffic through to my managed server?


-- 
Gabriel Filion

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 291 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20130729/25fc575f/attachment.bin>

From james at talkunafraid.co.uk  Wed Jul 31 14:50:59 2013
From: james at talkunafraid.co.uk (James Harrison)
Date: Wed, 31 Jul 2013 15:50:59 +0100
Subject: Sudden hit_for_pass
Message-ID: <51F92453.6060103@talkunafraid.co.uk>

Hi,

I have a varnish server sat in front of a website which was previously
sitting at 60% hitrate, no hit_for_pass. With no configuration change
aside from adding a single ban rule (which I have since removed) keyed
to a hostname the site runs (but which accounts for a tiny fraction of
the traffic), this has gone to 1% hitrate, 99% hit_for_pass. Headers
being sent out suggest caching is still behaving as expected but the shm
tools all report this high hit_for_pass rate. Looking at the number of
objects, there's now nothing but object heads in the store.

Any thoughts on what could be going on here?

-- 
Cheers,
James Harrison