From des at linpro.no  Wed May  2 15:26:27 2007
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Wed, 02 May 2007 17:26:27 +0200
Subject: Packaging varnish for Fedora
In-Reply-To: <1177527114.10357.9.camel@re.e37> (Ingvar Hagelund's message of
	"Wed, 25 Apr 2007 20:51:54 +0200")
References: <1177527114.10357.9.camel@re.e37>
Message-ID: <ujrwszrxn4s.fsf@false.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> Do you have a simple startup configuration that I can just use? Should I
> involve varnishncsa to make those nice apache-like logs? Can they work
> automatically together, with a named pipe or something like that?

No configuration is needed - varnishlog should "just work" as long as
varnishd is running.

I would recommend simply running 'varnishlog -w /var/log/varnish.log'
to store raw log data.  It will reopen the file if it receives a HUP,
so it plays nice with logrotate / newsyslog.

There is no need to run any of the other logging tools, as they can
all work on previously stored raw log data.  For instance, you can run
'varnishncsa -r /var/log/varnish.log' at any time to produce NCSA logs
which analog can read, etc.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Wed May  2 15:29:59 2007
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Wed, 02 May 2007 17:29:59 +0200
Subject: Planning for 1.0.4
In-Reply-To: <ujrvefdx2b4.fsf@false.linpro.no> (Dag-Erling
	=?iso-8859-1?Q?Sm=F8rgrav's?= message
	of "Mon, 30 Apr 2007 18:19:27 +0200")
References: <ujrvefdx2b4.fsf@false.linpro.no>
Message-ID: <ujrslafxmyw.fsf@false.linpro.no>

des at linpro.no (Dag-Erling Sm?rgrav) writes:
> There are a number of improvements in trunk (and branches/1.0) which I
> would like to release as 1.0.4.
>
> I am thinking primarily of trunk as of r1288 (or branches/1.0 as of
> r1331, no thanks to svk).  However, I'd like to ask you if there are
> any newer changes you think should be included.  I think r1364-r1369
> (autoconf cleanup + missing include) are good candidates.  So are the
> varnishncsa changes I made last week, but they need a shakedown.

I've gone through the entire changelog since r1288, and have concluded
that I want all of it in 1.0.4.  Apart from the addition of inline C
syntax to VCL and the ping_interval parameter, it's all bug fixes and
increased reliability / robustness.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Wed May  9 15:12:56 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Wed, 09 May 2007 17:12:56 +0200
Subject: Planning for 1.0.4
In-Reply-To: <ujrslafxmyw.fsf@false.linpro.no> ("Dag-Erling =?utf-8?Q?Sm?=
	=?utf-8?Q?=C3=B8rgrav=22's?=
	message of "Wed\, 02 May 2007 17\:29\:59 +0200")
References: <ujrvefdx2b4.fsf@false.linpro.no> <ujrslafxmyw.fsf@false.linpro.no>
Message-ID: <87ejlqvxmv.fsf@des.linpro.no>

des at linpro.no (Dag-Erling Sm?rgrav) writes:
> I've gone through the entire changelog since r1288, and have concluded
> that I want all of it in 1.0.4.  Apart from the addition of inline C
> syntax to VCL and the ping_interval parameter, it's all bug fixes and
> increased reliability / robustness.

A little further down the road, there are additional bug fixes I want to
bring in: everything up to and including r1387, plus r1399.  I'll merge
these to branches/1.0 as soon as I feel confident about svnmerge.

I am also working on a set of guidelines for packaging Varnish.  I would
like Varnish to ship with the same default configuration on all OSes /
distributions, so we won't have to guess when a user comes to us with a
problem.  I have created a Wiki page for these guidelines:

  http://varnish.projects.linpro.no/wiki/OutOfTheBox

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Wed May  9 18:55:40 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Wed, 09 May 2007 20:55:40 +0200
Subject: Planning for 1.0.4
In-Reply-To: <ujrvefdx2b4.fsf@false.linpro.no>
References: <ujrvefdx2b4.fsf@false.linpro.no>
Message-ID: <1178736940.15999.9.camel@re.e37>

* Dag-Erling Sm?rgrav
> Other priorities for 1.0.4: unify init scripts and out-of-the-box
> configurations across distributions

I think a unified init script will be very difficult to achieve if one
wants to keep the package included in mainstream Linux distributions
like Fedora/RedHat, SuSE or Debian. They have explicit rules on what an
init script should look like, and they are by no means compatible,
leaning on style, tools and macros that are very distribution specific.

A common configuration file, stored in /etc/sysconfig/varnish (RedHat,
SuSE) or /etc/defaults/varnish (Debian) would do nicely. The existing
Debian and Fedora versions are not too different.

> (...) include an init script for varnishlog

RedHat/Fedora allows several daemons started from a single init script.
Consider RedHat's syslog script, that starts both syslogd and klogd.

Ingvar

-- 



From ingvar at linpro.no  Wed May  9 19:52:49 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Wed, 09 May 2007 21:52:49 +0200
Subject: Packaging varnish for Fedora
In-Reply-To: <ujrwszrxn4s.fsf@false.linpro.no>
References: <1177527114.10357.9.camel@re.e37> <ujrwszrxn4s.fsf@false.linpro.no>
Message-ID: <1178740369.15999.13.camel@re.e37>

* Dag-Erling Sm?rgrav
> I would recommend simply running 'varnishlog -w /var/log/varnish.log'
> to store raw log data.  It will reopen the file if it receives a HUP,
> so it plays nice with logrotate / newsyslog.

Right. Adding '-a' not to overwrite old logs. 

It seems that varnishlog does not daemonize itself. Is it a lot of pain
to add some code to make it do this? The RedHat init scripts depend on
daemons daemonizing themselves.

Ingvar

-- 



From des at linpro.no  Thu May 10 07:34:07 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Thu, 10 May 2007 09:34:07 +0200
Subject: Planning for 1.0.4
In-Reply-To: <1178736940.15999.9.camel@re.e37> (Ingvar Hagelund's message of
	"Wed\, 09 May 2007 20\:55\:40 +0200")
References: <ujrvefdx2b4.fsf@false.linpro.no> <1178736940.15999.9.camel@re.e37>
Message-ID: <876471w2s0.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> * Dag-Erling Sm?rgrav
> > Other priorities for 1.0.4: unify init scripts and out-of-the-box
> > configurations across distributions
> I think a unified init script will be very difficult to achieve if one
> wants to keep the package included in mainstream Linux distributions
> like Fedora/RedHat, SuSE or Debian. They have explicit rules on what
> an init script should look like, and they are by no means compatible,
> leaning on style, tools and macros that are very distribution
> specific.

I'm not after "a single unified init script", but "appropriate init
scripts for each distribution which behave in as similar a manner as
possible".

> > (...) include an init script for varnishlog
> RedHat/Fedora allows several daemons started from a single init script.
> Consider RedHat's syslog script, that starts both syslogd and klogd.

What if the admin wants to stop varnishlog (which can produce enormous
amounts of data on high-traffic sites), or changes the varnishlog
configuration and wants to restart just varnishlog but not varnishd?

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Thu May 10 07:37:29 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Thu, 10 May 2007 09:37:29 +0200
Subject: Packaging varnish for Fedora
In-Reply-To: <1178740369.15999.13.camel@re.e37> (Ingvar Hagelund's message of
	"Wed\, 09 May 2007 21\:52\:49 +0200")
References: <1177527114.10357.9.camel@re.e37>
	<ujrwszrxn4s.fsf@false.linpro.no> <1178740369.15999.13.camel@re.e37>
Message-ID: <871whpw2me.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> * Dag-Erling Sm?rgrav
> > I would recommend simply running 'varnishlog -w /var/log/varnish.log'
> > to store raw log data.  It will reopen the file if it receives a HUP,
> > so it plays nice with logrotate / newsyslog.
> Right. Adding '-a' not to overwrite old logs.

Good point, thanks.

> It seems that varnishlog does not daemonize itself. Is it a lot of pain
> to add some code to make it do this? The RedHat init scripts depend on
> daemons daemonizing themselves.

No, it should be easy to do.  I also need to add support for a pid
file.  Since varnishd already has a -p option, I think I'll use -P for
both varnishd and varnishlog.

Is it OK to daemonize if and only if a pid file was specified, or should
we have a separate option (-D?) for that?

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Thu May 10 09:14:58 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Thu, 10 May 2007 11:14:58 +0200
Subject: Packaging varnish for Fedora
In-Reply-To: <871whpw2me.fsf@des.linpro.no>
References: <1177527114.10357.9.camel@re.e37>	<ujrwszrxn4s.fsf@false.linpro.no>
	<1178740369.15999.13.camel@re.e37> <871whpw2me.fsf@des.linpro.no>
Message-ID: <4642E292.8010506@linpro.no>

* Ingvar Hagelund
>> It seems that varnishlog does not daemonize itself. Is it a lot of pain
>> to add some code to make it do this? 

* Dag-Erling Sm?rgrav
> No, it should be easy to do (...) Is it OK to daemonize if and only if a
> pid file was specified, or should we have a separate option (-D?)
> for that?

If it's not to much hassle, I would prefer an option for each.
Ingvar

-- 
N?r alt annet feiler: Symlink


From ingvar at linpro.no  Thu May 10 09:24:29 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Thu, 10 May 2007 11:24:29 +0200
Subject: Planning for 1.0.4
In-Reply-To: <876471w2s0.fsf@des.linpro.no>
References: <ujrvefdx2b4.fsf@false.linpro.no>	<1178736940.15999.9.camel@re.e37>
	<876471w2s0.fsf@des.linpro.no>
Message-ID: <4642E4CD.9050401@linpro.no>

* Dag-Erling Sm?rgrav
>>> Other priorities for 1.0.4: unify init scripts and out-of-the-box
>>> configurations across distributions (...)

> I'm not after "a single unified init script", but "appropriate init
> scripts for each distribution which behave in as similar a manner as
> possible".

I've put my latest initrc and sysconfig scripts at
http://users.linpro.no/ingvar/varnish/fedora-extras-commit/. Lars and
Stig, could you comment on them, please.

Also, consider to glance over the Fedora review request at
http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=230275

> (...) What if the admin wants to stop varnishlog (which can produce enormous
> amounts of data on high-traffic sites), or changes the varnishlog
> configuration and wants to restart just varnishlog but not varnishd?

Reasonable. One might also consider not running varnishlog at all. Two
scripts it is, then.

Ingvar

-- 
N?r alt annet feiler: Symlink


From des at linpro.no  Thu May 10 09:43:55 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Thu, 10 May 2007 11:43:55 +0200
Subject: Packaging varnish for Fedora
In-Reply-To: <4642E292.8010506@linpro.no> (Ingvar Hagelund's message of "Thu\, 
	10 May 2007 11\:14\:58 +0200")
References: <1177527114.10357.9.camel@re.e37>
	<ujrwszrxn4s.fsf@false.linpro.no> <1178740369.15999.13.camel@re.e37>
	<871whpw2me.fsf@des.linpro.no> <4642E292.8010506@linpro.no>
Message-ID: <87ps59ypwk.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> * Dag-Erling Sm?rgrav
> > Is it OK to daemonize if and only if a pid file was specified, or
> > should we have a separate option (-D?)  for that?
> If it's not to much hassle, I would prefer an option for each.

OK, will do.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Thu May 10 09:53:07 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Thu, 10 May 2007 11:53:07 +0200
Subject: Planning for 1.0.4
In-Reply-To: <4642E4CD.9050401@linpro.no> (Ingvar Hagelund's message of "Thu\, 
	10 May 2007 11\:24\:29 +0200")
References: <ujrvefdx2b4.fsf@false.linpro.no>
	<1178736940.15999.9.camel@re.e37> <876471w2s0.fsf@des.linpro.no>
	<4642E4CD.9050401@linpro.no>
Message-ID: <87hcqlyph8.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> I've put my latest initrc and sysconfig scripts at
> http://users.linpro.no/ingvar/varnish/fedora-extras-commit/. Lars and
> Stig, could you comment on them, please.

These can be used unmodified for RedHat, Fedora and CentOS, right?

Could you update the contents of the redhat directory in trunk?

> Also, consider to glance over the Fedora review request at
> http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=230275

One thing I noticed was the mention of iconv - the man pages should
already be in UTF-8; if they aren't, let me know and I'll fix them.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Thu May 10 18:35:35 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Thu, 10 May 2007 20:35:35 +0200
Subject: Planning for 1.0.4
In-Reply-To: <87hcqlyph8.fsf@des.linpro.no>
References: <ujrvefdx2b4.fsf@false.linpro.no> <1178736940.15999.9.camel@re.e37>
	<876471w2s0.fsf@des.linpro.no> <4642E4CD.9050401@linpro.no>
	<87hcqlyph8.fsf@des.linpro.no>
Message-ID: <1178822135.9182.5.camel@re.e37>

* Ingvar Hagelund
> > I've put my latest initrc and sysconfig scripts at
> > http://users.linpro.no/ingvar/varnish/fedora-extras-commit/. Lars and
> > Stig, could you comment on them, please.

* Dag-Erling Sm?rgrav
> These can be used unmodified for RedHat, Fedora and CentOS, right?

The binary packages will probably work if the target cpu matches, yes,
though I prefer to recompile the source package specifically for the
target system. The source package should produce packages for any
RedHat-based system running the 2.6 kernel. 

> Could you update the contents of the redhat directory in trunk?

Will do.

> > Also, consider to glance over the Fedora review request at
> > http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=230275
> 
> One thing I noticed was the mention of iconv - the man pages should
> already be in UTF-8; if they aren't, let me know and I'll fix them.

They are not, at least not in the 1.0.3 tarball.

Ingvar

-- 



From des at linpro.no  Fri May 11 06:53:18 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Fri, 11 May 2007 08:53:18 +0200
Subject: Planning for 1.0.4
In-Reply-To: <1178822135.9182.5.camel@re.e37> (Ingvar Hagelund's message of
	"Thu\, 10 May 2007 20\:35\:35 +0200")
References: <ujrvefdx2b4.fsf@false.linpro.no>
	<1178736940.15999.9.camel@re.e37> <876471w2s0.fsf@des.linpro.no>
	<4642E4CD.9050401@linpro.no> <87hcqlyph8.fsf@des.linpro.no>
	<1178822135.9182.5.camel@re.e37>
Message-ID: <87hcqjx34x.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> * Dag-Erling Sm?rgrav
> > These can be used unmodified for RedHat, Fedora and CentOS, right?
> The binary packages will probably work if the target cpu matches, yes,
> though I prefer to recompile the source package specifically for the
> target system. The source package should produce packages for any
> RedHat-based system running the 2.6 kernel.

What I meant was that the same spec file can be used for each of the
aforementioned distributions.

> > Could you update the contents of the redhat directory in trunk?
> Will do.

In *trunk*, please.

> > One thing I noticed was the mention of iconv - the man pages should
> > already be in UTF-8; if they aren't, let me know and I'll fix them.
> They are not, at least not in the 1.0.3 tarball.

OK, I'll take care of it.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Fri May 11 07:46:53 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Fri, 11 May 2007 09:46:53 +0200
Subject: Planning for 1.0.4
In-Reply-To: <87hcqjx34x.fsf@des.linpro.no>
References: <ujrvefdx2b4.fsf@false.linpro.no>	<1178736940.15999.9.camel@re.e37>
	<876471w2s0.fsf@des.linpro.no>	<4642E4CD.9050401@linpro.no>
	<87hcqlyph8.fsf@des.linpro.no>	<1178822135.9182.5.camel@re.e37>
	<87hcqjx34x.fsf@des.linpro.no>
Message-ID: <46441F6D.9020101@linpro.no>

* Dag-Erling Sm?rgrav
>>> These can be used unmodified for RedHat, Fedora and CentOS, right?
> What I meant was that the same spec file can be used for each of the
> aforementioned distributions.

Untestet on rhel5/centos5, but yes.

>>> Could you update the contents of the redhat directory in trunk?
> In *trunk*, please.

Yeah, yeah, I'm coming there real soon now. Wanted to test some changes
in the 1.0 branch first.

I have used '-D' for daemonize and '-p' for pidfile in the initscript
for the upcoming daemonizing feature of varnishlog. Will '-p' take an
argument (the path to the pidfile)?

Ingvar

-- 
N?r alt annet feiler: Symlink


From des at linpro.no  Fri May 11 07:58:30 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Fri, 11 May 2007 09:58:30 +0200
Subject: Planning for 1.0.4
In-Reply-To: <46441F6D.9020101@linpro.no> (Ingvar Hagelund's message of "Fri\, 
	11 May 2007 09\:46\:53 +0200")
References: <ujrvefdx2b4.fsf@false.linpro.no>
	<1178736940.15999.9.camel@re.e37> <876471w2s0.fsf@des.linpro.no>
	<4642E4CD.9050401@linpro.no> <87hcqlyph8.fsf@des.linpro.no>
	<1178822135.9182.5.camel@re.e37> <87hcqjx34x.fsf@des.linpro.no>
	<46441F6D.9020101@linpro.no>
Message-ID: <871whnx049.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> I have used '-D' for daemonize and '-p' for pidfile in the initscript
> for the upcoming daemonizing feature of varnishlog. Will '-p' take an
> argument (the path to the pidfile)?

The correct option is -P (-p is already taken in varnishd) and yes, it
will take an argument.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Fri May 11 08:11:28 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Fri, 11 May 2007 10:11:28 +0200
Subject: Planning for 1.0.4
In-Reply-To: <87hcqjx34x.fsf@des.linpro.no>
References: <ujrvefdx2b4.fsf@false.linpro.no>	<1178736940.15999.9.camel@re.e37>
	<876471w2s0.fsf@des.linpro.no>	<4642E4CD.9050401@linpro.no>
	<87hcqlyph8.fsf@des.linpro.no>	<1178822135.9182.5.camel@re.e37>
	<87hcqjx34x.fsf@des.linpro.no>
Message-ID: <46442530.40309@linpro.no>

* Dag-Erling Sm?rgrav
>>> Could you update the contents of the redhat directory in trunk?

Done. Seems to build and work from today's svn:

[ingvar at uname ~/svn/varnish/trunk]$ cat buildrpm.sh
#!/bin/bash
VERSION=$( awk ' /^Version:/ { print $2 } '
varnish-cache/redhat/varnish.spec )
ln -s varnish-cache varnish-$VERSION
tar $(
    find varnish-$VERSION/ -type d -name .svn | \
        while read i; do
            echo -n "--exclude $i "
            done
) -cvzf varnish-$VERSION.tar.gz varnish-$VERSION/*
rm varnish-$VERSION
rpmbuild -tb varnish-$VERSION.tar.gz

[ingvar at uname ~/svn/varnish/trunk]$ sh buildrpm.sh
(...)
Wrote: ~/rpm/RPMS/x86_64/varnish-1.0.svn-20070511.x86_64.rpm
Wrote: ~/rpm/RPMS/x86_64/varnish-libs-1.0.svn-20070511.x86_64.rpm
Wrote: ~/rpm/RPMS/x86_64/varnish-debuginfo-1.0.svn-20070511.x86_64.rpm
(...)
+ exit 0

$ rpmlint ~/rpm/RPMS/x86_64/*20070511*rpm | wc -l
0

[ingvar at uname ~]$ cat /etc/redhat-release; uname -a
Red Hat Enterprise Linux ES release 4 (Nahant Update 5)
Linux uname 2.6.9-42.0.3.EL #1 Mon Sep 25 17:14:08 EDT 2006 x86_64
x86_64 x86_64 GNU/Linux

[ingvar at uname ~]$ sudo rpm -i ~/rpm/RPMS/x86_64/*20070511*rpm
[ingvar at uname ~]$ sudo /sbin/service varnish start
Starting varnish HTTP accelerator:                         [  OK  ]

[ingvar at uname ~]$ time curl http://localhost:6081/index.html \
  > /dev/null 2>&1
real    0m0.084s
user    0m0.006s
sys     0m0.006s

[ingvar at uname ~]$ time curl http://localhost:6081/index.html \
  > /dev/null 2>&1
real    0m0.014s
user    0m0.005s
sys     0m0.006s

-- 
When everything else fail: Symlink


From des at linpro.no  Fri May 11 08:46:53 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Fri, 11 May 2007 10:46:53 +0200
Subject: Planning for 1.0.4
In-Reply-To: <46442530.40309@linpro.no> (Ingvar Hagelund's message of "Fri\, 11
	May 2007 10\:11\:28 +0200")
References: <ujrvefdx2b4.fsf@false.linpro.no>
	<1178736940.15999.9.camel@re.e37> <876471w2s0.fsf@des.linpro.no>
	<4642E4CD.9050401@linpro.no> <87hcqlyph8.fsf@des.linpro.no>
	<1178822135.9182.5.camel@re.e37> <87hcqjx34x.fsf@des.linpro.no>
	<46442530.40309@linpro.no>
Message-ID: <87fy63vjb6.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> * Dag-Erling Sm?rgrav
> > Could you update the contents of the redhat directory in trunk?
> Done. Seems to build and work from today's svn:

Great!

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Wed May 16 12:23:40 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Wed, 16 May 2007 14:23:40 +0200
Subject: Default ports
Message-ID: <87sl9xq7n7.fsf@des.linpro.no>

Considering that the existing Fedora and Debian packages already use
6081 and 6082 as the default HTTP and management port, respectively, I
think I'll take the easy way out and canonicize these defaults.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Wed May 16 13:03:09 2007
From: des at linpro.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=)
Date: Wed, 16 May 2007 15:03:09 +0200
Subject: Stille =?utf-8?q?f=C3=B8r?= stormen: varnish-1.0.4
In-Reply-To: <1179259637.15106.19.camel@re.e37> (Ingvar Hagelund's message of
	"Tue\, 15 May 2007 22\:07\:17 +0200")
References: <861whh6ght.fsf@dwp.des.no> <1179259637.15106.19.camel@re.e37>
Message-ID: <87k5v8rkdu.fsf_-_@des.linpro.no>

[actually cc: the list this time]

Ingvar Hagelund <ingvar at linpro.no> writes:
> Denne postingen burde sikkert v?rt p? engelsk, og postet til
> varnish-dist.

I'll bite :)

> Det er fint om du (des) ser over og kommenterer
>
> * varnish.sysconfig

The following variables:

 VARNISH_MIN_WORKER_THREADS=1
 VARNISH_MAX_WORKER_THREADS=1000
 VARNISH_WORKER_THREAD_TIMEOUT=10
 VARNISH_HASHOPTION=classic

should be removed; changing them is generally not a good idea.

There should be a variable for "extra flags".

The following variable:

 VARNISH_BACKEND_STORAGE_SIZE=10240000

could be written as

 VARNISH_BACKEND_STORAGE_SIZE=10m

but are you sure you want a 10 MB file by default?  The built-in default
is "50%", which means min(50% of available disk space, SIZE_MAX) rounded
down to the nearest page boundary.

The following variable:

 VARNISH_TTL=120

is not very useful (the default is already 120, and in most cases you
want to tweak the TTL on a per-object basis, which must be done in VCL).
The preferred way to change it these days is to set the corresponding
run-time parameter either with -p on the command line or through the
managment port.

The startup knobs which I consider critical are the following (taken
from the updated FreeBSD port):

varnishd_listen=":6081"
varnishd_config="%%PREFIX%%/etc/varnish/default.vcl"
varnishd_telnet="localhost:6082"
varnishd_storage="file,/tmp,50%"
varnishd_flags="-P ${pidfile} -a ${varnishd_listen} -f ${varnishd_config} -T ${varnishd_telnet} -s ${varnishd_storage}"

("varnishd_telnet" should probably be called something else)

That being said, I see no harm in offering additional options, as long
as they neither confuse the user nor encourage him to twiddle knobs he
shouldn't twiddle.

> * varnishlog.sysinit

404

> * varnish.initrc

Is the lock file necessary?  Isn't it better to check for the presence
of the PID file, and of the process it references?

> Disse ligger i svn, i redhat-katalogen. Videre m? vi ha en fin
> default.vcl som erstatter vcl.conf med korrekt backend, og resten
> utkommentert (de er jo default-innstillinger, sant?)

Done.

> redhat/varnish.sysconfig og debian/varnish.default m? sannsynligvis
> oppdateres, i alle fall med verdier for lytteadresse (tom verdi => ipv4
> +ipv6, sant?) og lytteport (80? 8080? 8000?) . Disse filene kunne
> forresten gjerne v?re samme fil, eller i hvert fall rimelig identiske.
> Stig? Lars?

Considering that Debian, Fedora and OpenSUSE packages already bind to
:6081 and localhost:6082, I think we should keep those defaults.

> For ? f? svn-utgaven til ? bygge m? jeg legge til en './autogen.sh' for
> ? f? generert configure. Jeg antar at denne blir overfl?dig i den
> endelige tarballen?

I use 'make dist' to create the tarball.  It automatically creates and
includes the required files, including a working configure script.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Wed May 16 14:14:26 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Wed, 16 May 2007 16:14:26 +0200
Subject: Default ports
In-Reply-To: <87sl9xq7n7.fsf@des.linpro.no>
References: <87sl9xq7n7.fsf@des.linpro.no>
Message-ID: <464B11C2.6050302@linpro.no>

* Dag-Erling Sm?rgrav
> Considering that the existing Fedora and Debian packages already use
> 6081 and 6082 as the default HTTP and management port, respectively, I
> think I'll take the easy way out and canonicize these defaults.

Works for me.

We should probably change the name of the default vcl file to something
like default.vcl. This will not affect updating users of the Debian or
RedHat package, as this is an overrideable parameter in
varnish.defaults/varnish.sysconfig (which will not be overwritten).

I see Stig did some work on syncing the init scripts and default
parameter scripts today. I will look into this on friday.

I did a build of the 1.0 branch yesterday on RHEL4 and Fedora6. The
daemonizing function in varnishlog seems to work flawlessly.

Ingvar

-- 
N?r alt annet feiler: Symlink


From des at linpro.no  Wed May 16 14:21:01 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Wed, 16 May 2007 16:21:01 +0200
Subject: Default ports
In-Reply-To: <464B11C2.6050302@linpro.no> (Ingvar Hagelund's message of "Wed\, 
	16 May 2007 16\:14\:26 +0200")
References: <87sl9xq7n7.fsf@des.linpro.no> <464B11C2.6050302@linpro.no>
Message-ID: <87veesq27m.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> We should probably change the name of the default vcl file to something
> like default.vcl.

It *is* default.vcl...  I changed it last night while we talked on the
phone :)

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Wed May 16 14:39:57 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Wed, 16 May 2007 16:39:57 +0200
Subject: Package-building procedure
Message-ID: <87mz04q1c2.fsf@des.linpro.no>

I would like to have the following items documented for each
distribution we support (Debian, Fedora, OpenSUSE):

 - assuming the only things that need changing are the version number
   and the changelog, which files need to be updated for a new release?

 - assuming all you have is the release tarball and the necessary build
   tools, which command(s) are needed to produce a binary package?

One additional question for OpenSUSE:

 - what do we need to include to allow OpenSUSE packages to be built
   straight from the release tarball?  I assume that it's not very
   different from Fedora, since OpenSUSE is rpm-based, but I know that
   the init scripts, at least, are different.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Wed May 16 21:06:05 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Wed, 16 May 2007 23:06:05 +0200
Subject: A few comments of the updates debian files
Message-ID: <1179349565.16129.11.camel@re.e37>

Just a few comments on the updated files in trunk/varnish-cache/debian:

The default script should have some variable defining the maximum number
of open files, and the init script should use this with 'ulimit -n', as
the default (at least on redhat) is 1024, probably too small for
varnish.

The redhat sysconfig file has

# Maximum number of open files (for ulimit -n)
NFILES=131072

and the init script has

# Open files (usually 1024, which is way too small for varnish)
[ ! "${NFILES}" ] && NFILES="131072"
ulimit -n ${NFILES}


Also, the debian init script has some default varnishd options in case
there is no defaults file:

# If $DAEMON_OPTS is not set at all in /etc/default/varnish, use minimal
useful
# defaults (Backend at localhost:8080, a common place to put a locally
# installed application server.)
DAEMON_OPTS=${DAEMON_OPTS:--b localhost}

As this puts the varnish cache file on /tmp, which often is a in-memory
filesystem on Linux, this is probably a bad idea. More robust options or
an error message complaining of the lack of options would be better. I
would prefer the latter. (Thanks, Dag-Erling).

Ingvar

-- 



From des at linpro.no  Thu May 17 07:00:11 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Thu, 17 May 2007 09:00:11 +0200
Subject: A few comments of the updates debian files
In-Reply-To: <1179349565.16129.11.camel@re.e37> (Ingvar Hagelund's message of
	"Wed\, 16 May 2007 23\:06\:05 +0200")
References: <1179349565.16129.11.camel@re.e37>
Message-ID: <87646soryc.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> # Open files (usually 1024, which is way too small for varnish)
> [ ! "${NFILES}" ] && NFILES="131072"
> ulimit -n ${NFILES}

Just a nit - this can be written more simply as

ulimit -n ${NFILES:-131072}

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Fri May 18 07:42:07 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Fri, 18 May 2007 09:42:07 +0200
Subject: Package-building procedure
In-Reply-To: <87mz04q1c2.fsf@des.linpro.no>
References: <87mz04q1c2.fsf@des.linpro.no>
Message-ID: <464D58CF.3090207@linpro.no>

* Dag-Erling Sm?rgrav
> I would like to have the following items documented for each
> distribution we support (Debian, Fedora, OpenSUSE):
> 
>  - assuming the only things that need changing are the version number
>    and the changelog, which files need to be updated for a new release?

For the RedHat (and its relatives') package, edit redhat/varnish.spec.
The tags to edit are "Version", for example "1.0.4", "Release", a
counting number starting at 1, and a matching changelog item below
"%changelog", like this:

* Sun May 20 2007 Dag Erling Sm?rgrav <des at linpro.no> - 1.0.4-1
- Bumping version to 1.0.4
- Other changes
- Even more changes

Note that only changes to the redhat files should be noted. The general
ChangeLog file is included in the package.

You should also remove or comment out the line under "%prep" running
autogen.sh, as the necessary files should be included in the tarball.

>  - assuming all you have is the release tarball and the necessary build
>    tools, which command(s) are needed to produce a binary package?

First: Always build rpms as a non priveleged user, see
http://users.linpro.no/ingvar/rpm.html

On a redhat or derivate system, run

rpmbuild -tb varnish-1.0.4.tar.gz

This will produce a package for the current system/arch, for example

varnish-1.0.4-1.x86_64.rpm
varnish-libs-1.0.4-1.x86_64.rpm

You might want to name the package with a proper %dist, if this is not
done automatically (some systems don't)

rpmbuild -tb --define "dist el4" varnish-1.0.4.tar.gz

varnish-1.0.4-1el4.x86_64.rpm
varnish-libs-1.0.4-1el4.x86_64.rpm

To build a source package, use -ts

To do a quick test build from svn, try something like this:

#!/bin/bash
VERSION=$( awk ' /^Version:/ { print $2 } ' \
  varnish-cache/redhat/varnish.spec )
ln -s varnish-cache varnish-$VERSION
tar $(
    find varnish-$VERSION/ -type d -name .svn | \
        while read i; do
            echo -n "--exclude $i "
            done
) -cvzf varnish-$VERSION.tar.gz varnish-$VERSION/*
rm varnish-$VERSION
rpmbuild -tb --define "dist el4" varnish-$VERSION.tar.gz
rpmbuild -ts varnish-$VERSION.tar.gz
#EOF


Ingvar

-- 
When everything else fail: Symlink


From des at linpro.no  Sun May 20 16:22:27 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Sun, 20 May 2007 18:22:27 +0200
Subject: Ready for 1.0.4
Message-ID: <87sl9r1n3w.fsf@des.linpro.no>

I believe everything is ready for 1.0.4 now.  I'll return after dinner
to tag the release, upload tarballs to SourceForge and mail out the
announcement.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ssm at linpro.no  Mon May 21 05:07:26 2007
From: ssm at linpro.no (Stig Sandbeck Mathisen)
Date: Mon, 21 May 2007 07:07:26 +0200
Subject: Package-building procedure
In-Reply-To: <87mz04q1c2.fsf@des.linpro.no> (Dag-Erling =?utf-8?Q?Sm=C3=B8?=
	=?utf-8?Q?rgrav's?= message of "Wed, 16 May 2007 16:39:57 +0200")
References: <87mz04q1c2.fsf@des.linpro.no>
Message-ID: <7xbqgekbn5.fsf@iostat.e.linpro.no>

Dag-Erling Sm?rgrav <des at linpro.no> writes:

> I would like to have the following items documented for each
> distribution we support (Debian, Fedora, OpenSUSE):
>
>  - assuming the only things that need changing are the version
>  number and the changelog, which files need to be updated for a new
>  release?

The file needed to be updated is debian/changelog.  The version number
of the generated package is taken from there.  It should also contain
an entry with:

 * New upstream release

There is a tool in the debian package "devscripts" which can be used
to add changelog entries with the correct magic format:

dch -v 1.0.5 "New upstream release"

(This will also rename the current directory to "varnish-1.0.5")

>  - assuming all you have is the release tarball and the necessary
>  build tools, which command(s) are needed to produce a binary
>  package?

To build a package without signing it with gnupg, and package via
"fakeroot", cd to the source directory, and use:

dpkg-buildpackage -us -uc -rfakeroot

The following files will be generated in the parent directory:

-rw-r--r--  1 ssm ssm     367 2007-05-21 06:46 varnish_1.0.4-1.dsc
-rw-r--r--  1 ssm ssm    1135 2007-05-21 06:48 varnish_1.0.4-1_i386.changes
-rw-r--r--  1 ssm ssm  198964 2007-05-21 06:48 varnish_1.0.4-1_i386.deb
-rw-r--r--  1 ssm ssm  703947 2007-05-21 06:46 varnish_1.0.4-1.tar.gz

-- 
Stig Sandbeck Mathisen, Linpro


From ingvar at linpro.no  Mon May 21 19:23:21 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Mon, 21 May 2007 21:23:21 +0200
Subject: varnish got into fedora-devel today
Message-ID: <1179775401.13923.3.camel@re.e37>

varnish got into fedora-devel today. As fedora-devel is in deep freeze,
the packages won't be available from fedora mirrors till fedora-7 is
released, and devel is bumped to f8. The release of fedora-7 is due to
May 30. After this, varnish will be available from fedora-devel, and
I'll add the package to f7, and probably fc6-extras.

Ingvar

-- 



From ingvar at linpro.no  Mon May 21 20:27:18 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Mon, 21 May 2007 22:27:18 +0200
Subject: Don't upload rpm packages to sf yet
Message-ID: <1179779238.1829.6.camel@re.e37>

Don't upload rpm packages of varnish-1.0.4 to sf yet. There's a small
upgrade problem. Users upgrading from the previous version released as
rpms on sourceforge will need it a lacking 'obsoletes' in the libs
package, I think. There is no need for changes in the upstream sources,
and the package will be identical to the upstream one in all other
cases.

I will fix this and build and upload el4 and el5 packages tomorrow.

Ingvar

-- 



From ingvar at linpro.no  Tue May 22 13:49:19 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Tue, 22 May 2007 15:49:19 +0200
Subject: Don't upload rpm packages to sf yet
In-Reply-To: <1179779238.1829.6.camel@re.e37>
References: <1179779238.1829.6.camel@re.e37>
Message-ID: <4652F4DF.5010501@linpro.no>

* Ingvar Hagelund
> Don't upload rpm packages of varnish-1.0.4 to sf yet. There's a small
> upgrade problem. Users upgrading from the previous version released as
> rpms on sourceforge will need it a lacking 'obsoletes' in the libs
> package, I think. There is no need for changes in the upstream sources,
> and the package will be identical to the upstream one in all other
> cases.
> 
> I will fix this and build and upload el4 and el5 packages tomorrow.

Fixed and uploaded to sf.

Ingvar

-- 
N?r alt annet feiler: Symlink


From des at linpro.no  Tue May 22 14:02:47 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Tue, 22 May 2007 16:02:47 +0200
Subject: Don't upload rpm packages to sf yet
In-Reply-To: <4652F4DF.5010501@linpro.no> (Ingvar Hagelund's message of "Tue\, 
	22 May 2007 15\:49\:19 +0200")
References: <1179779238.1829.6.camel@re.e37> <4652F4DF.5010501@linpro.no>
Message-ID: <87myzxx8fs.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> Fixed and uploaded to sf.

Thank you!

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ingvar at linpro.no  Tue May 29 07:50:00 2007
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Tue, 29 May 2007 09:50:00 +0200
Subject: Remember to exclude .svn directories from relase tarballs
Message-ID: <465BDB28.8030204@linpro.no>

ssia, actually

$ wget http://downloads.sourceforge.net/varnish/varnish-1.0.4.tar.gz

$ tar xvzf varnish-1.0.4.tar.gz

$ find varnish-1.0.4 -type d -name .svn
varnish-1.0.4/redhat/.svn
varnish-1.0.4/debian/.svn


Ingvar

-- 
N?r alt annet feiler: Symlink


From des at linpro.no  Tue May 29 09:35:22 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Tue, 29 May 2007 11:35:22 +0200
Subject: Remember to exclude .svn directories from relase tarballs
In-Reply-To: <465BDB28.8030204@linpro.no> (Ingvar Hagelund's message of "Tue\, 
	29 May 2007 09\:50\:00 +0200")
References: <465BDB28.8030204@linpro.no>
Message-ID: <878xb8nfad.fsf@des.linpro.no>

Ingvar Hagelund <ingvar at linpro.no> writes:
> ssia, actually
>
> $ wget http://downloads.sourceforge.net/varnish/varnish-1.0.4.tar.gz
>
> $ tar xvzf varnish-1.0.4.tar.gz
>
> $ find varnish-1.0.4 -type d -name .svn
> varnish-1.0.4/redhat/.svn
> varnish-1.0.4/debian/.svn

This is because the redhat and debian directories are listed as
EXTRA_DIST in the root Makefile.am instead of having their own
makefiles.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Thu May 31 13:00:24 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Thu, 31 May 2007 15:00:24 +0200
Subject: r1482 - trunk/varnish-cache/bin/varnishd
In-Reply-To: <20070531125733.045FC1EC400@projects.linpro.no>
	(des@projects.linpro.no's message of "Thu\, 31 May 2007 14\:57\:32 +0200
	\(CEST\)")
References: <20070531125733.045FC1EC400@projects.linpro.no>
Message-ID: <87bqg1b11z.fsf@des.linpro.no>

des at projects.linpro.no writes:
> Log:
> Add two run-time parameters, "user" and "group", which specify an unprivileged
> user and group to which the child process will switch immediately after fork()
> returns, before it starts accepting connections.  The default values are
> "nobody" and "nogroup" (they should probably be tweakable at compile time...)
>
> Note that this does not provide full privilege separation, as there are still
> channels between the parent and child processes which need to be monitored,
> but it is an improvement on the previous situation.

These settings should be documented (and tweakable) in varnish.default
etc.  Depending on the distribution, there may be more appropriate
default values for user and group (e.g. "www-data" on Debian)

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no