Varnish DAEMON_OPTS Options Errors - daemon

Question

When using inline C with Varnish I've not been able to get /etc/varnish/default
to be happy at startup.
I've tested inline C with varnish for two things: GeoIP detection and Anti-Site-Scraping functions.
The DAEMON_OPTS always complains even though I'm following what other seem
to indicate works fine.
My problem is that this command line start up works:
varnishd -f /etc/varnish/varnish-default.conf -s file,/var/lib/varnish/varnish_storage.bin,512M -T 127.0.0.1:2000 -a 0.0.0.0:8080 -p 'cc_command=exec cc -fpic -shared -Wl,-x -L/usr/include/libmemcached/memcached.h -lmemcached -o %o %s'
But it errors out with trying to start up from default start scripts:
/etc/default/varnish has this in it:
DAEMON_OPTS="-a :8080 \
-T localhost:2000 \
-f /etc/varnish/varnish-default.conf \
-s file,/var/lib/varnish/varnish_storage.bin,512M \
-p 'cc_command=exec cc -fpic -shared -Wl,-x -L/usr/include/libmemcached/memcached.h -lmemcached -o %o %s'"
The error is:
# /etc/init.d/varnish start
Starting HTTP accelerator: varnishd failed!
storage_file: filename: /var/lib/varnish/vbox.local/varnish_storage.bin size 512 MB.
Error:
Unknown parameter "'cc_command".
If I try change the last line to:
-p cc_command='exec cc -fpic -shared -Wl,-x -L/usr/include/libmemcached/memcached.h -lmemcached -o %o %s'"
It's error is now:
# /etc/init.d/varnish start
Starting HTTP accelerator: varnishd failed!
storage_file: filename: /var/lib/varnish/vbox.local/varnish_storage.bin size 512 MB.
Error: Unknown storage method "hared"
It's trying to interpret the '-shared' as -s hared and 'hared' is not a storage type.
For both GeoIP and the Anti-Site-Scrape I've used the exact recommended daemon options
plus have tried all sorts of variations like adding ' and '' but no joy.
Here is a link to the instruction I've followed that work fine except the DAEMON_OPTS part.
http://drcarter.info/2010/04/how-fighting-against-scraping-using-varnish-vcl-inline-c-memcached/
I'm using Debian and the exact DAEMON_OPTS as stated in the instructions.
Can anyone help with a pointer on what's going wrong here?

Answer 1

Even if Jacob will probably never read this, visitors from the future might appreciate what I'm going to write.
I believe I know what's wrong, and it looks like a Debian-specific problem, at least verified on Ubuntu 11.04 and Debian Squeeze.
I traced the execution from my /etc/default/varnish that contains the $DAEMON_OPTS to the init script.
In the init script /etc/init.d/varnish, the start_varnishd() function is:
start_varnishd() {
log_daemon_msg "Starting $DESC" "$NAME"
output=$(/bin/tempfile -s.varnish)
if start-stop-daemon \
--start --quiet --pidfile ${PIDFILE} --exec ${DAEMON} -- \
-P ${PIDFILE} ${DAEMON_OPTS} > ${output} 2>&1; then
log_end_msg 0
else
log_end_msg 1
cat $output
exit 1
fi
rm $output
}
So I modified it to print the full start-stop-daemon command line, like:
start_varnishd() {
log_daemon_msg "Starting $DESC" "$NAME"
output=$(/bin/tempfile -s.varnish)
+ echo "start-stop-daemon --start --quiet --pidfile ${PIDFILE} --exec ${DAEMON} -- -P ${PIDFILE} ${DAEMON_OPTS} > ${output} 2>&1"
if start-stop-daemon \
--start --quiet --pidfile ${PIDFILE} --exec ${DAEMON} -- \
-P ${PIDFILE} ${DAEMON_OPTS} > ${output} 2>&1; then
log_end_msg 0
So I got a command line echoed on STDOUT, and copied-pasted it into my shell. And, surprise! It worked. WTF?
Repeated again to be sure. Yes, it works. Mmh. Could it be another of those bash/dash corner cases?
Let's try feeding the start-stop-daemon command line to bash, and see how it reacts:
start_varnishd() {
log_daemon_msg "Starting $DESC" "$NAME"
output=$(/bin/tempfile -s.varnish)
if bash -c "start-stop-daemon \
--start --quiet --pidfile ${PIDFILE} --exec ${DAEMON} -- \
-P ${PIDFILE} ${DAEMON_OPTS} > ${output} 2>&1"; then
log_end_msg 0
else
log_end_msg 1
cat $output
exit 1
fi
rm $output
}
Yes, it works just fine, at least for my case.
Here's the relevant part of my /etc/default/varnish:
...
## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request. Use a 1GB
# fixed-size cache file.
#
DAEMON_OPTS="-a :6081 \
-T localhost:6082 \
-f /etc/varnish/geoip-example.vcl \
-S /etc/varnish/secret \
-s malloc,100M \
-p 'cc_command=exec cc -fpic -shared -Wl,-x -L/usr/include/GeoIP.h -lGeoIP -o %o %s'"
...
I've seen posts where someone tried to work around this problem by moving the compile command into a separated shell script. Unfortunately that doesn't change the fact that start-stop-daemon is going to pass the $DAEMON_OPTS var through dash, and that will result in mangled options.
Would be something along the lines of:
-p 'cc_command=exec /etc/varnish/compile.sh %o %s'"
And then the compile.sh script as:
#!/bin/sh
cc -fpic -shared -Wl,-x -L/usr/include/GeoIP.h -lGeoIP -o $#
but it doesn't work, so just patch your init scripts, and you're good to go!
Hope you can find this information useful.

Answer 2

You can try using :-
DAEMON_OPTS="-a :8080 \
-T localhost:2000 \
-f /etc/varnish/varnish-default.conf \
-s file,/var/lib/varnish/varnish_storage.bin,512M \
-p cc_command='exec cc -fpic -shared -Wl,-x -L/usr/include/libmemcached/memcached.h -lmemcached -o %o %s'"

Answer 3

Obviously, your startup script interpreting the DAEMON_OPTS is not prepared for whitespace (even within single quotes). At my Fedora (15) installation, the suggested solution works fine; the arguments get interpreted correctly because the "$*" bash parameter is passed in /etc/init.d/varnish and in /etc/init.d/functions in daemon().
Did you get your startup scripts from a package or did you make custom scripts?

Answer 4

This isn't directly related to the question, but you may find yourself here if you are working through the Varnish Tutorial - Put Varnish on port 80.
For recent installs of Varnish on Debian systems the configuration for varnishd startup options can be found in /etc/systemd/system/multi-user.target.wants/varnish.service. The documented way of changing the port via /etc/default/varnish still exists, but is no longer functional unless you change your system to use init scripts rather than systemd.
After you've changed your options in /etc/systemd/system/multi-user.target.wants/varnish.service, don't forget to run systemctl daemon-reload, which will catalog the changes for executing the program.