Not able to build rpm from src - centos

I need to rebuild rabbitmq-server-3.1.5 with my changes to some of the static files.
But before doing that I thought of being sure if I can rebuild the rpm without my changes first, and struck with the below issue.
I just installed the source-rpm and edited the SPEC file and changed the version to next minor number, and run the command
rpmbuild -ba rabbitmq-server.spec
Below is the SPEC file,
%define debug_package %{nil}
Name: rabbitmq-server
Version: 3.1.5
Release: 2%{?dist}
License: MPLv1.1 and MIT and ASL 2.0 and BSD
Group: %{group_tag}
#Source: http://www.rabbitmq.com/releases/rabbitmq-server/v%{version}/%{name}-%{version}.tar.gz
Source:rabbitmq-server-3.1.5.tar.gz
Source1: rabbitmq-server.init
Source2: rabbitmq-script-wrapper
Source3: rabbitmq-server.logrotate
Source4: rabbitmq-server.ocf
URL: http://www.rabbitmq.com/
BuildArch: noarch
BuildRequires: erlang >= R12B-3, python-simplejson, xmlto, libxslt
Requires: erlang >= R12B-3, logrotate
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-%{_arch}-root
Summary: The RabbitMQ server
Requires(post): chkconfig initscripts
Requires(pre): chkconfig initscripts
%description
RabbitMQ is an implementation of AMQP, the emerging standard for high
performance enterprise messaging. The RabbitMQ server is a robust and
scalable implementation of an AMQP broker.
# We want to install into /usr/lib, even on 64-bit platforms
%define _rabbit_libdir %{_exec_prefix}/lib/rabbitmq
%define _rabbit_erllibdir %{_rabbit_libdir}/lib/rabbitmq_server-%{version}
%define _rabbit_wrapper %{_builddir}/`basename %{S:2}`
%define _rabbit_server_ocf %{_builddir}/`basename %{S:4}`
%define _plugins_state_dir %{_localstatedir}/lib/rabbitmq/plugins
%define _maindir %{buildroot}%{_rabbit_erllibdir}
%prep
%setup -q
%build
cp %{S:2} %{_rabbit_wrapper}
cp %{S:4} %{_rabbit_server_ocf}
make %{?_smp_mflags}
%install
rm -rf %{buildroot}
make install TARGET_DIR=%{_maindir} \
SBIN_DIR=%{buildroot}%{_rabbit_libdir}/bin \
MAN_DIR=%{buildroot}%{_mandir}
mkdir -p %{buildroot}%{_localstatedir}/lib/rabbitmq/mnesia
mkdir -p %{buildroot}%{_localstatedir}/log/rabbitmq
#Copy all necessary lib files etc.
install -p -D -m 0755 %{S:1} %{buildroot}%{_initrddir}/rabbitmq-server
install -p -D -m 0755 %{_rabbit_wrapper} %{buildroot}%{_sbindir}/rabbitmqctl
install -p -D -m 0755 %{_rabbit_wrapper} %{buildroot}%{_sbindir}/rabbitmq-server
install -p -D -m 0755 %{_rabbit_wrapper} %{buildroot}%{_sbindir}/rabbitmq-plugins
install -p -D -m 0755 %{_rabbit_server_ocf} %{buildroot}%{_exec_prefix}/lib/ocf/resource.d/rabbitmq/rabbitmq-server
install -p -D -m 0644 %{S:3} %{buildroot}%{_sysconfdir}/logrotate.d/rabbitmq-server
mkdir -p %{buildroot}%{_sysconfdir}/rabbitmq
rm %{_maindir}/LICENSE %{_maindir}/LICENSE-MPL-RabbitMQ %{_maindir}/INSTALL
#Build the list of files
echo '%defattr(-,root,root, -)' >%{_builddir}/%{name}.files
find %{buildroot} -path %{buildroot}%{_sysconfdir} -prune -o '!' -type d -printf "/%%P\n" >>%{_builddir}/%{name}.files
%pre
if [ $1 -gt 1 ]; then
# Upgrade - stop previous instance of rabbitmq-server init.d script
/sbin/service rabbitmq-server stop
fi
# create rabbitmq group
if ! getent group rabbitmq >/dev/null; then
groupadd -r rabbitmq
fi# create rabbitmq user
if ! getent passwd rabbitmq >/dev/null; then
useradd -r -g rabbitmq -d %{_localstatedir}/lib/rabbitmq rabbitmq \
-c "RabbitMQ messaging server"
fi
%post
/sbin/chkconfig --add %{name}
if [ -f %{_sysconfdir}/rabbitmq/rabbitmq.conf ] && [ ! -f %{_sysconfdir}/rabbitmq/rabbitmq-env.conf ]; then
mv %{_sysconfdir}/rabbitmq/rabbitmq.conf %{_sysconfdir}/rabbitmq/rabbitmq-env.conf
fi
%preun
if [ $1 = 0 ]; then
#Complete uninstall
/sbin/service rabbitmq-server stop
/sbin/chkconfig --del rabbitmq-server
# We do not remove /var/log and /var/lib directories
# Leave rabbitmq user and group
fi
# Clean out plugin activation state, both on uninstall and upgrade
rm -rf %{_plugins_state_dir}
for ext in rel script boot ; do
rm -f %{_rabbit_erllibdir}/ebin/rabbit.$ext
done
%files -f ../%{name}.files
%defattr(-,root,root,-)
%attr(0755, rabbitmq, rabbitmq) %dir %{_localstatedir}/lib/rabbitmq
%attr(0755, rabbitmq, rabbitmq) %dir %{_localstatedir}/log/rabbitmq
%dir %{_sysconfdir}/rabbitmq
%{_initrddir}/rabbitmq-server
%config(noreplace) %{_sysconfdir}/logrotate.d/rabbitmq-server
%doc LICENSE*
%clean
rm -rf %{buildroot}
I got the below error,
escript generate_app ebin/rabbit_app.in ebin/rabbit.app src
erlc -I include -o ebin -Wall -v +debug_info -Duse_specs -pa ebin src/app_utils.erl
erlc -I include -o ebin -Wall -v +debug_info -Duse_specs -pa ebin src/gen_server2.erl
erlc -I include -o ebin -Wall -v +debug_info -Duse_specs -pa ebin src/credit_flow.erl
src/credit_flow.erl:40: Warning: opaque type bump_msg() is not exported
erlc -I include -o ebin -Wall -v +debug_info -Duse_specs -pa ebin src/delegate_sup.erl
src/gen_server2.erl:526: Warning: erlang:now/0: Deprecated BIF. See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
src/gen_server2.erl:590: Warning: erlang:now/0: Deprecated BIF. See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
src/gen_server2.erl:598: Warning: erlang:now/0: Deprecated BIF. See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
erlc -I include -o ebin -Wall -v +debug_info -Duse_specs -pa ebin src/dtree.erl
erlc -I include -o ebin -Wall -v +debug_info -Duse_specs -pa ebin src/file_handle_cache.erl
src/dtree.erl:44: type gb_tree() undefined
make: *** [ebin/dtree.beam] Error 1
make: *** Waiting for unfinished jobs....
src/file_handle_cache.erl:476: Warning: erlang:now/0: Deprecated BIF. See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
src/file_handle_cache.erl:571: Warning: erlang:now/0: Deprecated BIF. See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
src/file_handle_cache.erl:605: Warning: erlang:now/0: Deprecated BIF. See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
src/file_handle_cache.erl:640: Warning: erlang:now/0: Deprecated BIF. See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
src/file_handle_cache.erl:1139: Warning: erlang:now/0: Deprecated BIF. See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
error: Bad exit status from /var/tmp/rpm-tmp.FjNb8Z (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.FjNb8Z (%build)
[david#thetemplate SPECS]$
whereas I didn't face this issue while performing these steps on the other higher version of Rabbitmq-server, and I successfully rebuilt the rpms too. I've no clue why this is happening and also I'm very new to these kind of builds.
would need some expert advice to solve this issue.

This is the important line in the error output:
src/dtree.erl:44: type gb_tree() undefined
In Erlang/OTP 17.0, the type gb_tree() was deprecated, and the type gb_trees:tree() was introduced to replace it. gb_tree() was finally removed in Erlang/OTP 18.0.
RabbitMQ 3.4.0 changed the type spec to use the newer type, and it is thus the earliest RabbitMQ version that can be built with Erlang/OTP 18. Here's the commit where the change was made.
To solve this problem, either use a newer version of RabbitMQ or an older version of Erlang/OTP.

Related

rpmbuild no such file or directory

I'm just learning making rpm packages for some custom builds of software that gets compiled from source (some legacy stuff needs this, so I'm trying to learn, as some packages can't use the latest versions), but hitting an error (I'm doing this in Vagrant, and also as root, but typically I'm trying not to use root as I'm aware it has potential for damage, its just this example seems to need some root changes).
sudo rpmbuild -ba testspec.spec --define "_topdir /tmp/"
So far it looks to be using the directory I expected, /tmp/rpmbuild
make[2]: Entering directory `/tmp/rpmbuild/BUILD/exim-4.80.1/build-Linux-x86_64/pdkim'
make[2]: `pdkim.a' is up to date.
make[2]: Leaving directory `/tmp/rpmbu
But then I see these errors...
/usr/lib/rpm/brp-compress: line 8: cd: /tmp/BUILDROOT/custom-exim-4.80.1-1.x86_64: No such file or directory
+ /usr/lib/rpm/brp-strip
find: `/tmp/BUILDROOT/custom-exim-4.80.1-1.x86_64': No such file or directory
+ /usr/lib/rpm/brp-strip-static-archive
find: `/tmp/BUILDROOT/custom-exim-4.80.1-1.x86_64': No such file or directory
+ /usr/lib/rpm/brp-strip-comment-note
So it now seems to be looking in /tmp/BUILDROOT
I'm new to rpmbuild, and don't quite understand some of the process.
My test spec file is at...
%define myversion exim-4.80.1
##%define mybase %{getenv:HOME}
%define mybase /tmp
%define _topdir %{mybase}/rpmbuild
%define _tmppath %{mybase}/rpmbuild/tmp
%define name custom-exim
%define release 1
%define version 4.80.1
%define buildroot %{_topdir}/%{name}-%{version}-root
BuildRoot: %{buildroot}
Summary: %{name}
Name: %{name}
Version: %{version}
Release: %{release}
Source0: ftp://exim.noris.de/exim/exim4/old/exim-4.80.1.tar.gz
License: GPLv1+
Group: Language
AutoReq: no
AutoProv: no
Requires: db4-devel pcre-devel libdb-devel libXt-devel libXaw-devel
%description
Custom Exim Build
%prep
#Do the following manually before building rpm
#mkdir -p /tmp/rpmbuild/BUILD /tmp/rpmbuild/SPECS /tmp/rpmbuild/SOURCES /tmp/rpmbuild/BUILDROOT /tmp/rpmbuild/RPMS /tmp/rpmbuild/SRPMS
#wget ftp://exim.noris.de/exim/exim4/old/exim-4.80.1.tar.gz -O /tmp/rpmbuild/SOURCES/exim-4.80.1.tar.gz
%setup -q -n %{myversion}
grep exim /etc/passwd || useradd -c "Exim" -d /var/spool/exim -m -s /bin/bash exim
%build
# exim needs to config changes before compiling, may do these first and repackage
cp %{mybase}/rpmbuild/BUILD/%{myversion}/src/EDITME %{mybase}/rpmbuild/BUILD/%{myversion}/Local/Makefile
cp %{mybase}/rpmbuild/BUILD/%{myversion}/exim_monitor/EDITME %{mybase}/rpmbuild/BUILD/%{myversion}/Local/eximon.conf
sed -i -e 's/EXIM_USER=$/EXIM_USER=exim/g' "%{mybase}/rpmbuild/BUILD/%{myversion}/Local/Makefile"
sed -i -e 's/LOOKUP_DNSDB=yes/#LOOKUP_DNSDB=yes/g' "%{mybase}/rpmbuild/BUILD/%{myversion}/Local/Makefile"
make
%install
rm -rf $RPM_BUILD_ROOT
#%{__mkdir_p} '%{buildroot}%{_sbindir}'
make install
%clean
rm -rf $RPM_BUILD_ROOT
%post
%postun
%files
Why is it using /tmp/BUILDROOT literally, instead of /tmp/rpmbuild, and are there other obvious things I'm doing wrong ? I've looked at a lot of other tutorials on rpmbuild, but aren't very clear on best practices or what happens during each phase.
Since the buildroot parm is not passed to rpmbuild, the default path is being used by your spec file:
BuildRoot: %{buildroot}
Try adding the buildroot parm... Add buildroot /tmp/rpmbuild to --define
Or if using a makefile:
BUILD_TMP=/tmp/rpmbuild
TOP_DIR=/tmp
rpmbuild -bb
--buildroot $(BUILD_TMP)
--topdir $(TOP_DIR)
$(SPEC_DIR)/testspec.spec
In my case rpm-build was missing.
So sudo yum install rpm-build solved the problem. Or if you use puppet:
package { 'rpm-build':
ensure => latest,
}

rpm build fails to make build root dir

I am working on making an rpm for a small program used within our enterprise. The %build section of the rpm process works. I'm having trouble with the install section. I've referenced this article response and I believe I am properly referring to the target location with respect to %{_buildroot}.
The program I'm making is to be installed as a system service. So, after the rpm actually is generated for this step, I've got to add the next step in my installation process which is to include the script that is installed to the init.d location and run that install. One step at a time though.
The build errors are as follows (omitting everything but %install):
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.eUDaCK
+ umask 022
+ cd /home/packager/rpmbuild/BUILD
+ '[' /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64 '!=' / ']'
+ rm -rf /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64
++ dirname /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64
+ mkdir -p /home/packager/rpmbuild/BUILDROOT
+ mkdir /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64
+ cd o2arbitord-1.0
+ LANG=C
+ export LANG
+ unset DISPLAY
+ install -m 555 /home/packager/rpmbuild/BUILD/o2arbitord-1.0/o2arbitord /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64/usr/sbin
install: cannot create regular file `/home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64/usr/sbin': No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.eUDaCK (%install)
Now, my rpmbuild directory does not have the directory /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64/usr/sbin. While I know that's part of the problem, the rpmbuild process isn't making the directory /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64 either. What I don't understand about that one is: why? Looking at the script output above you can clearly see the line: mkdir /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64. So, why isn't the directory made?
How does the line BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX) from whatever the definition of %{_buildroot} is? I thought that was the definition, but it appears to be something different.
For reference, my spec file
Name: o2arbitord
Version: 1.0
Release: 1%{?dist}
Summary: a daemon
Group: Applications/System
License: GPL
URL: http://My.site
Source0: %{name}-%{version}.tar.gz
BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)
BuildArch: x86_64
BuildRequires: libusb1-devel
#Requires:
%description
%prep
%setup -q
%build
make -f o2arbitord.mk
%install
install -m 555 %{_builddir}/%{name}-%{version}/%{name} %{buildroot}%{_sbindir}
%clean
rm -rf %{buildroot}
%files
%defattr(-,root,root,-)
/usr/sbin/o2arbitord
%changelog
You are attempting to install a file into a directory that doesn't exist (yet).
RPM only creates the %{buildroot} for you automatically. Anything under that you need to create yourself.
So when you run
install -m 555 %{_builddir}/%{name}-%{version}/%{name} %{buildroot}%{_sbindir}
where %{buildroot}%{_sbindir} expands to /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64/usr/sbin RPM has only created /home/packager/rpmbuild/BUILDROOT/o2arbitord-1.0-1.el6.x86_64 for you already.
You need to create the /usr/sbin part of that path and then copy the file into it.
You can do that with either
%{__mkdir_p} '%{buildroot}%{_sbindir}'
or
%{__install} -d '%{buildroot}%{_sbindir}'
Where
$ rpm -E '__mkdir_p = %{__mkdir_p}'
__mkdir_p = /bin/mkdir -p
$ rpm -E '__install = %{__install}'
__install = /usr/bin/install

"rpmbuild -bb package.spec" produces error: Package already exists: %package debuginfo

When i run the command rpmbuild -bb mypackage.spec, on centos, i get an error
error: Package already exists: %package debuginfo
following is part of the spec file:
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}
BuildRequires: gcc = 4.1.2
BuildRequires: make >= 3.81
BuildRequires: postgresql84-devel = 8.4.9
%description
ElementalCORE
%prep
%setup -qn pcore
%build
%{__make}
%install
mkdir -p %{_tmppath}/%{name}-%{version}-%{release}
mkdir -p %{name}
mkdir -p %{_bindir}
install -d %{name}
#This is to ensure that the BuildRoot will be created fresh during the %install section.
#mkdir -p %{buildroot}
#install -p -m 0755 $RPM_BUILD_ROOT/%{name}/obj/ggsnc/ggsnc %{_bindir}
%clean
%files
%defattr(-,root,root)
%doc NEWS
The problem is the %install in the comment. Replace it with %_install.

Keys for Net::SSH2

I have a simple perl test script which uses Net::SSH2, and I'm having trouble getting it to work with public key authentication.
Note: After reading a couple of replies, I realise that I should mention that I created the script to help narrow down a problem I was having with another application that uses Net::SSH2. It's therefore not possible for me to switch to an alternative package such as Net::OpenSSH or Net::OpenSSH::Compat::SSH2.
Note Update: As salva pointed out, Net::OpenSSH::Compat::SSH2 can be used without having to patch the original application.
Here's the code:
use strict;
use warnings;
use 5.10.0;
use Net::SSH2;
my $ssh2 = Net::SSH2->new();
my $auth;
if ( $ssh2->connect('hostname') ) {
$auth = $ssh2->auth_publickey(
'username',
'/home/mike/.ssh/id_rsa.pub',
'/home/mike/.ssh/id_rsa',
'password'
);
}
if ($auth && $ssh2->auth_ok) {
say 'Success';
} else {
say join ', ', $ssh2->error;
}
I have been generating key pairs using ssh-keygen on Ubuntu:
ssh-keygen -t rsa
ssh-copy-id user#server
ssh user#server
The above works correctly - I can SSH to the server.
When I run the Perl code, I get the following error when using a password protected key:
-16, LIBSSH2_ERROR_FILE, Unable to initialize private key from file
If I try with a non-password protected key, it works correctly.
I have the following libraries installed:
Net::SSH2 0.35
libssh2-1 1.2.2-1
libssh2-1-dev 1.2.2-1
ssh 1:5.3p1-3ubuntu6
openssh-server 1:5.3p1-3ubuntu6
openssh-client 1:5.3p1-3ubuntu6
openssl 0.9.8k-7ubuntu8.6
libssl-dev 0.9.8k-7ubuntu8.6
The plot thickens
If I remove libssh2-1 and libssh2-1-dev, I get an error, as expected:
Can't load '/usr/local/lib/perl/5.10.1/auto/Net/SSH2/SSH2.so'
If I then build libssh2 from source, I am unable to reinstall Net::SSH2, as it can't find the development headers. However, if I reinstall libssh2-1 and libssh2-1-dev and then build and install libssh2 from source, it works.
Does that mean that the build of libssh2-1-dev on Ubuntu 10.04 is at fault? If so, how do I install libssh2 from source and correctly install Net::SSH2, without requiring libssh2-1-dev. I presume that the build from source is overwriting or overriding the Ubuntu package.
UPDATE
As noted in Daniel Stenberg's reply, the Ubuntu 10.04 package is now a little out of date (version 1.2.2, compared with the latest stable build at 1.2.8). Installing libssh2 from source 'over the top' of the Ubuntu package solves the problem. However, this feels messy to me. How can I remove the Ubuntu package, install libssh2 from source, and still build Net::SSH2 — how do I tell Net::SSH2 where to find the libssh2 development headers?
UPDATE 2
salva came to the rescue again, and showed how the Net::SSH2 installer can be configured with lib and include directories, so that it may be used with the source install of libssh2. Thanks salva! Note that instead of patching Makefile.PL, it's possible to just override the $lib and $inc variables at the top of Makefile.PL. I wasn't aware of this until I looked at salva's patch.
I have been able to successfully log into a server using a pass-phrase protected key with Net::SSH2 from the the Ubuntu 11.04 libnet-ssh2-perl package.
Anyway, consider using Net::OpenSSH or Net::OpenSSH::Compat::SSH2 instead of Net::SSH2.
Update: Net::OpenSSH::Compat::SSH2 tries to be a replacement for Net::SSH2. There is no need to patch the application in order to try it:
perl -MNet::OpenSSH::Compat=Net::SSH2 yor_app.pl
When you install libssh2 from the source package it will by default install itself in /usr/local, while the default package installs in /usr which probably is why it only finds the "stock" installed version.
Unfortunately it seems your Ubuntu has a fairly old libssh2 version as it has been bugfixed a lot since then.
I recommend using the libssh2-devel mailing list for in-depth libssh2 questions. It's a small but friendly community.
I have created a patch for Net::SSH2 Makefile.PL that allows to set the location of libssh2 libs and headers from the command line.
Here is a capture of the full build process for libssh2 + Net::SSH2 using it:
salva#leon:/tmp/salva$ wget http://www.libssh2.org/download/libssh2-1.2.8.tar.gz
--2011-05-27 11:21:10-- http://www.libssh2.org/download/libssh2-1.2.8.tar.gz
Resolving www.libssh2.org... 80.67.6.50
Connecting to www.libssh2.org|80.67.6.50|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 637707 (623K) [application/x-gzip]
Saving to: `libssh2-1.2.8.tar.gz'
100%[================================================================================================================================================================================================>] 637,707 525K/s in 1.2s
2011-05-27 11:21:14 (525 KB/s) - `libssh2-1.2.8.tar.gz' saved [637707/637707]
salva#leon:/tmp/salva$ tar xzf libssh2-1.2.8.tar.gz
salva#leon:/tmp/salva$ cd libssh2-1.2.8
salva#leon:/tmp/salva/libssh2-1.2.8$ ./configure --prefix=/usr/local/libssh2
checking whether to enable maintainer-specific portions of Makefiles... no
checking for sed... /bin/sed
checking for a BSD-compatible install... /usr/bin/install -c
...
salva#leon:/tmp/salva/libssh2-1.2.8$ make
Making all in src
make[1]: Entering directory `/tmp/salva/libssh2-1.2.8/src'
make all-am
make[2]: Entering directory `/tmp/salva/libssh2-1.2.8/src'
if /bin/bash ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I../include -I../src -g -O2 -MT channel.lo -MD -MP -MF ".deps/channel.Tpo" -c -o channel.lo channel.c; \
then mv -f ".deps/channel.Tpo" ".deps/channel.Plo"; else rm -f ".deps/channel.Tpo"; exit 1; fi
libtool: compile: gcc -DHAVE_CONFIG_H -I../include -I../src -g -O2 -MT channel.lo -MD -MP -MF .deps/channel.Tpo -c channel.c -fPIC -DPIC -o .libs/channel.o
...
salva#leon:/tmp/salva/libssh2-1.2.8$ sudo make install
Making install in src
make[1]: Entering directory `/tmp/salva/libssh2-1.2.8/src'
make[2]: Entering directory `/tmp/salva/libssh2-1.2.8/src'
test -z "/usr/local/libssh2/lib" || mkdir -p -- "/usr/local/libssh2/lib"
/bin/bash ../libtool --mode=install /usr/bin/install -c 'libssh2.la' '/usr/local/libssh2/lib/libssh2.la'
libtool: install: /usr/bin/install -c .libs/libssh2.so.1.0.1 /usr/local/libssh2/lib/libssh2.so.1.0.1
...
salva#leon:/tmp/salva/libssh2-1.2.8$ cd ..
salva#leon:/tmp/salva$ wget http://search.cpan.org/CPAN/authors/id/R/RK/RKITOVER/Net-SSH2-0.35.tar.gz
--2011-05-27 11:22:56-- http://search.cpan.org/CPAN/authors/id/R/RK/RKITOVER/Net-SSH2-0.35.tar.gz
Resolving search.cpan.org... 207.115.101.144
Connecting to search.cpan.org|207.115.101.144|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://osl.ugr.es/CPAN/authors/id/R/RK/RKITOVER/Net-SSH2-0.35.tar.gz [following]
--2011-05-27 11:22:59-- http://osl.ugr.es/CPAN/authors/id/R/RK/RKITOVER/Net-SSH2-0.35.tar.gz
Resolving osl.ugr.es... 150.214.21.7
Connecting to osl.ugr.es|150.214.21.7|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 92434 (90K) [application/x-gzip]
Saving to: `Net-SSH2-0.35.tar.gz'
100%[================================================================================================================================================================================================>] 92,434 328K/s in 0.3s
2011-05-27 11:22:59 (328 KB/s) - `Net-SSH2-0.35.tar.gz' saved [92434/92434]
salva#leon:/tmp/salva$ tar xzf Net-SSH2-0.35.tar.gz
salva#leon:/tmp/salva$ cd Net-SSH2-0.35
salva#leon:/tmp/salva/Net-SSH2-0.35$ wget -q --no-check-certificate -O - https://github.com/salva/net-ssh2/commit/3c7261f4584137f4240d204731e20f709f1addb1.patch|patch -p1
patching file Makefile.PL
salva#leon:/tmp/salva/Net-SSH2-0.35$ perl Makefile.PL lib=/usr/local/libssh2/lib/ inc=/usr/local/libssh2/include/
The libssh2 library is required by this module. If you don't have it, you can
download it from http://www.libssh2.org; you may also need OpenSSL, which can be
obtained from http://www.openssl.org.
Debian: sudo aptitude install libssh2-1-dev
OpenSUSE: sudo zypper in libssh2-1 libssh2-devel
Checking if your kit is complete...
Looks good
Writing Makefile for Net::SSH2
salva#leon:/tmp/salva/Net-SSH2-0.35$ make
cp lib/Net/SSH2/File.pm blib/lib/Net/SSH2/File.pm
cp lib/Net/SSH2/PublicKey.pm blib/lib/Net/SSH2/PublicKey.pm
cp lib/Net/SSH2/Dir.pm blib/lib/Net/SSH2/Dir.pm
cp lib/Net/SSH2/SFTP.pm blib/lib/Net/SSH2/SFTP.pm
cp lib/Net/SSH2/Listener.pm blib/lib/Net/SSH2/Listener.pm
cp lib/Net/SSH2/Channel.pm blib/lib/Net/SSH2/Channel.pm
cp lib/Net/SSH2.pm blib/lib/Net/SSH2.pm
AutoSplitting blib/lib/Net/SSH2.pm (blib/lib/auto/Net/SSH2)
/usr/bin/perl "-Iinc" /usr/local/share/perl/5.10.1/ExtUtils/xsubpp -typemap /usr/share/perl/5.10/ExtUtils/typemap -typemap typemap SSH2.xs > SSH2.xsc && mv SSH2.xsc SSH2.c
cc -c /usr/local/libssh2/include/ -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -DVERSION=\"0.35\" -DXS_VERSION=\"0.35\" -fPIC "-I/usr/lib/perl/5.10/CORE" SSH2.c
In file included from SSH2.xs:11:0:
ppport.h:3042:0: warning: "PERL_UNUSED_DECL" redefined
/usr/lib/perl/5.10/CORE/perl.h:330:0: note: this is the location of the previous definition
cc: /usr/local/libssh2/include/: linker input file unused because linking not done
Running Mkbootstrap for Net::SSH2 ()
chmod 644 SSH2.bs
rm -f blib/arch/auto/Net/SSH2/SSH2.so
LD_RUN_PATH="/usr/local/libssh2/lib:/lib/x86_64-linux-gnu" cc -shared -O2 -g -L/usr/local/lib -fstack-protector SSH2.o -o blib/arch/auto/Net/SSH2/SSH2.so \
-L/usr/local/libssh2/lib/ -lssh2 -lz -lssl -lcrypto \
chmod 755 blib/arch/auto/Net/SSH2/SSH2.so
cp SSH2.bs blib/arch/auto/Net/SSH2/SSH2.bs
chmod 644 blib/arch/auto/Net/SSH2/SSH2.bs
Manifying blib/man3/Net::SSH2::File.3pm
Manifying blib/man3/Net::SSH2::Dir.3pm
Manifying blib/man3/Net::SSH2::PublicKey.3pm
Manifying blib/man3/Net::SSH2::SFTP.3pm
Manifying blib/man3/Net::SSH2::Listener.3pm
Manifying blib/man3/Net::SSH2::Channel.3pm
Manifying blib/man3/Net::SSH2.3pm
salva#leon:/tmp/salva/Net-SSH2-0.35$ ldd ./blib/arch/auto/Net/SSH2/SSH2.so
linux-vdso.so.1 => (0x00007fff20abc000)
libssh2.so.1 => /usr/local/libssh2/lib/libssh2.so.1 (0x00007f464d52f000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f464d317000)
libssl.so.0.9.8 => /lib/libssl.so.0.9.8 (0x00007f464d09a000)
libcrypto.so.0.9.8 => /lib/libcrypto.so.0.9.8 (0x00007f464cd0b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f464c977000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f464c772000)
/lib64/ld-linux-x86-64.so.2 (0x00007f464d97e000)
salva#leon:/tmp/salva/Net-SSH2-0.35$ sudo make install
...

Varnish DAEMON_OPTS Options Errors

When using inline C with Varnish I've not been able to get /etc/varnish/default
to be happy at startup.
I've tested inline C with varnish for two things: GeoIP detection and Anti-Site-Scraping functions.
The DAEMON_OPTS always complains even though I'm following what other seem
to indicate works fine.
My problem is that this command line start up works:
varnishd -f /etc/varnish/varnish-default.conf -s file,/var/lib/varnish/varnish_storage.bin,512M -T 127.0.0.1:2000 -a 0.0.0.0:8080 -p 'cc_command=exec cc -fpic -shared -Wl,-x -L/usr/include/libmemcached/memcached.h -lmemcached -o %o %s'
But it errors out with trying to start up from default start scripts:
/etc/default/varnish has this in it:
DAEMON_OPTS="-a :8080 \
-T localhost:2000 \
-f /etc/varnish/varnish-default.conf \
-s file,/var/lib/varnish/varnish_storage.bin,512M \
-p 'cc_command=exec cc -fpic -shared -Wl,-x -L/usr/include/libmemcached/memcached.h -lmemcached -o %o %s'"
The error is:
# /etc/init.d/varnish start
Starting HTTP accelerator: varnishd failed!
storage_file: filename: /var/lib/varnish/vbox.local/varnish_storage.bin size 512 MB.
Error:
Unknown parameter "'cc_command".
If I try change the last line to:
-p cc_command='exec cc -fpic -shared -Wl,-x -L/usr/include/libmemcached/memcached.h -lmemcached -o %o %s'"
It's error is now:
# /etc/init.d/varnish start
Starting HTTP accelerator: varnishd failed!
storage_file: filename: /var/lib/varnish/vbox.local/varnish_storage.bin size 512 MB.
Error: Unknown storage method "hared"
It's trying to interpret the '-shared' as -s hared and 'hared' is not a storage type.
For both GeoIP and the Anti-Site-Scrape I've used the exact recommended daemon options
plus have tried all sorts of variations like adding ' and '' but no joy.
Here is a link to the instruction I've followed that work fine except the DAEMON_OPTS part.
http://drcarter.info/2010/04/how-fighting-against-scraping-using-varnish-vcl-inline-c-memcached/
I'm using Debian and the exact DAEMON_OPTS as stated in the instructions.
Can anyone help with a pointer on what's going wrong here?
Even if Jacob will probably never read this, visitors from the future might appreciate what I'm going to write.
I believe I know what's wrong, and it looks like a Debian-specific problem, at least verified on Ubuntu 11.04 and Debian Squeeze.
I traced the execution from my /etc/default/varnish that contains the $DAEMON_OPTS to the init script.
In the init script /etc/init.d/varnish, the start_varnishd() function is:
start_varnishd() {
log_daemon_msg "Starting $DESC" "$NAME"
output=$(/bin/tempfile -s.varnish)
if start-stop-daemon \
--start --quiet --pidfile ${PIDFILE} --exec ${DAEMON} -- \
-P ${PIDFILE} ${DAEMON_OPTS} > ${output} 2>&1; then
log_end_msg 0
else
log_end_msg 1
cat $output
exit 1
fi
rm $output
}
So I modified it to print the full start-stop-daemon command line, like:
start_varnishd() {
log_daemon_msg "Starting $DESC" "$NAME"
output=$(/bin/tempfile -s.varnish)
+ echo "start-stop-daemon --start --quiet --pidfile ${PIDFILE} --exec ${DAEMON} -- -P ${PIDFILE} ${DAEMON_OPTS} > ${output} 2>&1"
if start-stop-daemon \
--start --quiet --pidfile ${PIDFILE} --exec ${DAEMON} -- \
-P ${PIDFILE} ${DAEMON_OPTS} > ${output} 2>&1; then
log_end_msg 0
So I got a command line echoed on STDOUT, and copied-pasted it into my shell. And, surprise! It worked. WTF?
Repeated again to be sure. Yes, it works. Mmh. Could it be another of those bash/dash corner cases?
Let's try feeding the start-stop-daemon command line to bash, and see how it reacts:
start_varnishd() {
log_daemon_msg "Starting $DESC" "$NAME"
output=$(/bin/tempfile -s.varnish)
if bash -c "start-stop-daemon \
--start --quiet --pidfile ${PIDFILE} --exec ${DAEMON} -- \
-P ${PIDFILE} ${DAEMON_OPTS} > ${output} 2>&1"; then
log_end_msg 0
else
log_end_msg 1
cat $output
exit 1
fi
rm $output
}
Yes, it works just fine, at least for my case.
Here's the relevant part of my /etc/default/varnish:
...
## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request. Use a 1GB
# fixed-size cache file.
#
DAEMON_OPTS="-a :6081 \
-T localhost:6082 \
-f /etc/varnish/geoip-example.vcl \
-S /etc/varnish/secret \
-s malloc,100M \
-p 'cc_command=exec cc -fpic -shared -Wl,-x -L/usr/include/GeoIP.h -lGeoIP -o %o %s'"
...
I've seen posts where someone tried to work around this problem by moving the compile command into a separated shell script. Unfortunately that doesn't change the fact that start-stop-daemon is going to pass the $DAEMON_OPTS var through dash, and that will result in mangled options.
Would be something along the lines of:
-p 'cc_command=exec /etc/varnish/compile.sh %o %s'"
And then the compile.sh script as:
#!/bin/sh
cc -fpic -shared -Wl,-x -L/usr/include/GeoIP.h -lGeoIP -o $#
but it doesn't work, so just patch your init scripts, and you're good to go!
Hope you can find this information useful.
You can try using :-
DAEMON_OPTS="-a :8080 \
-T localhost:2000 \
-f /etc/varnish/varnish-default.conf \
-s file,/var/lib/varnish/varnish_storage.bin,512M \
-p cc_command='exec cc -fpic -shared -Wl,-x -L/usr/include/libmemcached/memcached.h -lmemcached -o %o %s'"
Obviously, your startup script interpreting the DAEMON_OPTS is not prepared for whitespace (even within single quotes). At my Fedora (15) installation, the suggested solution works fine; the arguments get interpreted correctly because the "$*" bash parameter is passed in /etc/init.d/varnish and in /etc/init.d/functions in daemon().
Did you get your startup scripts from a package or did you make custom scripts?
This isn't directly related to the question, but you may find yourself here if you are working through the Varnish Tutorial - Put Varnish on port 80.
For recent installs of Varnish on Debian systems the configuration for varnishd startup options can be found in /etc/systemd/system/multi-user.target.wants/varnish.service. The documented way of changing the port via /etc/default/varnish still exists, but is no longer functional unless you change your system to use init scripts rather than systemd.
After you've changed your options in /etc/systemd/system/multi-user.target.wants/varnish.service, don't forget to run systemctl daemon-reload, which will catalog the changes for executing the program.