Cloudbees dev cloud + Postgresql + rubygems pg (native extension) how to use? - postgresql

I want to use postgres in cloudbees dev cloud, I read this article:
http://wiki.cloudbees.com/bin/view/DEV/PostgreSQL
But, I run bundle, I got error:
Installing pg (0.15.1) .......
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native
extension.
/usr/bin/ruby extconf.rb checking for pg_config... no No pg_config... trying anyway. If building fails, please try again with
--with-pg-config=/path/to/pg_config checking for libpq-fe.h... no Can't find the 'libpq-fe.h header
* extconf.rb failed * Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check
the mkmf.log file for more details. You may need configuration
options.
Provided configuration options: --with-opt-dir --without-opt-dir
--with-opt-include --without-opt-include=${opt-dir}/include
--with-opt-lib --without-opt-lib=${opt-dir}/ --with-make-prog
--without-make-prog --srcdir=. --curdir --ruby=/usr/bin/ruby
--with-pg --without-pg --with-pg-dir --without-pg-dir
--with-pg-include --without-pg-include=${pg-dir}/include
--with-pg-lib --without-pg-lib=${pg-dir}/ --with-pg-config
--without-pg-config --with-pg_config --without-pg_config
Gem files will remain installed in
/home/jenkins/.gem/ruby/1.9.1/gems/pg-0.15.1 for inspection. Results
logged to
/home/jenkins/.gem/ruby/1.9.1/gems/pg-0.15.1/ext/gem_make.out An error
occurred while installing pg (0.15.1), and Bundler cannot continue.
Make sure that gem install pg -v '0.15.1' succeeds before bundling.
What do I do next?

There are two approaches to resolving this:
1) Use the workaround above, by adding
PATH=/scratch/jenkins/postgresqls/postgresql-9.2.4-x86_64/bin:$PATH
This is not ideal as we are just about to move that path to support pull-request builds
2) Combine the two steps
In your build you will need to combine the PostgreSQL and Ruby installation steps so that the environment is configured properly.
I personally recommend using a script to do the build - ./bin/run-ci - as opposed to typing everything into Jenkins.
This keeps your build process change controlled and documented - and lets you roll out changes across multiple build servers quite rapidly.

I use single shell script, and I declare once at the top:
export POSTGRESQL_VERSION=9.2.4
curl -s -o use-postgresql https://repository-cloudbees.forge.cloudbees.com/distributions/ci-addons/postgresql/use-postgresql
source ./use-postgresql
Then I don't necessary pass environment variables between shells.

My work around:
Set PATH above bundle, and 9.2.4 is my installing version.
PATH=/scratch/hudson/postgresqls/postgresql-9.2.4-x86_64/bin:$PATH
Then I can bundle completed :)

Related

Specify RPM build order in mock

TLDR: Ensure to keep system OS up-to-date to help ensure consistency with current spec files.
Symptom
When rebuilding PostgreSQL 11.1 SRPM using mock, the build fails with:
BUILDSTDERR: /builddir/build/BUILD/postgresql-11.1/src/bin/psql/command.c:1814 undefined reference to `PQencryptPasswordConn`
NB: PQencryptPasswordConn is a libpq.so function (provided by postgresql-devel-10.3-5.fc27.x86_64 on my system...outside the mock chroot environment). Unless I'm mistaken, the Postgresql SRPM builds the postgresql-devel RPM along with others.
Steps to reproduce
I ran the following to rebuild the SRPM before attempting to apply any patches not already present in the SRPM:
# Obtain SRPM source
git clone https://src.fedoraproject.org/rpms/postgresql.git
cd postgresql
# Download local copies of SRPM sources
wget $(spectool -S *.spec | awk '/^Source.*:\/\//{IFS=" "; print $2}')
# ...check SHAs of downloaded sources...
# Run SRPM-specific prep scripts
./generate-pdf.sh
./generate-sources.sh
# Generate the SRPM
mock --root=fedora-27-x86_64 --resultdir="./SRPMS" --buildsrpm --spec postgresql.spec --sources .
# >>> Everything seems to work fine up to this point <<<
# Build the RPM inside mock chroot
mock --root=fedora-27-x86_64 --rebuild ./SRPMS/postgresql-11.1-4.fc27.src.rpm
# !!! Fail here (with symptom above) !!!
The Problem
I have so far been unable to have mock load the appropriate libpq library headers into the chroot environment to make sure rpmbuild builds against the libpq that contains the PQencryptPasswordConn header (which appears to exist on my system outside the build environment):
grep -lr "PQencryptPasswordConn" /usr/include
# /usr/include/libpq-fe.h
grep -lr "PQencryptPasswordConn" /var/lib/mock/fedora-27-x86_64/root/usr/include
# (Nothing returned)
When reviewing mock's installed_pkgs.log, the following were installed (the latter of which I expect would provide a version of libpq headers):
postgresql-libs-9.6.10-3.fc27.x86_64
postgresql-devel-9.6.10-3.fc27.x86_64
However, I cannot find a way to install the postgresql-* packages into the chroot environment that contain the updated library headers.
The Ask
Since postgresql SRPM is supposed to build postgresql-devel RPM, I think that mock will need to build and install the postgresql-devel RPM in the chroot before rpmbuild attempts to compile psql/command.c so that the latter compilation finds the appropriate library headers (unless the build process is intelligent enough to identify new libraries currently under build).
How can I best accomplish this (would prefer to avoid multiple mock calls for each RPM package built from the SRPM unless that's the only way to go)?
Please note that the build process on my system spawns multiple processes to parallel compilations.
I have also tried to use mockchain —recurse without success.
System Info
Linux 4.16.6-202.fc27.x86_64
First hint, you use the latest postgresql.spec version, but you try to build it against rather old (in fact unsupported nowadays) version 27 of Fedora distribution. I'd encourage you to migrate to a newer version of Fedora, or at least checkout the branch f27 in the same RPM git repository.
Second hint, we changed the layout of PostgreSQL packaging in Fedora 30+. We've cut out the library (libpq.so) into separate package, per announcement.
How to continue; always checkout appropriate branch based on what Fedora you build against, and adjust the spec file appropriately (checkout f27 and update to PostgreSQL 11.1 in this case).
JFTR (might help), there already is a testing modular build of PostgreSQL 11 against Fedora 28+, and the build scripts are maintained in separate branch stream-postgresql-11. With a bit of luck, you would be able to build that branch against old Fedora 27, too. Note that this version of postgresql.spec file is a little bit complicated (it needs to be because we build it against different versions of Fedora).

gdcmanon - Requested cryptoraphic library not configured

I am using gdcmanon to anonimize dicom files like the following:
LD_LIBRARY_PATH=../lib ./gdcmanon --certificate CA_cert.cer -e ./dump ./dump/anon
But it gives the following error:
Error: In /home/travis/build/malaterre/GDCM/Source/Common/gdcmCryptoFactory.cxx, line 61, function static gdcm::CryptoFactory* gdcm::CryptoFactory::GetFactoryInstance(gdcm::CryptoFactory::CryptoLib)
No crypto factory registered with id 1
Requested cryptoraphic library not configured.
How can I solve this?
The error is because of binaries you use, it looks like SSL dependencies are not satisfied.
As a solution (workaround?), you can get the binaries via apt system. For Ubuntu 16.04 Xenial distribution, there is version 2.6.3 of libgdcm-tools package.
Another possibility is to compile gdcm from scratch, but you need development files of OpenSSL (apt install libssl-dev). Moreover, to use anonymisation with certificates, in CMake system set GDCM_USE_SYSTEM_OPENSSL=YES.

Failure: gem install rbczmq due to autogen error

I am trying to add IRuby kernel into my Jupyter so that I can write notes on Ruby, as I just started learning to solve problems in Ruby.
On the IRuby doc, I am supposed to do the following commands:
gem install rbczmq
gem install iruby
However, when I do gem install rbczmq I get permission denied. When I do sudo gem install rbczmq, I get the following:
Building native extensions. This could take a while...
ERROR: Error installing rbczmq:
ERROR: Failed to build gem native extension.
current directory: /var/lib/gems/2.3.0/gems/rbczmq-1.7.9/ext/rbczmq
/usr/bin/ruby2.3 -r ./siteconf20160623-27886-pogml8.rb extconf.rb
"./autogen.sh"
autogen.sh: error: could not find libtool. libtool is required to run autogen.sh.
ZeroMQ autogen failed!
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/$(RUBY_BASE_NAME)2.3
--with-system-libs
--without-system-libs
extconf failed, exit code 1
Gem files will remain installed in /var/lib/gems/2.3.0/gems/rbczmq-1.7.9 for inspection.
Results logged to /var/lib/gems/2.3.0/extensions/x86_64- linux/2.3.0/rbczmq-1.7.9/gem_make.out
I would like to know how I can fix this problem and install IRuby to my Jupyter? Thank you!
I had the same problem, after hours of trial and error, I finally manage to install it.
I used sudo apt-get install libtool-bin and then sudo gem install rbczmq
I hope this helps

Sinatra/Rack/Bundler application unable to install postgres gem

I'm trying to install the postgres gem "pg" in my rack/bundler app and keep failing when attempting to install through RubyMine.
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using addressable (2.3.5)
Using bcrypt-ruby (3.1.2)
Using data_objects (0.10.13)
Using dm-core (1.2.1)
Using dm-aggregates (1.2.0)
Using dm-constraints (1.2.0)
Using dm-migrations (1.2.0)
Using fastercsv (1.5.5)
Using json (1.8.1)
Using json_pure (1.8.0)
Using multi_json (1.8.2)
Using dm-serializer (1.2.2)
Using dm-timestamps (1.2.0)
Using dm-transactions (1.2.0)
Using stringex (1.5.1)
Using uuidtools (2.1.4)
Using dm-types (1.2.2)
Using dm-validations (1.2.0)
Using datamapper (1.2.0)
Using dm-do-adapter (1.2.0)
Installing do_postgres (0.10.13)
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
/Users/adam419/.rvm/rubies/ruby-2.0.0-p195/bin/ruby extconf.rb
checking for main() in -lpq... yes
checking for libpq-fe.h... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/Users/adam419/.rvm/rubies/ruby-2.0.0-p195/bin/ruby
--with-pgsql-server-dir
--without-pgsql-server-dir
--with-pgsql-server-include
--without-pgsql-server-include=${pgsql-server-dir}/include
--with-pgsql-server-lib
--without-pgsql-server-lib=${pgsql-server-dir}/
--with-pgsql-client-dir
--without-pgsql-client-dir
--with-pgsql-client-include
--without-pgsql-client-include=${pgsql-client-dir}/include
--with-pgsql-client-lib
--without-pgsql-client-lib=${pgsql-client-dir}/
--with-pqlib
--without-pqlib
Could not find PostgreSQL build environment (libraries & headers): Makefile not created
Gem files will remain installed in /Users/adam419/.rvm/gems/ruby-2.0.0-p195#global/gems/do_postgres-0.10.13 for inspection.
Results logged to /Users/adam419/.rvm/gems/ruby-2.0.0-p195#global/gems/do_postgres-0.10.13/ext/do_postgres/gem_make.out
An error occurred while installing do_postgres (0.10.13), and Bundler cannot
continue.
Make sure that `gem install do_postgres -v '0.10.13'` succeeds before bundling.
So for some reason, with my current project, if I install a gem through the command line it doesn't get added to the External Libraries folder in Rubymine, and have so far been relying on it's own bundle/gem installer. So from reading other questions, I saw to download the Postgres93 support for Mac os x and in the command line I attempted to install postgres gem by specifying the path to the pg_config file, and it downloaded successfully, but didn't get added to the libraries in the project in Rubymine.
sudo gem install pg -- --with-pg config=/Applications/Postgres93.app/Contents/MacOS/bin/pg_config
The above command worked, but didn't cause the gem to appear in Rubymine, and thus I am still unable to push to Heroku. So when attempting to give the additional arguments to the bundle installer I get syntax errors when entering in
--with-pg-config=/Applications/Postgres93.app/Contents/MacOS/bin/pg_config
Any ideas how to resolve this?
Solution -- Reinstalled Postgresql with homebrew

Jruby gem error win32-service for rsense

I am trying to install Rsense, I am following the directions from here Rsense Docs
RSENSE_HOME is set but i cannot install the jruby win32-service to complete the steps to have the rsense service started at boot.
This is the full error, I also ran as error suggested specifying jruby version. Any ideas on completing the install?
C:\Users\RenshawFamily>echo %RSENSE_HOME%
C:\Rsense
C:\Users\RenshawFamily>jruby -S gem install win32-service
Fetching: win32-api-1.4.8.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing win32-service:
ERROR: Failed to build gem native extension.
c:/jruby-1.6.3/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf' library very wel
l.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
checking for strncpy_s()... IOError: Cannot run program "cc" (in directory "C:\j
ruby-1.6.3\lib\ruby\gems\1.8\gems\win32-api-1.4.8\ext"): CreateProcess error=2,
The system cannot find the file specified
popen at org/jruby/RubyIO.java:3540
xpopen at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:323
open at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:260
xpopen at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:316
egrep_cpp at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:538
have_func at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:758
checking_for at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:652
postpone at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:286
open at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:260
postpone at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:286
open at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:260
postpone at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:282
checking_for at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:651
have_func at c:/jruby-1.6.3/lib/ruby/site_ruby/shared/mkmf.rb:755
(root) at extconf.rb:9
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=c:/jruby-1.6.3/bin/jruby
Gem files will remain installed in c:/jruby-1.6.3/lib/ruby/gems/1.8/gems/win32-a
pi-1.4.8 for inspection.
Results logged to c:/jruby-1.6.3/lib/ruby/gems/1.8/gems/win32-api-1.4.8/ext/gem_
make.out
C:\Users\RenshawFamily>jruby -S gem install win32-service --ruby=c:/jruby-1.6.3/
bin/jruby
ERROR: While executing gem ... (OptionParser::InvalidOption)
invalid option: --ruby=c:/jruby-1.6.3/bin/jruby
win32-service uses C extensions. Gems that use C extensions are generally not compatible with JRuby, although there are efforts being made to address that - see here for background
Are you able to use MRI/CRuby for your project?
Or you could use a java based service management lib, but then that might not work with emacs/rsense.
Or you could roll up your sleeves and make win32-service JRuby compatible!
I had the same problem when installing the win-32-service gem. This solved it:
gem install win32-service --platform=mswin32
So your command should probably be
jruby -S gem install win32-service --platform=mswin32
although I'm guessing here.