While trying to build kcat (Github edenhill/kcat - formerly known as kafkacat), I've stumbled upon some issues with library dependencies after the build has been successful, even though the build script has specific parameters for a static build.
The use case is for building a binary that should theoretically work on multiple versions of the OS, where in my case is RHEL 7 & 8.
Building kcat on either RHEL 7 or RHEL 8, will make the build dependent on the libraries (libcrypto.so.1.1 and libssl.so.1.1) from the version hosting the build:
(rhel7)# ldd kcat
./kcat: /lib64/libpthread.so.0: version `GLIBC_2.28' not found (required by ./kcat)
./kcat: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./kcat)
linux-vdso.so.1 => (0x00007ffdb4dfc000)
libm.so.6 => /lib64/libm.so.6 (0x00007f0be4261000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f0be405d000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0be3e41000)
librt.so.1 => /lib64/librt.so.1 (0x00007f0be3c39000)
libz.so.1 => /lib64/libz.so.1 (0x00007f0be3a23000)
libcrypto.so.1.1 => not found
libssl.so.1.1 => not found
libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f0be3806000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f0be35ec000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f0be33b5000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f0be3168000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f0be2e7f000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f0be2c4c000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f0be2a48000)
libcurl.so.4 => /lib64/libcurl.so.4 (0x00007f0be27de000)
libc.so.6 => /lib64/libc.so.6 (0x00007f0be2410000)
/lib64/ld-linux-x86-64.so.2 (0x00007f0be4563000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f0be2200000)
libfreebl3.so => /lib64/libfreebl3.so (0x00007f0be1ffd000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f0be1df9000)
libidn.so.11 => /lib64/libidn.so.11 (0x00007f0be1bc6000)
libssh2.so.1 => /lib64/libssh2.so.1 (0x00007f0be1999000)
libssl3.so => /lib64/libssl3.so (0x00007f0be1740000)
libsmime3.so => /lib64/libsmime3.so (0x00007f0be1518000)
libnss3.so => /lib64/libnss3.so (0x00007f0be11e9000)
libnssutil3.so => /lib64/libnssutil3.so (0x00007f0be0fb9000)
libplds4.so => /lib64/libplds4.so (0x00007f0be0db5000)
libplc4.so => /lib64/libplc4.so (0x00007f0be0bb0000)
libnspr4.so => /lib64/libnspr4.so (0x00007f0be0972000)
liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f0be0763000)
libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f0be050e000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f0be02e7000)
libssl.so.10 => /lib64/libssl.so.10 (0x00007f0be0075000)
libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f0bdfc12000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f0bdf9b0000)
On the building host:
(rhel8)# ldd kcat
linux-vdso.so.1 (0x00007ffd9096e000)
libm.so.6 => /lib64/libm.so.6 (0x00007febf8734000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007febf8530000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007febf8310000)
librt.so.1 => /lib64/librt.so.1 (0x00007febf8108000)
libz.so.1 => /lib64/libz.so.1 (0x00007febf7ef1000)
libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007febf7a08000)
libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007febf7774000)
libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007febf7556000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007febf733f000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007febf7116000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007febf6ec1000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007febf6bd8000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007febf69c1000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007febf67bd000)
libcurl.so.4 => /lib64/libcurl.so.4 (0x00007febf652f000)
libc.so.6 => /lib64/libc.so.6 (0x00007febf616a000)
/lib64/ld-linux-x86-64.so.2 (0x00007febf8ab6000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007febf5f59000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007febf5d55000)
libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007febf5b2e000)
libidn2.so.0 => /lib64/libidn2.so.0 (0x00007febf5910000)
libssh.so.4 => /lib64/libssh.so.4 (0x00007febf56a1000)
libpsl.so.5 => /lib64/libpsl.so.5 (0x00007febf5490000)
libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007febf5242000)
liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007febf5032000)
libbrotlidec.so.1 => /lib64/libbrotlidec.so.1 (0x00007febf4e25000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007febf4bfb000)
libunistring.so.2 => /lib64/libunistring.so.2 (0x00007febf487a000)
libbrotlicommon.so.1 => /lib64/libbrotlicommon.so.1 (0x00007febf4659000)
libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007febf43d5000)
The boostrap script seems to mark the following libraries as dynamic (most of them, if not all of them are installed from RHEL packages):
MKL_DYNAMIC_LIBS -lavro -lm -ldl -lpthread -lrt -lz -lcrypto -lssl -lsasl2 -lm -ldl -lpthread -lrt -lpthread -lrt -lz -lcrypto -lz -ldl -lssl -lcrypto -lz -ldl -lsasl2 -ldl -lresolv -lcrypt -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcurl
Are there any tools, build or compile flags that might enforce building a statically linked binary (that of course, I can still use in my case)?
Thanks!
I am trying to find CPAN version programmatically, but ends up in failure.
Tried with perl %Config & %ENV, but couldn't find anything there.
my $cpanv = `cpan -v`;
# gives: Loading internal null logger. Install Log::Log4perl for logging messages
My goal is to auto configure cpan programmatically. CPAN config prompt varies by versions.
CPAN asks like[similar]:
1. "Do you want to manually configure?[yes]"
2. "Do you want cpan to configure?[no]"
Any way to find CPAN version or avoiding unattended CPAN configuration?
The cpan command-line program is a wrapper around App::Cpan. As far as I can see, the two files always have the same version number. That means you can do something like this:
use App::Cpan;
my $cpanv = $App::Cpan::VERSION;
Update: There are a few concepts to get straight here.
CPAN.pm is a CPAN exploration module that was added to Perl version 5.004. It includes its own basic shell.
App::Cpan is a wrapper around CPAN.pm which adds an improved command interface. It was added to Perl 5.12.
cpan is a tiny script that uses App::Cpan. It was also added to Perl version 5.12. [Update: that's the current version of cpan - there was also an earlier one that was based on CPAN.pm.]
App::Cpan and cpan share the same version number (currently 1.675). CPAN.pm is older and therefore has a larger version number (currently 2.27).
The version number from App::CPAN is what you get when you run cpan -v - so that's what I've given you in my code.
My goal is to auto configure cpan programmatically.
If you're talking about systems with a known configuration, just create (or modify) ~/.cpan/CPAN/MyConfig.pm. For example, the following is mine:
$CPAN::Config = {
'auto_commit' => q[0],
'build_cache' => q[5],
'build_dir' => q[/home/ikegami/.cpan/build],
'build_requires_install_policy' => q[yes],
'cache_metadata' => q[1],
'check_sigs' => q[0],
'commandnumber_in_prompt' => q[1],
'connect_to_internet_ok' => q[1],
'cpan_home' => q[/home/ikegami/.cpan],
'dontload_hash' => { },
'ftp' => q[/usr/bin/ftp],
'ftp_passive' => q[1],
'ftp_proxy' => q[],
'getcwd' => q[cwd],
'gpg' => q[/usr/bin/gpg],
'gzip' => q[/bin/gzip],
'halt_on_failure' => q[0],
'histfile' => q[],
'http_proxy' => q[],
'inactivity_timeout' => q[0],
'index_expire' => q[1],
'inhibit_startup_message' => q[0],
'keep_source_where' => q[/home/ikegami/.cpan/sources],
'load_module_verbosity' => q[none],
'lynx' => q[],
'make' => q[/usr/bin/make],
'make_arg' => q[],
'make_install_arg' => q[],
'make_install_make_command' => q[/usr/bin/make],
'makepl_arg' => q[],
'mbuild_arg' => q[],
'mbuild_install_arg' => q[],
'mbuild_install_build_command' => q[./Build],
'mbuildpl_arg' => q[],
'ncftpget' => q[/usr/bin/ncftpget],
'no_proxy' => q[],
'pager' => q[less],
'perl5lib_verbosity' => q[none],
'prefer_external_tar' => q[1],
'prefer_installer' => q[MB],
'prerequisites_policy' => q[follow],
'scan_cache' => q[atstart],
'shell' => q[/bin/bash],
'show_upload_date' => q[0],
'tar' => q[/bin/tar],
'tar_verbosity' => q[none],
'term_is_latin' => q[1],
'term_ornaments' => q[1],
'trust_test_report_history' => q[0],
'unzip' => q[/usr/bin/unzip],
'urllist' => [q[http://ftp.osuosl.org/pub/CPAN/], q[http://mirrors.hub.co/CPAN/], q[http://cpan.arcticnetwork.ca/]],
'use_sqlite' => q[0],
'version_timeout' => q[15],
'wget' => q[/usr/bin/wget],
'yaml_load_code' => q[0],
};
1;
__END__
Optionally, you can force the defaults to be used, and modify what needs to be changed. For example,
cpan <<<'o conf init
yes
o conf prerequisites_policy ask
o conf commit
quit
'
o conf init causes the initalization process to be started. yes accepts the defaults. o conf prerequisites_policy ask is an example of changing a default. o conf commit saves the changes.
Background
My final objective is to run https://github.com/tkarras/progressive_growing_of_gans/blob/master/h5tool.py
which requires Pillow3.1.1 with libjpeg8.
It occurred an error when checking the MD5 of the first image of the dataset.
It seems to be caused by the different version of libjpeg used in Pillow.
I use the libjpeg9 in Pillow3.1.1 while the suggested version is libjpeg8.
I tested it on the other machine and got these results:
On MY machine
# python
import hashlib
from PIL import Image
md = hashlib.md5()
md.update(Image.open('000001.jpg').tobytes())
print(md.hexdigest())
# 92a0d5f96769de4e68a0ce40e7f0c13f
# It is incorrect
On the other machine
# python
import hashlib
from PIL import Image
md = hashlib.md5()
md.update(Image.open('000001.jpg').tobytes())
print(md.hexdigest())
# 9cad8178d6cb0196b36f7b34bc5eb6d3
# It is correct.
However, when I tried to use md5sum to check these two images on different machines, I got the same code. That means files are exactly the same. So I think the problem might be the version of libjpeg.
On MY machine
ldd /home/liyh/anaconda2/envs/theano/lib/python2.7/site-packages/PIL/_imaging.so
linux-vdso.so.1 => (0x00007ffe4e3f6000)
**libjpeg.so.9 => /home/liyh/anaconda2/envs/theano/lib/libjpeg.so.9 (0x00007fbeb418a000)**
libz.so.1 => /home/liyh/anaconda2/envs/theano/lib/libz.so.1 (0x00007fbeb3f74000)
libtiff.so.5 => /home/liyh/anaconda2/envs/theano/lib/libtiff.so.5 (0x00007fbeb3cf6000)
libpython2.7.so.1.0 => /home/liyh/anaconda2/envs/theano/lib/libpython2.7.so.1.0 (0x00007fbeb38fa000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbeb36dc000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbeb3313000)
liblzma.so.5 => /home/liyh/anaconda2/envs/theano/lib/./liblzma.so.5 (0x00007fbeb30ee000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbeb2ee6000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbeb2be0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbeb29dc000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fbeb27d9000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbeb4631000)
On the other machine
$ldd /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/_imaging.so
linux-vdso.so.1 => (0x00007ffff4791000)
**libjpeg.so.8 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libjpeg.so.8 (0x00007f217847f000)**
libz.so.1 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libz.so.1 (0x00007f2178268000)
libtiff.so.5 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libtiff.so.5 (0x00007f2177fea000)
libpython2.7.so.1.0 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libpython2.7.so.1.0 (0x00007f2177bf0000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f21779b5000)
libc.so.6 => /lib64/libc.so.6 (0x00007f21775f4000)
liblzma.so.5 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../.././liblzma.so.5 (0x00007f21773ce000)
librt.so.1 => /lib64/librt.so.1 (0x00007f21771c6000)
libm.so.6 => /lib64/libm.so.6 (0x00007f2176ec4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f2176cc0000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f2176abc000)
/lib64/ld-linux-x86-64.so.2 (0x00007f217890e000)
I tried to re-install Pillow==3.1.1, but it kept using libjpeg9 instead of libjpeg8.
If I delete the /home/liyh/anaconda2/envs/theano/lib/libjpeg.so.9 and add libjpeg.so.8 into the directory and re-install Pillow==3.1.1, I got
ImportError: libjpeg.so.9: cannot open shared object file: No such file or directory
My question is:
How can I install Pillow3.1.1 with libjpeg8, or
How can I generate the correct MD5 code using libjpeg9, or
What other possible reasons that cause the different MD5 code?
Ubuntu 4.8.4-2ubuntu1~14.04.3
Python 2.7
Edited:
The image I used is the first image (000001.jpg) in CelebA dataset ("in-the-wild" version, not the "align&cropped").
$md5sum /data2/pub_data/CelebA/Img/img_celeba/000001.jpg
39d4f4f149cf541601ad334ea98c822d
I happened to solve this problem by installing pillow==3.1.1 with conda instead of pip. In this way, pillow uses libjpeg8 as its jpeg decoder as we want.
conda create -n new_env
source activate new_env
conda install pillow==3.1.1
conda install ...(other needed libs)
I am trying to use the latest mongodb binary bits, but for some reason it doesn't load the crypto | ssl libraries.
ldd bin/mongod
bin/mongod: /lib/x86_64-linux-gnu/libcrypto.so.10: version `libcrypto.so.10' not found (required by bin/mongod)
bin/mongod: /lib/x86_64-linux-gnu/libssl.so.10: version `libssl.so.10' not found (required by bin/mongod)
linux-vdso.so.1 => (0x00007fff186f4000)
libssl.so.10 => /lib/x86_64-linux-gnu/libssl.so.10 (0x00007f08a48c1000)
libcrypto.so.10 => /lib/x86_64-linux-gnu/libcrypto.so.10 (0x00007f08a44e6000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f08a42dd000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f08a40d9000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f08a3dd5000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f08a3ace000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f08a38b8000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f08a369a000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f08a32d4000)
/lib64/ld-linux-x86-64.so.2 (0x00007f08a4b3d000)
This error message is typically received when the enterprise dependencies have not been installed prior to startup of mongo.
This is documented here - https://docs.mongodb.org/manual/tutorial/install-mongodb-enterprise-on-red-hat/#install-mongodb-enterprise-from-tarball
PostgreSQL 8.4 Fails to Start on Ubuntu 10.10
sudo /etc/init.d/postgresql start
* Starting PostgreSQL 8.4 database server
* Error: could not exec /usr/lib/postgresql/8.4/bin/pg_ctl /usr/lib/postgresql/8.4/bin/pg_ctl start -D /var/lib/postgresql/8.4/main -l /var/log/postgresql/postgresql-8.4-main.log -s -o -c config_file="/etc/postgresql/8.4/main/postgresql.conf" :
running ldd on /usr/lib/postgresql/8.4/bin/pg_ctl
ldd /usr/lib/postgresql/8.4/bin/pg_ctl
linux-gate.so.1 => (0xb77e4000)
libpq.so.5 => /usr/lib/libpq.so.5 (0xb77a2000)
libc.so.6 => /lib/libc.so.6 (0xb7645000)
libssl.so.0.9.8 => /lib/libssl.so.0.9.8 (0xb75fb000)
libcrypto.so.0.9.8 => /lib/libcrypto.so.0.9.8 (0xb74ac000)
libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0xb73fd000)
libcom_err.so.2 => /lib/libcom_err.so.2 (0xb73f9000)
libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0xb73ca000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0xb7398000)
libldap_r-2.4.so.2 => /usr/lib/libldap_r-2.4.so.2 (0xb7351000)
libpthread.so.0 => /lib/libpthread.so.0 (0xb7337000)
/lib/ld-linux.so.2 (0xb77e5000)
libdl.so.2 => /lib/libdl.so.2 (0xb7333000)
libz.so.1 => /lib/libz.so.1 (0xb731e000)
libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0xb72fa000)
libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0xb72f1000)
libkeyutils.so.1 => /lib/libkeyutils.so.1 (0xb72ed000)
libresolv.so.2 => /lib/libresolv.so.2 (0xb72d9000)
liblber-2.4.so.2 => /usr/lib/liblber-2.4.so.2 (0xb72cc000)
libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb72b4000)
libgnutls.so.26 => /usr/lib/libgnutls.so.26 (0xb7218000)
libtasn1.so.3 => /usr/lib/libtasn1.so.3 (0xb7207000)
libgcrypt.so.11 => /lib/libgcrypt.so.11 (0xb7193000)
libgpg-error.so.0 => /lib/libgpg-error.so.0 (0xb718e000)
I have checked permissions and have given root appropriate permissions, and I am trying to run postgresql as root.
PostgreSQL refuses to run as root. This is so that bugs in the back-end, plus bugs in functions written in untrusted languages, cannot be used to do arbitrary actions over the whole system. PostgreSQL must run as a relatively limited user, usually the postgres user and the start-up scripts normally enforce this.
Grant permissions to the postgres user rather than root. Then everything should work fine.