While trying to build kcat (Github edenhill/kcat - formerly known as kafkacat), I've stumbled upon some issues with library dependencies after the build has been successful, even though the build script has specific parameters for a static build.
The use case is for building a binary that should theoretically work on multiple versions of the OS, where in my case is RHEL 7 & 8.
Building kcat on either RHEL 7 or RHEL 8, will make the build dependent on the libraries (libcrypto.so.1.1 and libssl.so.1.1) from the version hosting the build:
(rhel7)# ldd kcat
./kcat: /lib64/libpthread.so.0: version `GLIBC_2.28' not found (required by ./kcat)
./kcat: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./kcat)
linux-vdso.so.1 => (0x00007ffdb4dfc000)
libm.so.6 => /lib64/libm.so.6 (0x00007f0be4261000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f0be405d000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0be3e41000)
librt.so.1 => /lib64/librt.so.1 (0x00007f0be3c39000)
libz.so.1 => /lib64/libz.so.1 (0x00007f0be3a23000)
libcrypto.so.1.1 => not found
libssl.so.1.1 => not found
libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f0be3806000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f0be35ec000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f0be33b5000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f0be3168000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f0be2e7f000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f0be2c4c000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f0be2a48000)
libcurl.so.4 => /lib64/libcurl.so.4 (0x00007f0be27de000)
libc.so.6 => /lib64/libc.so.6 (0x00007f0be2410000)
/lib64/ld-linux-x86-64.so.2 (0x00007f0be4563000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f0be2200000)
libfreebl3.so => /lib64/libfreebl3.so (0x00007f0be1ffd000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f0be1df9000)
libidn.so.11 => /lib64/libidn.so.11 (0x00007f0be1bc6000)
libssh2.so.1 => /lib64/libssh2.so.1 (0x00007f0be1999000)
libssl3.so => /lib64/libssl3.so (0x00007f0be1740000)
libsmime3.so => /lib64/libsmime3.so (0x00007f0be1518000)
libnss3.so => /lib64/libnss3.so (0x00007f0be11e9000)
libnssutil3.so => /lib64/libnssutil3.so (0x00007f0be0fb9000)
libplds4.so => /lib64/libplds4.so (0x00007f0be0db5000)
libplc4.so => /lib64/libplc4.so (0x00007f0be0bb0000)
libnspr4.so => /lib64/libnspr4.so (0x00007f0be0972000)
liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f0be0763000)
libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f0be050e000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f0be02e7000)
libssl.so.10 => /lib64/libssl.so.10 (0x00007f0be0075000)
libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f0bdfc12000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f0bdf9b0000)
On the building host:
(rhel8)# ldd kcat
linux-vdso.so.1 (0x00007ffd9096e000)
libm.so.6 => /lib64/libm.so.6 (0x00007febf8734000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007febf8530000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007febf8310000)
librt.so.1 => /lib64/librt.so.1 (0x00007febf8108000)
libz.so.1 => /lib64/libz.so.1 (0x00007febf7ef1000)
libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007febf7a08000)
libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007febf7774000)
libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007febf7556000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007febf733f000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007febf7116000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007febf6ec1000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007febf6bd8000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007febf69c1000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007febf67bd000)
libcurl.so.4 => /lib64/libcurl.so.4 (0x00007febf652f000)
libc.so.6 => /lib64/libc.so.6 (0x00007febf616a000)
/lib64/ld-linux-x86-64.so.2 (0x00007febf8ab6000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007febf5f59000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007febf5d55000)
libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007febf5b2e000)
libidn2.so.0 => /lib64/libidn2.so.0 (0x00007febf5910000)
libssh.so.4 => /lib64/libssh.so.4 (0x00007febf56a1000)
libpsl.so.5 => /lib64/libpsl.so.5 (0x00007febf5490000)
libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007febf5242000)
liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007febf5032000)
libbrotlidec.so.1 => /lib64/libbrotlidec.so.1 (0x00007febf4e25000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007febf4bfb000)
libunistring.so.2 => /lib64/libunistring.so.2 (0x00007febf487a000)
libbrotlicommon.so.1 => /lib64/libbrotlicommon.so.1 (0x00007febf4659000)
libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007febf43d5000)
The boostrap script seems to mark the following libraries as dynamic (most of them, if not all of them are installed from RHEL packages):
MKL_DYNAMIC_LIBS -lavro -lm -ldl -lpthread -lrt -lz -lcrypto -lssl -lsasl2 -lm -ldl -lpthread -lrt -lpthread -lrt -lz -lcrypto -lz -ldl -lssl -lcrypto -lz -ldl -lsasl2 -ldl -lresolv -lcrypt -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcurl
Are there any tools, build or compile flags that might enforce building a statically linked binary (that of course, I can still use in my case)?
Thanks!
Related
I have a CentOS 8 server and I try to connect to a PostgreSQL DB.
My python-file looks like
def read_df(query):
"""
Function for fetching data from the DB.
"""
con_str = (
"DRIVER={PostgreSQL};"
"DATABASE=db;"
f"UID={user};"
f"PWD={pwd};"
f"SERVER={host};"
f"PORT={port};"
)
with pyodbc.connect(con_str) as con:
df = pd.read_sql(query,con=con)
return df
and /etc/odbcinst.ini
[PostgreSQL]
Driver = /usr/lib64/psqlodbcw.so
Setup = /usr/lib64/libodbcpsqlS.so
Driver64 = /usr/lib64/psqlodbcw.so
Setup64 = /usr/lib64/libodbcpsqlS.so
the files specified above do exist:
>ls /usr/lib64 | grep psq
psqlodbcw.so
psqlodbca.so
psqlodbc.so
libodbcpsqlS.so
and I can access the file
with open("/usr/lib64/psqlodbcw.so","r") as f:
print("It works")
"It works"
but running read_df throws
File "/home/user/db_connect.py", line 192, in read_df
with pyodbc.connect(con_str) as con:
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/usr/lib64/psqlodbcw.so' : file not found (0) (SQLDriverConnect)")
I can see the file also from python import os; os.listdir("/usr/lib64") - and am completely lost.
EDIT: Running ldd /usr/lib64/psqlodbcw.so yields
$ ldd /usr/lib64/psqlodbcw.so
linux-vdso.so.1 (0x00007ffce158b000)
libpq.so.5 => /lib64/libpq.so.5 (0x00007f1e90be1000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1e909c1000)
libodbcinst.so.2 => /lib64/libodbcinst.so.2 (0x00007f1e907ab000)
libc.so.6 => /lib64/libc.so.6 (0x00007f1e903e6000)
libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007f1e90152000)
libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007f1e8fc6c000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f1e8fa17000)
libldap_r-2.4.so.2 => /lib64/libldap_r-2.4.so.2 (0x00007f1e8f7c0000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1e910bb000)
libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f1e8f5b6000)
libz.so.1 => /lib64/libz.so.1 (0x00007f1e8f39f000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f1e8f19b000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f1e8eeb2000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f1e8ec9b000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f1e8ea97000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f1e8e886000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f1e8e682000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f1e8e46b000)
liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f1e8e25b000)
libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f1e8e03d000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f1e8de13000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f1e8dbea000)
libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f1e8d966000)
Background
My final objective is to run https://github.com/tkarras/progressive_growing_of_gans/blob/master/h5tool.py
which requires Pillow3.1.1 with libjpeg8.
It occurred an error when checking the MD5 of the first image of the dataset.
It seems to be caused by the different version of libjpeg used in Pillow.
I use the libjpeg9 in Pillow3.1.1 while the suggested version is libjpeg8.
I tested it on the other machine and got these results:
On MY machine
# python
import hashlib
from PIL import Image
md = hashlib.md5()
md.update(Image.open('000001.jpg').tobytes())
print(md.hexdigest())
# 92a0d5f96769de4e68a0ce40e7f0c13f
# It is incorrect
On the other machine
# python
import hashlib
from PIL import Image
md = hashlib.md5()
md.update(Image.open('000001.jpg').tobytes())
print(md.hexdigest())
# 9cad8178d6cb0196b36f7b34bc5eb6d3
# It is correct.
However, when I tried to use md5sum to check these two images on different machines, I got the same code. That means files are exactly the same. So I think the problem might be the version of libjpeg.
On MY machine
ldd /home/liyh/anaconda2/envs/theano/lib/python2.7/site-packages/PIL/_imaging.so
linux-vdso.so.1 => (0x00007ffe4e3f6000)
**libjpeg.so.9 => /home/liyh/anaconda2/envs/theano/lib/libjpeg.so.9 (0x00007fbeb418a000)**
libz.so.1 => /home/liyh/anaconda2/envs/theano/lib/libz.so.1 (0x00007fbeb3f74000)
libtiff.so.5 => /home/liyh/anaconda2/envs/theano/lib/libtiff.so.5 (0x00007fbeb3cf6000)
libpython2.7.so.1.0 => /home/liyh/anaconda2/envs/theano/lib/libpython2.7.so.1.0 (0x00007fbeb38fa000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbeb36dc000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbeb3313000)
liblzma.so.5 => /home/liyh/anaconda2/envs/theano/lib/./liblzma.so.5 (0x00007fbeb30ee000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbeb2ee6000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbeb2be0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbeb29dc000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fbeb27d9000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbeb4631000)
On the other machine
$ldd /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/_imaging.so
linux-vdso.so.1 => (0x00007ffff4791000)
**libjpeg.so.8 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libjpeg.so.8 (0x00007f217847f000)**
libz.so.1 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libz.so.1 (0x00007f2178268000)
libtiff.so.5 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libtiff.so.5 (0x00007f2177fea000)
libpython2.7.so.1.0 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libpython2.7.so.1.0 (0x00007f2177bf0000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f21779b5000)
libc.so.6 => /lib64/libc.so.6 (0x00007f21775f4000)
liblzma.so.5 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../.././liblzma.so.5 (0x00007f21773ce000)
librt.so.1 => /lib64/librt.so.1 (0x00007f21771c6000)
libm.so.6 => /lib64/libm.so.6 (0x00007f2176ec4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f2176cc0000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f2176abc000)
/lib64/ld-linux-x86-64.so.2 (0x00007f217890e000)
I tried to re-install Pillow==3.1.1, but it kept using libjpeg9 instead of libjpeg8.
If I delete the /home/liyh/anaconda2/envs/theano/lib/libjpeg.so.9 and add libjpeg.so.8 into the directory and re-install Pillow==3.1.1, I got
ImportError: libjpeg.so.9: cannot open shared object file: No such file or directory
My question is:
How can I install Pillow3.1.1 with libjpeg8, or
How can I generate the correct MD5 code using libjpeg9, or
What other possible reasons that cause the different MD5 code?
Ubuntu 4.8.4-2ubuntu1~14.04.3
Python 2.7
Edited:
The image I used is the first image (000001.jpg) in CelebA dataset ("in-the-wild" version, not the "align&cropped").
$md5sum /data2/pub_data/CelebA/Img/img_celeba/000001.jpg
39d4f4f149cf541601ad334ea98c822d
I happened to solve this problem by installing pillow==3.1.1 with conda instead of pip. In this way, pillow uses libjpeg8 as its jpeg decoder as we want.
conda create -n new_env
source activate new_env
conda install pillow==3.1.1
conda install ...(other needed libs)
I am trying to use the latest mongodb binary bits, but for some reason it doesn't load the crypto | ssl libraries.
ldd bin/mongod
bin/mongod: /lib/x86_64-linux-gnu/libcrypto.so.10: version `libcrypto.so.10' not found (required by bin/mongod)
bin/mongod: /lib/x86_64-linux-gnu/libssl.so.10: version `libssl.so.10' not found (required by bin/mongod)
linux-vdso.so.1 => (0x00007fff186f4000)
libssl.so.10 => /lib/x86_64-linux-gnu/libssl.so.10 (0x00007f08a48c1000)
libcrypto.so.10 => /lib/x86_64-linux-gnu/libcrypto.so.10 (0x00007f08a44e6000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f08a42dd000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f08a40d9000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f08a3dd5000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f08a3ace000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f08a38b8000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f08a369a000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f08a32d4000)
/lib64/ld-linux-x86-64.so.2 (0x00007f08a4b3d000)
This error message is typically received when the enterprise dependencies have not been installed prior to startup of mongo.
This is documented here - https://docs.mongodb.org/manual/tutorial/install-mongodb-enterprise-on-red-hat/#install-mongodb-enterprise-from-tarball
A piece of software I compiled on a CentOS 6 machine is giving me an error that I don't see with the exact same software compiled on my Ubuntu 14.10 laptop:
auxgen: error while loading shared libraries: libquadmath.so.0: cannot open shared object file: No such file or directory
The library in question does seem to be there on the CentOS machine:
$ locate libquadmath
/usr/local/lib/libquadmath.a
/usr/local/lib/libquadmath.la
/usr/local/lib/libquadmath.so
/usr/local/lib/libquadmath.so.0
/usr/local/lib/libquadmath.so.0.0.0
/usr/local/lib64/libquadmath.a
/usr/local/lib64/libquadmath.la
/usr/local/lib64/libquadmath.so
/usr/local/lib64/libquadmath.so.0
/usr/local/lib64/libquadmath.so.0.0.0
/usr/local/share/info/libquadmath.info
$ ldd /usr/local/bin/auxgen
/usr/local/bin/auxgen: /usr/lib64/libgfortran.so.3: version `GFORTRAN_1.4' not found (required by /usr/local/bin/auxgen)
linux-vdso.so.1 => (0x00007fff6cdb0000)
libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x0000003b64a00000)
libm.so.6 => /lib64/libm.so.6 (0x000000330d000000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003c0b400000)
libquadmath.so.0 => not found
libc.so.6 => /lib64/libc.so.6 (0x000000330c400000)
/lib64/ld-linux-x86-64.so.2 (0x000000330c000000)
On my Ubuntu 14.10 laptop, where it works:
$ ldd /usr/local/bin/auxgen
linux-vdso.so.1 => (0x00007fff7aaf2000)
libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007fd71b5da000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd71b215000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fd71afd6000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd71acd0000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd71aab9000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd71b922000)
On the Ubuntu laptop where it does work:
$ ldconfig -p | grep 'quad\|gfort'
libquadmath.so.0 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libquadmath.so.0
libgfortran.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libgfortran.so.3
On the CentOS where it doesn't work:
$ ldconfig -p | grep 'quad\|gfort'
libgfortran.so.3 (libc6,x86-64) => /usr/lib64/libgfortran.so.3
libgfortran.so.3 (libc6) => /usr/lib/libgfortran.so.3
Any ideas?
I am using MongoDB Solaris package. When I use ldd command to check the mongod, the output is:
bash-3.2# ldd mongod
libsocket.so.1 => /lib/64/libsocket.so.1
libresolv.so.2 => /lib/64/libresolv.so.2
librt.so.1 => /lib/64/librt.so.1
libnsl.so.1 => /lib/64/libnsl.so.1
libstdc++.so.6 => /usr/sfw/lib/amd64/libstdc++.so.6
libm.so.2 => /lib/64/libm.so.2
libgcc_s.so.1 => /usr/sfw/lib/amd64/libgcc_s.so.1
libgcc_s.so.1 (GCC_4.0.0) => (version not found)
libpthread.so.1 => /lib/64/libpthread.so.1
libc.so.1 => /lib/64/libc.so.1
libaio.so.1 => /lib/64/libaio.so.1
libmd.so.1 => /lib/64/libmd.so.1
libmp.so.2 => /lib/64/libmp.so.2
libscf.so.1 => /lib/64/libscf.so.1
libdoor.so.1 => /lib/64/libdoor.so.1
libuutil.so.1 => /lib/64/libuutil.so.1
libgen.so.1 => /lib/64/libgen.so.1
From the output, I notice the following lines:
libgcc_s.so.1 => /usr/sfw/lib/amd64/libgcc_s.so.1
libgcc_s.so.1 (GCC_4.0.0) => (version not found)
Per my understanding, I think it means the version of libgcc_s.so.1 must be 4.0.0 or higher. Does MongoDB require the gcc version be 4.0.0 or higher? I can't find this requirement in its website.
Yes.
From the source code on Github, we can see:
#if __GNUC__ < 4
#error gcc < 4 not supported
#endif
https://github.com/mongodb/mongo/blob/master/src/mongo/pch.cpp
I can't find this in the official docs though, but the docs do state that you need GNU C++ Toolchain, which I would imagine implies whatever is the most current release.