How to switch libjpeg9 used in Pillow3.1.1 to libjpeg8? - python-imaging-library

Background
My final objective is to run https://github.com/tkarras/progressive_growing_of_gans/blob/master/h5tool.py
which requires Pillow3.1.1 with libjpeg8.
It occurred an error when checking the MD5 of the first image of the dataset.
It seems to be caused by the different version of libjpeg used in Pillow.
I use the libjpeg9 in Pillow3.1.1 while the suggested version is libjpeg8.
I tested it on the other machine and got these results:
On MY machine
# python
import hashlib
from PIL import Image
md = hashlib.md5()
md.update(Image.open('000001.jpg').tobytes())
print(md.hexdigest())
# 92a0d5f96769de4e68a0ce40e7f0c13f
# It is incorrect
On the other machine
# python
import hashlib
from PIL import Image
md = hashlib.md5()
md.update(Image.open('000001.jpg').tobytes())
print(md.hexdigest())
# 9cad8178d6cb0196b36f7b34bc5eb6d3
# It is correct.
However, when I tried to use md5sum to check these two images on different machines, I got the same code. That means files are exactly the same. So I think the problem might be the version of libjpeg.
On MY machine
ldd /home/liyh/anaconda2/envs/theano/lib/python2.7/site-packages/PIL/_imaging.so
linux-vdso.so.1 => (0x00007ffe4e3f6000)
**libjpeg.so.9 => /home/liyh/anaconda2/envs/theano/lib/libjpeg.so.9 (0x00007fbeb418a000)**
libz.so.1 => /home/liyh/anaconda2/envs/theano/lib/libz.so.1 (0x00007fbeb3f74000)
libtiff.so.5 => /home/liyh/anaconda2/envs/theano/lib/libtiff.so.5 (0x00007fbeb3cf6000)
libpython2.7.so.1.0 => /home/liyh/anaconda2/envs/theano/lib/libpython2.7.so.1.0 (0x00007fbeb38fa000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbeb36dc000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbeb3313000)
liblzma.so.5 => /home/liyh/anaconda2/envs/theano/lib/./liblzma.so.5 (0x00007fbeb30ee000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbeb2ee6000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbeb2be0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbeb29dc000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fbeb27d9000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbeb4631000)
On the other machine
$ldd /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/_imaging.so
linux-vdso.so.1 => (0x00007ffff4791000)
**libjpeg.so.8 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libjpeg.so.8 (0x00007f217847f000)**
libz.so.1 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libz.so.1 (0x00007f2178268000)
libtiff.so.5 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libtiff.so.5 (0x00007f2177fea000)
libpython2.7.so.1.0 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../../libpython2.7.so.1.0 (0x00007f2177bf0000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f21779b5000)
libc.so.6 => /lib64/libc.so.6 (0x00007f21775f4000)
liblzma.so.5 => /home/liyh/anaconda2/lib/python2.7/site-packages/PIL/../../.././liblzma.so.5 (0x00007f21773ce000)
librt.so.1 => /lib64/librt.so.1 (0x00007f21771c6000)
libm.so.6 => /lib64/libm.so.6 (0x00007f2176ec4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f2176cc0000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f2176abc000)
/lib64/ld-linux-x86-64.so.2 (0x00007f217890e000)
I tried to re-install Pillow==3.1.1, but it kept using libjpeg9 instead of libjpeg8.
If I delete the /home/liyh/anaconda2/envs/theano/lib/libjpeg.so.9 and add libjpeg.so.8 into the directory and re-install Pillow==3.1.1, I got
ImportError: libjpeg.so.9: cannot open shared object file: No such file or directory
My question is:
How can I install Pillow3.1.1 with libjpeg8, or
How can I generate the correct MD5 code using libjpeg9, or
What other possible reasons that cause the different MD5 code?
Ubuntu 4.8.4-2ubuntu1~14.04.3
Python 2.7
Edited:
The image I used is the first image (000001.jpg) in CelebA dataset ("in-the-wild" version, not the "align&cropped").
$md5sum /data2/pub_data/CelebA/Img/img_celeba/000001.jpg
39d4f4f149cf541601ad334ea98c822d

I happened to solve this problem by installing pillow==3.1.1 with conda instead of pip. In this way, pillow uses libjpeg8 as its jpeg decoder as we want.
conda create -n new_env
source activate new_env
conda install pillow==3.1.1
conda install ...(other needed libs)

Related

Is there any way to find available CPAN version programmatically?

I am trying to find CPAN version programmatically, but ends up in failure.
Tried with perl %Config & %ENV, but couldn't find anything there.
my $cpanv = `cpan -v`;
# gives: Loading internal null logger. Install Log::Log4perl for logging messages
My goal is to auto configure cpan programmatically. CPAN config prompt varies by versions.
CPAN asks like[similar]:
1. "Do you want to manually configure?[yes]"
2. "Do you want cpan to configure?[no]"
Any way to find CPAN version or avoiding unattended CPAN configuration?
The cpan command-line program is a wrapper around App::Cpan. As far as I can see, the two files always have the same version number. That means you can do something like this:
use App::Cpan;
my $cpanv = $App::Cpan::VERSION;
Update: There are a few concepts to get straight here.
CPAN.pm is a CPAN exploration module that was added to Perl version 5.004. It includes its own basic shell.
App::Cpan is a wrapper around CPAN.pm which adds an improved command interface. It was added to Perl 5.12.
cpan is a tiny script that uses App::Cpan. It was also added to Perl version 5.12. [Update: that's the current version of cpan - there was also an earlier one that was based on CPAN.pm.]
App::Cpan and cpan share the same version number (currently 1.675). CPAN.pm is older and therefore has a larger version number (currently 2.27).
The version number from App::CPAN is what you get when you run cpan -v - so that's what I've given you in my code.
My goal is to auto configure cpan programmatically.
If you're talking about systems with a known configuration, just create (or modify) ~/.cpan/CPAN/MyConfig.pm. For example, the following is mine:
$CPAN::Config = {
'auto_commit' => q[0],
'build_cache' => q[5],
'build_dir' => q[/home/ikegami/.cpan/build],
'build_requires_install_policy' => q[yes],
'cache_metadata' => q[1],
'check_sigs' => q[0],
'commandnumber_in_prompt' => q[1],
'connect_to_internet_ok' => q[1],
'cpan_home' => q[/home/ikegami/.cpan],
'dontload_hash' => { },
'ftp' => q[/usr/bin/ftp],
'ftp_passive' => q[1],
'ftp_proxy' => q[],
'getcwd' => q[cwd],
'gpg' => q[/usr/bin/gpg],
'gzip' => q[/bin/gzip],
'halt_on_failure' => q[0],
'histfile' => q[],
'http_proxy' => q[],
'inactivity_timeout' => q[0],
'index_expire' => q[1],
'inhibit_startup_message' => q[0],
'keep_source_where' => q[/home/ikegami/.cpan/sources],
'load_module_verbosity' => q[none],
'lynx' => q[],
'make' => q[/usr/bin/make],
'make_arg' => q[],
'make_install_arg' => q[],
'make_install_make_command' => q[/usr/bin/make],
'makepl_arg' => q[],
'mbuild_arg' => q[],
'mbuild_install_arg' => q[],
'mbuild_install_build_command' => q[./Build],
'mbuildpl_arg' => q[],
'ncftpget' => q[/usr/bin/ncftpget],
'no_proxy' => q[],
'pager' => q[less],
'perl5lib_verbosity' => q[none],
'prefer_external_tar' => q[1],
'prefer_installer' => q[MB],
'prerequisites_policy' => q[follow],
'scan_cache' => q[atstart],
'shell' => q[/bin/bash],
'show_upload_date' => q[0],
'tar' => q[/bin/tar],
'tar_verbosity' => q[none],
'term_is_latin' => q[1],
'term_ornaments' => q[1],
'trust_test_report_history' => q[0],
'unzip' => q[/usr/bin/unzip],
'urllist' => [q[http://ftp.osuosl.org/pub/CPAN/], q[http://mirrors.hub.co/CPAN/], q[http://cpan.arcticnetwork.ca/]],
'use_sqlite' => q[0],
'version_timeout' => q[15],
'wget' => q[/usr/bin/wget],
'yaml_load_code' => q[0],
};
1;
__END__
Optionally, you can force the defaults to be used, and modify what needs to be changed. For example,
cpan <<<'o conf init
yes
o conf prerequisites_policy ask
o conf commit
quit
'
o conf init causes the initalization process to be started. yes accepts the defaults. o conf prerequisites_policy ask is an example of changing a default. o conf commit saves the changes.

Mongodb doesn't load crypto | ssl libraries during startup

I am trying to use the latest mongodb binary bits, but for some reason it doesn't load the crypto | ssl libraries.
ldd bin/mongod
bin/mongod: /lib/x86_64-linux-gnu/libcrypto.so.10: version `libcrypto.so.10' not found (required by bin/mongod)
bin/mongod: /lib/x86_64-linux-gnu/libssl.so.10: version `libssl.so.10' not found (required by bin/mongod)
linux-vdso.so.1 => (0x00007fff186f4000)
libssl.so.10 => /lib/x86_64-linux-gnu/libssl.so.10 (0x00007f08a48c1000)
libcrypto.so.10 => /lib/x86_64-linux-gnu/libcrypto.so.10 (0x00007f08a44e6000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f08a42dd000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f08a40d9000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f08a3dd5000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f08a3ace000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f08a38b8000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f08a369a000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f08a32d4000)
/lib64/ld-linux-x86-64.so.2 (0x00007f08a4b3d000)
This error message is typically received when the enterprise dependencies have not been installed prior to startup of mongo.
This is documented here - https://docs.mongodb.org/manual/tutorial/install-mongodb-enterprise-on-red-hat/#install-mongodb-enterprise-from-tarball

libquadmath error on CentOS 6: not found by software

A piece of software I compiled on a CentOS 6 machine is giving me an error that I don't see with the exact same software compiled on my Ubuntu 14.10 laptop:
auxgen: error while loading shared libraries: libquadmath.so.0: cannot open shared object file: No such file or directory
The library in question does seem to be there on the CentOS machine:
$ locate libquadmath
/usr/local/lib/libquadmath.a
/usr/local/lib/libquadmath.la
/usr/local/lib/libquadmath.so
/usr/local/lib/libquadmath.so.0
/usr/local/lib/libquadmath.so.0.0.0
/usr/local/lib64/libquadmath.a
/usr/local/lib64/libquadmath.la
/usr/local/lib64/libquadmath.so
/usr/local/lib64/libquadmath.so.0
/usr/local/lib64/libquadmath.so.0.0.0
/usr/local/share/info/libquadmath.info
$ ldd /usr/local/bin/auxgen
/usr/local/bin/auxgen: /usr/lib64/libgfortran.so.3: version `GFORTRAN_1.4' not found (required by /usr/local/bin/auxgen)
linux-vdso.so.1 => (0x00007fff6cdb0000)
libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x0000003b64a00000)
libm.so.6 => /lib64/libm.so.6 (0x000000330d000000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003c0b400000)
libquadmath.so.0 => not found
libc.so.6 => /lib64/libc.so.6 (0x000000330c400000)
/lib64/ld-linux-x86-64.so.2 (0x000000330c000000)
On my Ubuntu 14.10 laptop, where it works:
$ ldd /usr/local/bin/auxgen
linux-vdso.so.1 => (0x00007fff7aaf2000)
libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007fd71b5da000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd71b215000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fd71afd6000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd71acd0000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd71aab9000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd71b922000)
On the Ubuntu laptop where it does work:
$ ldconfig -p | grep 'quad\|gfort'
libquadmath.so.0 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libquadmath.so.0
libgfortran.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libgfortran.so.3
On the CentOS where it doesn't work:
$ ldconfig -p | grep 'quad\|gfort'
libgfortran.so.3 (libc6,x86-64) => /usr/lib64/libgfortran.so.3
libgfortran.so.3 (libc6) => /usr/lib/libgfortran.so.3
Any ideas?

Does MongoDB require the gcc version 4.0.0 or higher?

I am using MongoDB Solaris package. When I use ldd command to check the mongod, the output is:
bash-3.2# ldd mongod
libsocket.so.1 => /lib/64/libsocket.so.1
libresolv.so.2 => /lib/64/libresolv.so.2
librt.so.1 => /lib/64/librt.so.1
libnsl.so.1 => /lib/64/libnsl.so.1
libstdc++.so.6 => /usr/sfw/lib/amd64/libstdc++.so.6
libm.so.2 => /lib/64/libm.so.2
libgcc_s.so.1 => /usr/sfw/lib/amd64/libgcc_s.so.1
libgcc_s.so.1 (GCC_4.0.0) => (version not found)
libpthread.so.1 => /lib/64/libpthread.so.1
libc.so.1 => /lib/64/libc.so.1
libaio.so.1 => /lib/64/libaio.so.1
libmd.so.1 => /lib/64/libmd.so.1
libmp.so.2 => /lib/64/libmp.so.2
libscf.so.1 => /lib/64/libscf.so.1
libdoor.so.1 => /lib/64/libdoor.so.1
libuutil.so.1 => /lib/64/libuutil.so.1
libgen.so.1 => /lib/64/libgen.so.1
From the output, I notice the following lines:
libgcc_s.so.1 => /usr/sfw/lib/amd64/libgcc_s.so.1
libgcc_s.so.1 (GCC_4.0.0) => (version not found)
Per my understanding, I think it means the version of libgcc_s.so.1 must be 4.0.0 or higher. Does MongoDB require the gcc version be 4.0.0 or higher? I can't find this requirement in its website.
Yes.
From the source code on Github, we can see:
#if __GNUC__ < 4
#error gcc < 4 not supported
#endif
https://github.com/mongodb/mongo/blob/master/src/mongo/pch.cpp
I can't find this in the official docs though, but the docs do state that you need GNU C++ Toolchain, which I would imagine implies whatever is the most current release.

PostgreSQL 8.4 Fails to Start on Ubuntu 10.10 Bug?

PostgreSQL 8.4 Fails to Start on Ubuntu 10.10
sudo /etc/init.d/postgresql start
* Starting PostgreSQL 8.4 database server
* Error: could not exec /usr/lib/postgresql/8.4/bin/pg_ctl /usr/lib/postgresql/8.4/bin/pg_ctl start -D /var/lib/postgresql/8.4/main -l /var/log/postgresql/postgresql-8.4-main.log -s -o -c config_file="/etc/postgresql/8.4/main/postgresql.conf" :
running ldd on /usr/lib/postgresql/8.4/bin/pg_ctl
ldd /usr/lib/postgresql/8.4/bin/pg_ctl
linux-gate.so.1 => (0xb77e4000)
libpq.so.5 => /usr/lib/libpq.so.5 (0xb77a2000)
libc.so.6 => /lib/libc.so.6 (0xb7645000)
libssl.so.0.9.8 => /lib/libssl.so.0.9.8 (0xb75fb000)
libcrypto.so.0.9.8 => /lib/libcrypto.so.0.9.8 (0xb74ac000)
libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0xb73fd000)
libcom_err.so.2 => /lib/libcom_err.so.2 (0xb73f9000)
libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0xb73ca000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0xb7398000)
libldap_r-2.4.so.2 => /usr/lib/libldap_r-2.4.so.2 (0xb7351000)
libpthread.so.0 => /lib/libpthread.so.0 (0xb7337000)
/lib/ld-linux.so.2 (0xb77e5000)
libdl.so.2 => /lib/libdl.so.2 (0xb7333000)
libz.so.1 => /lib/libz.so.1 (0xb731e000)
libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0xb72fa000)
libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0xb72f1000)
libkeyutils.so.1 => /lib/libkeyutils.so.1 (0xb72ed000)
libresolv.so.2 => /lib/libresolv.so.2 (0xb72d9000)
liblber-2.4.so.2 => /usr/lib/liblber-2.4.so.2 (0xb72cc000)
libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb72b4000)
libgnutls.so.26 => /usr/lib/libgnutls.so.26 (0xb7218000)
libtasn1.so.3 => /usr/lib/libtasn1.so.3 (0xb7207000)
libgcrypt.so.11 => /lib/libgcrypt.so.11 (0xb7193000)
libgpg-error.so.0 => /lib/libgpg-error.so.0 (0xb718e000)
I have checked permissions and have given root appropriate permissions, and I am trying to run postgresql as root.
PostgreSQL refuses to run as root. This is so that bugs in the back-end, plus bugs in functions written in untrusted languages, cannot be used to do arbitrary actions over the whole system. PostgreSQL must run as a relatively limited user, usually the postgres user and the start-up scripts normally enforce this.
Grant permissions to the postgres user rather than root. Then everything should work fine.