Why does gcc not allow undefined symbols in shared object on MSYS2? - perl

I am trying to build Math::GSL on MSYS2 and got some problems with creating shared objects. Here is a simplified version of my problem, consider this C program sample.c:
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
static void SWIG_croak_null()
{
SV *err = get_sv("#", GV_ADD);
croak("%s", SvPV_nolen(err));
}
On Ubuntu I can create a shared object sample.so like this using a run.sh shell script:
#! /bin/bash
perl_dir=$(perl -MConfig -e'print $Config{archlib}')"/CORE"
set -x
gcc -I"$perl_dir" -c -fPIC -g -o sample.o sample.c
gcc -o sample.so sample.o -shared -fPIC
The output is:
+ gcc -I/home/hakon/perlbrew/perls/perl-5.30.0/lib/5.30.0/x86_64-linux/CORE -c -fPIC -g -o sample.o sample.c
+ gcc -o sample.so sample.o -shared -fPIC
But if I run the same program on Windows 10 with MSYS2, perl version 5.32.0, I get this output:
+ gcc -I/usr/lib/perl5/core_perl/CORE -c -fPIC -g -o sample.o sample.c
+ gcc -o sample.so sample.o -shared -fPIC
/usr/lib/gcc/x86_64-pc-msys/9.3.0/../../../../x86_64-pc-msys/bin/ld: sample.o: in function `SWIG_croak_null':
/home/hakon/perl/test/perl_get_sv/sample.c:7: undefined reference to `Perl_get_sv'
/home/hakon/perl/test/perl_get_sv/sample.c:7:(.text+0x2b): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `Perl_get_sv'
/usr/lib/gcc/x86_64-pc-msys/9.3.0/../../../../x86_64-pc-msys/bin/ld: /home/hakon/perl/test/perl_get_sv/sample.c:8: undefined reference to `Perl_sv_2pv_flags'
/home/hakon/perl/test/perl_get_sv/sample.c:8:(.text+0x79): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `Perl_sv_2pv_flags'
/usr/lib/gcc/x86_64-pc-msys/9.3.0/../../../../x86_64-pc-msys/bin/ld: /home/hakon/perl/test/perl_get_sv/sample.c:8: undefined reference to `Perl_croak_nocontext'
/home/hakon/perl/test/perl_get_sv/sample.c:8:(.text+0x88): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `Perl_croak_nocontext'
/usr/lib/gcc/x86_64-pc-msys/9.3.0/../../../../x86_64-pc-msys/bin/ld: sample.o:sample.c:(.rdata$.refptr.PL_thr_key[.refptr.PL_thr_key]+0x0): undefined reference to `PL_thr_key'
collect2: error: ld returned 1 exit status
However, if I explicitly link with libperl.dll.a it works fine:
gcc -o sample.so sample.o -shared -L"$perl_dir" -lperl
Why is it not possible to create a shared object with undefined symbols in MSYS2?

According to this page:
Because of the explicit nature of the table of imported symbols, it is
not possible to leave a symbol in a PE DLL undefined at link time, to
be satisfied at runtime, as it is instead possible with most UNIX
shared objects.

Related

Unable to run C++ kafka-serdes-avro-console-producer

When I try to run the console producer getting following. Looks like some type of C++ linking issue need help to debug etc.
Error Detail
/libserdes/examples/kafka-serdes-avro-console-producer.cpp:331: undefined reference to RdKafka::Producer::create(RdKafka::Conf const*, std::string&)' /remote/vgrnd104/rachana/081222/libserdes/examples/kafka-serdes-avro-console-producer.cpp:363: undefined reference to RdKafka::err2str(RdKafka::ErrorCode)’
collect2: error: ld returned 1 exit status
make: *** [kafka-serdes-avro-console-producer] Error 1
Make Detail
/libserdes/examples % make kafka-serdes-avro-console-producer
g++ -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align -I…/src -Wno-non-virtual-dtor --std=c++11 -I…/src -I…/src-cpp -D_GLIBCXX_USE_CXX11_ABI=0 -I…/output/include …/output/lib/libserdes.a …/output/lib/libserdes++.a …/output/lib/librdkafka++.a kafka-serdes-avro-console-producer.cpp
-o kafka-serdes-avro-console-producer …/output/lib/libserdes++.a …/output/lib/librdkafka++.a …/output/lib/libserdes.a -L…/output/lib -lrt -lpthread -lcurl -ljansson -lavrocpp -lavro -lrdkafka++ -lrdkafka

Boost.Python Link Error with Python3 and virtualenv

I got the following error when I tried to compile .cpp file with clang++.
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I set up Python and boost-python as follows.
Python
I installed Python 3.6.0 using pyenv-virtualenv (via homebrew)
Boost-Python
I used homebrew in Python 3 environment.
$ source activate py36
$ brew install boost-python --with-python3 --without-python
~/.bash_profile
My ~/.bash_profile looks like this:
export PYENV_ROOT="/usr/local/var/pyenv"
export PYENV_VIRTUALENV_DISABLE_PROMPT=1
export CPLUS_INCLUDE_PATH="$CPLUS_INCLUDE_PATH:/usr/local/var/pyenv/versions/3.6.0/include/python3.6m/"
if which pyenv > /dev/null; then eval "$(pyenv init -)"; fi
if which pyenv-virtualenv-init > /dev/null; then eval "$(pyenv virtualenv-init -)" ; fi
Test cpp
This is the file I tried to compile and failed (saved as test.cpp).
#define BOOST_PYTHON_STATIC_LIB
#include <boost/python.hpp>
using namespace boost::python;
char const* say(){
return "hello, world";
}
BOOST_PYTHON_MODULE( hello ){
boost::python::def( "say", say );
}
makefile
CC = clang++
INCLUDE = -I`python -c 'from distutils.sysconfig import *; print(get_python_inc())’`
BOOST = -lboost_python3.6 -lboost_serialization
FMATH = -fomit-frame-pointer -fno-operator-names -msse2 -mfpmath=sse -march=native
CFLAGS = -std=c++11 -L/usr/local/lib -O3
CFLAGS_SO = -shared -fPIC -std=c++11 -L/usr/local/lib -O3
LINK = -L/usr/local/Cellar/boost-python/1.64.0/lib/
install:
$(CC) test.cpp -o hello.so $(INCLUDE) $(BOOST) $(LINK) $(FMATH) $(CFLAGS_SO)
Error
make install returns this error.
clang++ test.cpp -o hello.so -I`python -c 'from distutils.sysconfig import *; print(get_python_inc())’` -lboost_python3.6 -lboost_serialization -L/usr/local/Cellar/boost-python/1.64.0/lib/ -fomit-frame-pointer -fno-operator-names -msse2 -mfpmath=sse -march=native -shared -fPIC -std=c++11 -L/usr/local/lib -O3 -v
/bin/sh: command substitution: line 0: unexpected EOF while looking for matching `''
/bin/sh: command substitution: line 1: syntax error: unexpected end of file
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
clang -cc1 version 8.1.0 (clang-802.0.42) default target x86_64-apple-darwin16.6.0
ignoring nonexistent directory "-lboost_python3.6"
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
.
/usr/local/var/pyenv/versions/3.6.0/include/python3.6m
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1
/usr/local/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.1.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/usr/include
/System/Library/Frameworks (framework directory)
/Library/Frameworks (framework directory)
End of search list.
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -dylib -arch x86_64 -macosx_version_min 10.12.0 -o hello.so -L/usr/local/Cellar/boost-python/1.64.0/lib/ -L/usr/local/lib /var/folders/rm/qv81g4ss29j_gv1b366rl2080000gp/T/test-16790a.o -lboost_serialization -lc++ -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.1.0/lib/darwin/libclang_rt.osx.a
Undefined symbols for architecture x86_64:
<LINES ARE OMITTED>
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [install] Error 1

How to recompile libperl.a object files with -fPIC flag?

This question arose when trying to fix some installation problems
with QtCore4. At some point make tried to run the following command:
/usr/bin/c++ -fPIC -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong \
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG \
-shared -Wl,-soname,QtCore4.so -o ../../blib/arch/auto/QtCore4/QtCore4.so \
CMakeFiles/perlqtcore4.dir/binding.cpp.o \
CMakeFiles/perlqtcore4.dir/handlers.cpp.o \
CMakeFiles/perlqtcore4.dir/marshall_types.cpp.o \
CMakeFiles/perlqtcore4.dir/util.cpp.o \
CMakeFiles/perlqtcore4.dir/QtCore4.c.o \
-lQtCore -lQtGui -lQtNetwork \
/home/hakon/perlbrew/perls/perl-5.24.1/lib/5.24.1/x86_64-linux/CORE/libperl.a \
../../smokeqt/qtgui/libsmokeqtgui.so.3.0.0 \
../../smokeqt/qtnetwork/libsmokeqtnetwork.so.3.0.0 \
-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc -lQtGui -lQtNetwork \
../../smokeqt/qtcore/libsmokeqtcore.so.3.0.0 -lQtCore \
../../smokegen/bin/libsmokebase.so.3.0.0 \
-Wl,-rpath,/home/hakon/Qt4-0.99.0/smokeqt/qtgui:/home/hakon/Qt4-0.99.0/smokeqt/qtnetwork:/home/hakon/Qt4-0.99.0/smokeqt/qtcore:/home/hakon/Qt4-0.99.0/smokegen/bin:
which failed with the following error message from the linker:
/usr/bin/ld: /home/hakon/perlbrew/perls/perl-5.24.1/lib/5.24.1/x86_64-linux/CORE/libperl.a(toke.o):
relocation R_X86_64_PC32 against symbol `PL_curcop' can not be used when making
a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
From the error message, it seems to me that the object files (here: toke.o) in
libperl.a was compiled without the -fPIC flag set.
The questions I have now are:
How can I recompile the object files in libperl.a with -fPIC ?
Could this recompilation cause other problems not related to
QtCore4 (since it is likely that libperl.a will be used (linked
with) by other applications/programs not related to QtCore4)?
I am using Ubuntu 16.10 and perl version 5.24.1.
How can I recompile the object files in libperl.a with -fPIC
By recompiling libperl.a with the -fPIC flag.
./Configure -des -Accflags=-fPIC ...
Could this recompilation cause other problems not related to QtCore4 (since it is likely that libperl.a will be used (linked with) by other applications/programs not related to QtCore4)?
Yes. Prefer a shared libperl. -Duseshrplib
Which is required when embedding perl into shared libs. You can do away with the -fPIC trick which is required on ELF, but a shared libperl makes it much easier.

Unable to compile / install Orange Data Mining on Raspberry Pi

I have been trying to compile and install Orange 2.7.8 on Raspberry Pi 2 but was unsuccessful. I tried compiling by hand and also used pip install orange as per suggestion from: How can I install python-Orange on ubuntu 12.10
The first warning I got this:
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -I/usr/lib/pymodules/python2.7/numpy/core/include -Isource/include -Isource/orange/liblinear -Isource/orange/ppp -Isource/orange/px -I/usr/include/python2.7 -c source/orange/basstat.cpp -o build/temp.linux-armv7l-2.7/source/orange/basstat.o -fPIC -w -DLINUX -DORANGE_EXPORTS
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for Ada/C/ObjC but not for C++ [enabled by default]
But it kept going without an error. Then when it started compiling with g++, it started getting an error on multiple definition of TOrangeVector. There are many errors like the lines below:
build/temp.linux-armv7l-2.7/source/orange/distance.o:(.data.rel.ro+0x6c): multiple definition of `typeinfo for TOrangeVector<GCPtr<TVariable>, true>'
build/temp.linux-armv7l-2.7/source/orange/basstat.o:(.data.rel.ro+0x6c): first defined here
build/temp.linux-armv7l-2.7/source/orange/distance.o: In function `TExamplesDistance::classDescription() const':
/home/pi/build/orange/source/orange/ppp/distance.ppp:17: multiple definition of `typeinfo for TOrangeVector<bool, false>'
In the end it got kicked out:
/home/pi/build/orange/source/orange/vectortemplates.hpp:1075: multiple definition of `typeinfo name for TOrangeVector<GCPtr<TOrangeVector<GCPtr<TVariable>, true> >, true>'
build/temp.linux-armv7l-2.7/source/orange/basstat.o:/home/pi/build/orange/source/orange/ppp/../basstat.hpp:8: first defined here
collect2: ld returned 1 exit status
error: command 'g++' failed with exit status 1
Any guide on how to tweak this to make it compiled would be appreciated.

Mex compiling on 64 bit linux - /usr/bin/ld: cannot find -lstdc++

Okay I am trying to compile a mex file on 64 bit linux, ubuntu to be precise with Matlab 2013a.
First it gave a error that it could not find GLIBCXX_3.4.15. Which was not part of the
/usr/local/MATLAB/R2013a/sys/os/glnxa64/libstdc++.so.6. I found this thread /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.15' not found and succesfully created a symbolic link ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17 libstdc++.so.6 in /usr/local/MATLAB/R2013a/sys/os/glnxa64
Now I tried to compile again and now I get /usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
This is the output
>> mex -v tload3.c
**************************************************************************
Warning: Neither -compatibleArrayDims nor -largeArrayDims is selected.
Using -compatibleArrayDims. In the future, MATLAB will require
the use of -largeArrayDims and remove the -compatibleArrayDims
option. For more information, see:
http://www.mathworks.com/help/matlab/matlab_external/upgrading-mex-files-to-use-64-bit-api.html
**************************************************************************
-> mexopts.sh sourced from directory (DIR = $PREF_DIR)
FILE = /home/wouter/.matlab/R2013a/mexopts.sh
----------------------------------------------------------------
-> MATLAB = /usr/local/MATLAB/R2013a
-> CC = gcc
-> CC flags:
CFLAGS = -ansi -D_GNU_SOURCE -fexceptions -fPIC -fno-omit-frame-pointer -pthread
CDEBUGFLAGS = -g
COPTIMFLAGS = -O -DNDEBUG
CLIBS = -Wl,-rpath-link,/usr/local/MATLAB/R2013a/bin/glnxa64 -L/usr/local/MATLAB/R2013a/bin/glnxa64 -lmx -lmex -lmat -lm -lstdc++
arguments = -DMX_COMPAT_32
-> CXX = g++
-> CXX flags:
CXXFLAGS = -ansi -D_GNU_SOURCE -fPIC -fno-omit-frame-pointer -pthread
CXXDEBUGFLAGS = -g
CXXOPTIMFLAGS = -O -DNDEBUG
CXXLIBS = -Wl,-rpath-link,/usr/local/MATLAB/R2013a/bin/glnxa64 -L/usr/local/MATLAB/R2013a/bin/glnxa64 -lmx -lmex -lmat -lm
arguments = -DMX_COMPAT_32
-> FC = gfortran
-> FC flags:
FFLAGS = -fexceptions -fbackslash -fPIC -fno-omit-frame-pointer
FDEBUGFLAGS = -g
FOPTIMFLAGS = -O
FLIBS = -Wl,-rpath-link,/usr/local/MATLAB/R2013a/bin/glnxa64 -L/usr/local/MATLAB/R2013a/bin/glnxa64 -lmx -lmex -lmat -lm
arguments = -DMX_COMPAT_32
-> LD = gcc
-> Link flags:
LDFLAGS = -pthread -shared -Wl,--version-script,/usr/local/MATLAB/R2013a/extern/lib/glnxa64/mexFunction.map -Wl,--no-undefined
LDDEBUGFLAGS = -g
LDOPTIMFLAGS = -O
LDEXTENSION = .mexa64
arguments =
-> LDCXX =
-> Link flags:
LDCXXFLAGS =
LDCXXDEBUGFLAGS =
LDCXXOPTIMFLAGS =
LDCXXEXTENSION =
arguments =
----------------------------------------------------------------
Warning: You are using gcc version "4.7.3". The version
currently supported with MEX is "4.4.x".
For a list of currently supported compilers see:
http://www.mathworks.com/support/compilers/current_release/
-> gcc -c -I/usr/local/MATLAB/R2013a/extern/include -I/usr/local/MATLAB/R2013a/simulink/include -DMATLAB_MEX_FILE -ansi -D_GNU_SOURCE -fexceptions -fPIC -fno-omit-frame-pointer -pthread -DMX_COMPAT_32 -O -DNDEBUG "tload3.c"
-> gcc -O -pthread -shared -Wl,--version-script,/usr/local/MATLAB/R2013a/extern/lib/glnxa64/mexFunction.map -Wl,--no-undefined -o "tload3.mexa64" tload3.o -Wl,-rpath-link,/usr/local/MATLAB/R2013a/bin/glnxa64 -L/usr/local/MATLAB/R2013a/bin/glnxa64 -lmx -lmex -lmat -lm -lstdc++
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
mex: link of ' "tload3.mexa64"' failed.
Error using mex (line 206)
Unable to complete successfully.
ps there is a another thread with about the same title, Error building MEX-files (MATLAB 2012a) on Kubuntu '/usr/bin/ld: cannot find -lstdc++', unfortunatly the answer, installing sudo apt-get install libstdc++6-4.4-dev did not work for me.
Same problem here on Ubuntu 13.04 using MATLAB 2013a.
I solved it doing:
sudo mv /usr/local/MATLAB/R2013a/sys/os/glnxa64/libstdc++.so.6 /usr/local/MATLAB/R2013a/sys/os/glnxa64/libstdc++.so.6.old
To keep a backup just in case.
And then a symbolic link:
sudo ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17 /usr/local/MATLAB/R2013a/sys/os/glnxa64/libstdc++.so.6
I had the same problem and this worked for me
Navigate to the following path cd /usr/local/MATLAB/R2013a/sys/os/glnxa64/, (your path might varies) then remove the libstdc library (or safer rename it)
sudo mv libstdc++.so.6 libstdc++.so.6.old
I have the same configuration, Ubuntu Linux 64 bit with Matlab 2013a
Look at this line gcc -O -pthread -shared -Wl,--version-script,/usr/local/MATLAB/R2013a/extern/lib/glnxa64/mexFunction.map -Wl,--no-undefined -o "tload3.mexa64" tload3.o -Wl,-rpath-link,/usr/local/MATLAB/R2013a/bin/glnxa64 -L/usr/local/MATLAB/R2013a/bin/glnxa64 -lmx -lmex -lmat -lm -lstdc++.
The options with L are the places where gcc/the linker looks for libraries. Id sugest trying to put the symlink in /usr/local/MATLAB/R2013a/bin/glnxa64 ( libstdc++.so.6 is there in 2012a), and start matlab from a terminal, it spits some errors there. Or add the new folder as a CLIB argument.
And I also would delete the -ansi part for c ( replace it with -std=c99), so that it won't give you errors on using \ style comments in your c code.
You created the link libstdc++.so.6 in /usr/local/MATLAB/R2013a/bin/glnxa64 to the library in your system, installed by apt-get install libstdc++6-4.4-dev .
Now try creating also a link named libstdc++.so to libstdc++.so.6 in
/usr/local/MATLAB/R2013a/bin/glnxa64.
I had the same problem and none of the posted solutions worked for me (i.e. deleting or adding new symlinks), and it turned out that I hadn't installed the appropriate compiler package!
For me, the solution was:
sudo yum install gcc-g++
I'm running MATLAB 2014b on 64-bit Linux (RHEL 6).