I'm trying to build Scipy from source. However, a linking step...
/usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwg/miniconda3/envs/scipy-dev/lib -arch x86_64 -L/Users/gwg/miniconda3/envs/scipy-dev/lib -arch x86_64 -L/usr/local/opt/libomp/lib -lomp -I/usr/local/opt/libomp/include -Xpreprocessor -fopenmp -arch x86_64 build/temp.macosx-10.9-x86_64-3.7/scipy/spatial/src/distance_wrap.o -L/Users/gwg/miniconda3/envs/scipy-dev/lib/python3.7/site-packages/numpy/core/lib -Lbuild/temp.macosx-10.9-x86_64-3.7 -lnpymath -o build/lib.macosx-10.9-x86_64-3.7/scipy/spatial/_distance_wrap.cpython-37m-darwin.so
...fails, and I get the following error:
ld: in /Users/gwg/miniconda3/envs/scipy-dev/lib/python3.7/site-packages/numpy/core/lib/libnpymath.a(npy_math.o), could not parse object file /Users/gwg/miniconda3/envs/scipy-dev/lib/python3.7/site-packages/numpy/core/lib/libnpymath.a(npy_math.o): 'Unknown attribute kind (61) (Producer: 'LLVM10.0.0' Reader: 'LLVM APPLE_1_1000.10.44.4_0')', using libLTO version 'LLVM version 10.0.0, (clang-1000.10.44.4)' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I suspect it is due to the Producer: 'LLVM10.0.0' Reader: 'LLVM APPLE_1_1000.10.44.4_0' mismatch. But I don't know how to investigate this error.
Some details:
MacOS: High Sierra, 10.13.6
gcc --version:
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
clang --version:
Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
I had the same issue. The suggestions here to use the conda compilers resolved the issue for me:
conda create -n scipy-dev
conda activate scipy-dev
conda config --env --add channels conda-forge
conda install numpy cython pybind11 pytest compilers openblas
python setup.py develop
It seems you're using different clang (judging from the paths like /usr/local/opt/libomp/lib). Likely the one provided by conda. In order to perform build with LTO you need to use the compatible toolchain (e.g. build everything with Apple-provided toolchain or with conda-provided one).
Related
From the latest CS:GO update, a valve dev commented this on the reddit post:
We are upgrading the compiler toolchain we use to build CS:GO. This
pre-release branch has CS:GO built with the new toolchain. We are
releasing it as a pre-release branch to allow for testing ahead of it
being used in the next update.
There are no functional changes expected, though the new compiler
toolchain may yield a small performance boost.
Can anyone explain what a compiler toolchain is?
A tool chain is a set of tools (chain of tools) that are used to create a programmed product (runnable/executable program etc.).
Usually toolchains are used in the embedded world for cross-compiling, which means creating a program on a host which will eventually run on a different kind of target - therefore there is a need to create it with a specific compiler, linker, debugger etc.
In order to create from scratch a compiler for the target, you need to cross-compile it with a specific toolchain - that is the compiler toolchain.
So, by using the compiler toolchain you will be able to create a compiler for your target system.
A compiler toolchain is a set of tools, supporting libraries and header files that help build a program from source to an executable that can run on a machine. To see what different tools are invoked during a compilation process pass '-v' flag. e.g., gcc -v hello.c will print the following (Note comments # are mine):
$ /usr/local/Cellar/gcc/11.2.0/bin/gcc-11 -v hello.c
# Configuration options which were used to build gcc itself.
Using built-in specs.
COLLECT_GCC=/usr/local/Cellar/gcc/11.2.0/bin/gcc-11
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc/11.2.0/libexec/gcc/x86_64-apple-darwin20/11.2.0/lto-wrapper
Target: x86_64-apple-darwin20
Configured with: ../configure --prefix=/usr/local/Cellar/gcc/11.2.0 --libdir=/usr/local/Cellar/gcc/11.2.0/lib/gcc/11 --disable-nls --enable-checking=release --enable-languages=c,c++,objc,obj-c++,fortran,d --program-suffix=-11 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-isl=/usr/local/opt/isl --with-zstd=/usr/local/opt/zstd --with-pkgversion='Homebrew GCC 11.2.0' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --enable-libphobos --build=x86_64-apple-darwin20 --with-system-zlib --disable-multilib --without-build-config --with-native-system-header-dir=/usr/include --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.2.0 (Homebrew GCC 11.2.0)
COLLECT_GCC_OPTIONS='-v' '-mmacosx-version-min=11.5.0' '-asm_macosx_version_min=11.5' '-mtune=core2' '-dumpdir' 'a-'
# Compiler invocation of cc1
/usr/local/Cellar/gcc/11.2.0/libexec/gcc/x86_64-apple-darwin20/11.2.0/cc1 -quiet -v -D__DYNAMIC__ hello.c -fPIC -quiet -dumpdir a- -dumpbase hello.c -dumpbase-ext .c -mmacosx-version-min=11.5.0 -mtune=core2 -version -o /var/folders/1n/lll3n8mj6wj975zjw804hg300000gq/T//ccfvEkdl.s
GNU C17 (Homebrew GCC 11.2.0) version 11.2.0 (x86_64-apple-darwin20)
compiled by GNU C version 11.2.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.24-GMP
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0/../../../../../../x86_64-apple-darwin20/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0/include
/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0/include-fixed
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
End of search list.
GNU C17 (Homebrew GCC 11.2.0) version 11.2.0 (x86_64-apple-darwin20)
compiled by GNU C version 11.2.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.24-GMP
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: d0f2649375ab2f2193374e77e0a72e7c
COLLECT_GCC_OPTIONS='-v' '-mmacosx-version-min=11.5.0' '-mtune=core2' '-dumpdir' 'a-'
# Assembler invocation
as -arch x86_64 -v -force_cpusubtype_ALL -mmacosx-version-min=11.5 -o /var/folders/1n/lll3n8mj6wj975zjw804hg300000gq/T//cc2nzxXo.o /var/folders/1n/lll3n8mj6wj975zjw804hg300000gq/T//ccfvEkdl.s
Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1as -triple x86_64-apple-macosx11.5.0 -filetype obj -main-file-name ccfvEkdl.s -target-cpu penryn -fdebug-compilation-dir /Users/Coolest -dwarf-debug-producer "Apple clang version 12.0.5 (clang-1205.0.22.9)" -dwarf-version=4 -mrelocation-model pic -mllvm -disable-aligned-alloc-awareness=1 -o /var/folders/1n/lll3n8mj6wj975zjw804hg300000gq/T//cc2nzxXo.o /var/folders/1n/lll3n8mj6wj975zjw804hg300000gq/T//ccfvEkdl.s
COMPILER_PATH=/usr/local/Cellar/gcc/11.2.0/libexec/gcc/x86_64-apple-darwin20/11.2.0/:/usr/local/Cellar/gcc/11.2.0/libexec/gcc/x86_64-apple-darwin20/11.2.0/:/usr/local/Cellar/gcc/11.2.0/libexec/gcc/x86_64-apple-darwin20/:/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0/:/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/
LIBRARY_PATH=/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0/:/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0/../../../
COLLECT_GCC_OPTIONS='-v' '-mmacosx-version-min=11.5.0' '-mtune=core2' '-dumpdir' 'a.'
/usr/local/Cellar/gcc/11.2.0/libexec/gcc/x86_64-apple-darwin20/11.2.0/collect2 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/ -dynamic -arch x86_64 -macosx_version_min 11.5.0 -weak_reference_mismatches non-weak -o a.out -L/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0 -L/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0/../../.. /var/folders/1n/lll3n8mj6wj975zjw804hg300000gq/T//cc2nzxXo.o -lSystem -lgcc_ext.10.5 -lgcc -lSystem -no_compact_unwind -v
collect2 version 11.2.0
# linker invocation
/usr/bin/ld -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/ -dynamic -arch x86_64 -macosx_version_min 11.5.0 -weak_reference_mismatches non-weak -o a.out -L/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0 -L/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0/../../.. /var/folders/1n/lll3n8mj6wj975zjw804hg300000gq/T//cc2nzxXo.o -lSystem -lgcc_ext.10.5 -lgcc -lSystem -no_compact_unwind -v
#(#)PROGRAM:ld PROJECT:ld64-650.9
BUILD 00:19:30 Mar 17 2021
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
/usr/local/Cellar/gcc/11.2.0/lib/gcc/11/gcc/x86_64-apple-darwin20/11.2.0
/usr/local/Cellar/gcc/11.2.0/lib/gcc/11
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
Framework search paths:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/
Note that a compiler toolchain is necessary to build executables but it is not sufficient. What is missing from the toolchain to have ‘everything’ that is needed to build executable programs is the ‘sysroot’.
Normally when we compile a program for the same machine the compiler uses the standard headers available in ‘/usr/include‘ and libraries from ‘/usr/lib‘. These paths are hardcoded in the compiler itself so we never have to think about it. However, when building a custom compiler or when cross-compiling programs we have to tell the compiler where the sysroot is by passing a flag e.g.
gcc --sysroot="/path/to/arm64/sysroot/usr" hello.c
Most often pre-packaged cross compilers come with a script/binary that has a ‘sysroot’ path embedded into it. e.g., aarch64-linux-gnu-gcc (https://packages.ubuntu.com/xenial/devel/gcc-aarch64-linux-gnu).
First of all, I know there are similar questions but I'm having no luck, so decided to ask a new one myself.
I'm trying to brew install the 64-bit version pjsip, but I always end up with the i386 version.
I've seen people whose brew --env yields CFLAGS and CXXFLAGS, but for some reason, mine doesn't, so I have no clue where to set the correct architecture.
$ brew --env
HOMEBREW_CC: clang
HOMEBREW_CXX: clang++
MAKEFLAGS: -j2
CMAKE_PREFIX_PATH: /usr/local
CMAKE_INCLUDE_PATH: /usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
CMAKE_LIBRARY_PATH: /System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
PKG_CONFIG_LIBDIR: /usr/lib/pkgconfig:/usr/local/Library/ENV/pkgconfig/10.9
ACLOCAL_PATH: /usr/local/share/aclocal
PATH: /usr/local/Library/ENV/4.3:/usr/bin:/bin:/usr/sbin:/sbin
$ brew --config
HOMEBREW_VERSION: 0.9.5
ORIGIN: https://github.com/mxcl/homebrew
HEAD: 8c19edbdd364200fb2cc9276b0bc49ec4cb98aae
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CELLAR: /usr/local/Cellar
CPU: dual-core 64-bit penryn
OS X: 10.9-x86_64
Xcode: 5.0.1
CLT: 5.0.1.0.1.1382131676
GCC-4.0: build 5494
Clang: 5.0 build 500
X11: 2.7.4 => /opt/X11
System Ruby: 1.8.7-358
Perl: /usr/bin/perl
Python: /Library/Frameworks/Python.framework/Versions/2.7/bin/python => /Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
Ruby: /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
I've seen people referring to the version of ruby and I'm not sure it matters, but mine is a Universal build:
$ file /usr/bin/ruby
/usr/bin/ruby: Mach-O universal binary with 2 architectures
/usr/bin/ruby (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/bin/ruby (for architecture i386): Mach-O executable i386
brew install -v pjsip shows:
==> ./configure --prefix=/usr/local/Cellar/pjsip/2.1
checking build system type... i386-apple-darwin13.0.0
checking host system type... i386-apple-darwin13.0.0
checking target system type... i386-apple-darwin13.0.0
Somehow it thinks that we're on a 64bit system, which is odd, as I'm definitely running a 64bit OS. Checking through the arch-host-OS detection code shows that is uses arch to determine the architecture. For some odd reason, this is reporting that we're a 32bit system:
$ arch
i386
While under linux:
$ arch
x86_64
The quick solution is to do a brew edit pjsip, and change line 22 to read:
system "./configure", "--prefix=#{prefix}", "--host=x86_64-apple-darwin13.0.0", "--target=x86_64-apple-darwin13.0.0", "--host=x86_64-apple-darwin13.0.0"
then perform a brew install pjsip, and it looks like you have 64bit pjsip.
Probably fixable with an edit of the recipe to get the proper arch, rather than just passing in the --build, --host and --target options.
I am trying to compile a matlab program using mex. I am facing the following error and was wondering if you have any suggestions. I have installed latest version of mpfr at /usr/local/opt/mpfr but it is still picking /usr/local/opt/mpfr2.
The error is as below:
dyld: Library not loaded: /usr/local/opt/mpfr2/lib/libmpfr.1.dylib
Referenced from: /usr/local/Cellar/gcc48/4.8.1/gcc/libexec/gcc/x86_64-apple-darwin12.5.0/4.8.1/cc1plus
Reason: Incompatible library version: cc1plus requires version 4.0.0 or later, but libmpfr.1.dylib provides version 3.0.0
g++-4.8: internal compiler error: Trace/BPT trap: 5 (program cc1plus)
/Applications/MATLAB_R2012b.app/bin/mex: line 1326: 15075 Abort trap: 6 /usr/local/bin/g++-4.8 -c -I/Applications/MATLAB_R2012b.app/extern/include -I/Applications/MATLAB_R2012b.app/simulink/include -DMATLAB_MEX_FILE -fno-common -fexceptions -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/ -mmacosx-version-min=10.7 -DMX_COMPAT_32 -O2 -DNDEBUG "face-detection/src/resizef.cc" -o face-detection/private/resizef.o
Try running this command before compiling:
setenv('DYLD_LIBRARY_PATH','')
I was trying to compile something else (not a Matlab program) and received the same error. The same command worked in bash, so I inspected the environment variables and found that Matlab's DYLD_LIBRARY_PATH differed from bash's DYLD_LIBRARY_PATH. The bash DYLD_LIBRARY_PATH was not defined, so I set it to empty in Matlab and it fixed the path error.
Matlab's setenv documentation:
Values assigned to variables using setenv are picked up by any
process that is spawned using the MATLAB SYSTEM, UNIX, DOS or '!'
functions. You can retrieve any value set with setenv by using
GETENV(NAME).
Try reinstalling gcc and its mpfr2 package, e.g.:
brew reinstall gcc48 mpfr2
Also make sure you're not overriding any DYLD_LIBRARY_PATH, DYLD_FALLBACK_LIBRARY_PATH or DYLD_FRAMEWORK_PATH variables from the startup shell scripts (e.g. ~/.bash_profile).
Recently I had to work with glibc to be installed on Solaris OS on the x86 platform.
The problem I encountered with was the configuration of the glibc package. I was getting multiple errors.Some of them have been higlighted under:
configure:3641: g++ -c -g -O2 conftest.cc >&5
conftest.cc:10:31: error: 'void std::exit(int)' should have been declared inside
'std'
configure:3647: $? = 1
To be noted I am using glibc-2.3.5.
I could only find out that the glibc ports have not been developed for Solaris platform.
I am referring to the following source: http://clpbar.sourceforge.net
Build process is the standard: ./configure followed by make
If I build on 10.5 I get a binary whose file contains: Mach-O executable i386
If I build on 10.6 I get a binary whose file contains: Mach-O 64-bit executable x86_64
How can I build from the command line on 10.6 and produce an executable of the type Mach-O executable i386, or even better a Universal binary containing executables of both types.
Please test any suggested solutions.
Thanks,
matt
By default, the version of GCC distributed with OS X 10.6 builds 64-bit binaries. The version of GCC distributed with 10.5 builds 32-bit binaries by default.
Before running ./configure, set a few environment variables. If you want it to be an Intel universal binary with i386 and x86_64 code, set the following variables (assuming you're using bash):
export CFLAGS="-arch i386 -arch x86_64"
export CXXFLAGS=$CFLAGS
export LDFLAGS=$CFLAGS
This will tell the configure script to pass these compiler and linker options, and thus build a binary with both the i386 and x86_64 architectures.
Or, if you want it to only build as an i386 binary:
export CFLAGS="-arch i386"
export CXXFLAGS=$CFLAGS
export LDFLAGS=$CFLAGS
From the Apple dev forums:
./configure CC="gcc -arch i386" CXX="g++ -arch i386"
Which works perfectly.
Use a -arch flag for each target. So, e.g.
gcc -arch i386 -arch x86_64 ...
would build a fat binary with both. I believe by default gcc builds to the target architecture, which is x86_64 with OSX 10.6.