Beam has support for Cython, but there are no instructions how to make use of it in an application (https://beam.apache.org/get-started/quickstart-py/). Are there any extra dependencies that need to be installed?
You are correct that the Python SDK has some Cython optimizations.
To ensure that these are available to you when you install Beam, make sure to have Cython installed in your environment. If you have Cython installed, Beam will go ahead and compile its modules when possible.
In short, before running pip install apache-beam, you'd want to run pip install Cython first.
To add some more detail, check out Beam's setup.py. When Beam is installed, it will check for a Cython installation, and if it does find one, it will attempt to cythonize its modules.
Related
I'm running Centos 6.7 on my server and am trying to install Erlang/Rabbitmq following these instructions:
Erlang Installation
RabbitMQ Installation
The trouble is that at time of writing these install Erlang 19.0 with RabbitMQ 3.6.3, which leads to a pretty major bug as far as my client who occasionally looks at the management interface to monitor queues is concerned.
The guidance in the error ticket is not to use erlang 19 until RabbitMQ 3.6.4 is released. But how can I install a specific version of Erlang?
These steps worked for me:
Go to the download page here: https://packages.erlang-solutions.com/erlang/
Select your appropriate package -- you can copy/peek the link then download it using wget.
Install it using rpm.
Example:
# Download erlang 19
$ wget http://packages.erlang-solutions.com/site/esl/esl-erlang/FLAVOUR_1_general/esl-erlang_19.0~centos~7_amd64.rpm
# Install
rpm -Uvh esl-erlang_19.0~centos~7_amd64.rpm
You can always build install from source.
Go to the Erlang.org Downloads page, pick your version from the right side.
From there you can follow the instructions. Although they are for Ubuntu, the commands are the same except for the dependencies part where you can use the command below to install what you need:
sudo yum install g++ openssl-devel unixodbc-devel autoconf ncurses-devel
Another option would be to use kerl, which is similar to rvm in some sense and very (very!) easy to use. It will let you install different Erlang versions and switch between them any time you want.
I prefer this approach instead of looking up packages myself (with possible incompatibilities in the dependencies required) or downloading and compiling everything myself every time I want to try a new Erlang version.
Is there an option to install SciPy on Windows without installing Anaconda as well? I could not do it via pip and everywhere it says to use Anaconda.
More details:
I want the SciPy package without any additional programs like Python(x, y) or Canopy.
The error with pip is: numpy.distutils.system_info.NotFoundError: no lapack/blas resources found. From research I found that I need to use additional packages but it sounds strange to me. I couldn't install LAPACK or BLAS.
There are unofficial builds: http://www.lfd.uci.edu/~gohlke/pythonlibs. Here's a link to scipy: http://www.lfd.uci.edu/~gohlke/pythonlibs#scipy
You can proceed with installing numpy or scipy using pip, after you install lapack and blas, which are system libraries. It shouldn't be very hard, but depends on you OS.
For RedHat/CentOS/Fedora this could be done with:
yum install lapack lapack-devel blas blas-devel
The packages can be found e.g. in CentOS base repository.
However, the scikit-learn website says as follows:
We don’t recommend installing scipy or numpy using pip on linux, as this will involve a lengthy build-process with many dependencies. Without careful configuration, building numpy yourself can lead to an installation that is much slower than it should be. If you are using Linux, consider using your package manager to install scikit-learn. It is usually the easiest way, but might not provide the newest version. If you haven’t already installed numpy and scipy and can’t install them via your operation system, it is recommended to use a third party distribution.
Package managers are usually yum or apt-get and again on RedHat/CentOS/Fedora you can skip using pip and install this way:
yum install scipy
Third party distributions mentioned above are things like anaconda or Python(x,y).
I want to setup a cross compile environment on Linux for the Raspberry Pi 1.
Especially I want to try bleeding edge version, i.e. Raspbian testing + Qt5 dev branch.
This question:
How can I create a modern cross compile toolchain for the Raspberry Pi 1?
...explains how to get a gcc compiler, which can create code for the Raspberry Pi 1. Are there changes necessary on Raspbian itself to use it? If so, which ones?
A full toolchain is what you need
A toolchain is a set of tools working together to generate binaries for your system. Depending on how you build your toolchain, it might end up in being only functional for your own image, that's not, in fact a problem, you just clone your image and upgrade it at will.
First, understand what you need:
Functional flagship system. This is your reference board and your reference distro bundle, your packages and your stuff. You might want a standard Raspbian or you might want some extra stuff, like OpenCV or less stuff like removing Xorg. You say you want bleeding edge, so fit your taste.
Sysroot. Ideally this is a copy of your Functional Flagship System with the added development headers. In my case its exactly the same, for Raspbian this is an image of your second partition, the one that hosts /.
Cross compiler. This is a compiler that generates code for ARM while running on x86 or x86_64. This is generally a specialized gcc.
Cross compiled qmake. For Qt you need a cross qmake, this is a qmake that will generate Qt binaries and uses things you need to generate your arm Qt software.
ARM Qt libraries. This is part of your Functional Flagship System, I just enumerate it here for the sake of clarity. They will get compiled by you using your sysroot and your cross compiler.
Qt Libraries for Cross Compiling. This are a product of the steps you will follow when generating your cross compiling qmake and ARM Qt Libraries. This will be installed in your host x86 system.
So how do you get all of this?
Gather Your Very Own Toolchain
Functional Flagship System (FFF). Just get your raspbian image and install your additional software at will, whatever you want to be in, just install it on a live Raspberry.
Sysroot. Once you have your FFF, then use dd to generate an image of your second raspbian partition. Get your card off, insert it into a x86 system and use dd. There are other ways using mount and offsets but this is a lot simpler.
Cross compiler. Unless you really know what you are doing, just refrain from creating it yourself. There are functional cross compilers.
Qmake for cross compiling, ARM Qt and Qt Libraries. This is the interesting part...
Cross Compiling Qt 5
You can go as bleeding edge as you please with Qt as you get it from git. As this is not really a Wiki I will just enumerate the steps. This guide explains it with a lot more detail.
Get your FFF, image and cross compiler working.
git clone your Qt, pick a tag (version)
mount your sysroot
Get ia32-libs if you are under x64
Compile qtbase then make install. IMPORTANT: After you get qtbase it generates it's own qmake, use it from now on.
Use the generated and installed qmake from qtbase to build any other Qt module you want.
Remember to use make install on all Qt modules you build. All these 'installs' will copy those binaries to your sysroot.
Get your Qt into your FFF. Either you copy the folder and avoid messing with permissions, or more easily just umount your sysroot, then use dd to dump the modified image to the very same physical partition you got it from. These are the ARM Qt Libraries.
When building qtbase it will install some stuff into your own x86 system. This is Qmake for cross compiling, use it into Qt Creator to generate cross compiled binaries along with your cross compiler.
Some notes nobody tells you
There seem to be no toolchains ready to download. This is because they depend a lot on your specific setup.
Do not use system or regular qmake to cross compile. Use your generated qmake, as it fits perfectly with your FFF, it has paths and other specific stuff baked in.
I repeat, do not bother creating your cross compiler
What if you need additional development files? Install them on your FFF, then copy your partition to have your new sysroot.
Yes, you can auto-deploy with Qt and even debug remotely on a live Pi.
Installing a bleeding edge development system/toolchain is a bit of a problem... It is a moving target. The following steps did work for me March 2015. If they still 100% work or how long they will work... But if one have read and understood the following 'walktrough' it should not be difficult to adjust the process for future Raspian or Qt5 versions.
Fist step should be to update Raspian. I upgraded to testing. To do this, change the repository in /etc/apt/sources.list to:
deb http://mirrordirector.raspbian.org/raspbian/ testing main contrib
non-free rpi
Followed by the usual 'apt-get update, apt-get upgrade, apt-get dist-upgrade'. Or an analogue aptitude command. After this step one has upgraded to the most recent Raspian. With all the risks and benefits of a testing release.
Next a couple of packages needs to be installed. Probably not all necessary, e.g. xcb does not work on a RPi, and the RPi hat its own set of opengl files. But some Raspian packages don't know this and might pull them in anyways. The packages below allow to compile a Qt5 with QMultimedia and
apt-get install -y "^libxcb.*" libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libicu-dev libxslt1-dev
apt-get install -y libssl-dev libxcursor-dev libxrandr-dev libfontconfig1-dev libcap-dev libbz2-dev libgcrypt11-dev
apt-get install -y libpci-dev libnss3-dev libxtst-dev libasound2-dev libcups2-dev libpulse-dev libudev-dev
apt-get install -y libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libproxy-dev libmtdev-dev libts-dev
apt-get install -y libxkbcommon-x11-dev libxkbcommon-dev libinput-dev libgbm-dev libjpeg8-dev libgif-dev libopenjpeg-dev
apt-get install -y libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev sqlite3 libsqlite3-dev libwayland-dev
apt-get install -y libdirectfb-dev libegl1-mesa-dev libsystemd-journal-dev libharfbuzz-dev xutils-dev libcairo2-dev
apt-get install -y libffi-dev libpam0g-dev
The next and most important step is also the most unpleasant one. A couple of libraries in Raspian are symbolic links with absolute paths. This is bad since those libraries are later on not found when Qt5 is compiled. All symlinks of relevant libs must be turned into symlinks with relative paths. With Google's help a script can be found, which did this almost automatically, but for some reason it did not work for me. Therefore I did it manually. If I have to do this more often, I certainly will write my own. This is also the step, which is most likely to break. Library versions change... so don't blindly copy/paste the commands below.
Not all of the libs below are necessary to compile Qt5, but all of them could be a problem eventually. After this step the Raspberry Pi is ready to be used. Next step is to compile and install Qt5.
EDIT: One of the side effects of writing such a mini-tutorial: One thinks again about certain things one has done. There is a much easier way to convert absolute links into relative links: symlinks.
So:
apt-get install symlinks
And then in /usr/lib/ on the Raspberry Pi:
symlinks -cr .
I would like to use psd-tools package from github.
Given I am not a programer, I found the installation process difficult to follow.
Can someone advice in simple terms how to install and use the psd-tool package.
Apparently, this is all done via the terminal in OSX.
If I understand correctly the package requires python, pip, pillow and packbits to run correctly.
Python is pre installed in OSX. Pip and packbits need to be installed. This where I start to get lost with the installation procedure.
Once the psd-tools package is installed, I am not sure how to call the package to test a psd file.
github psd-tools package
Download Python package for working with Adobe Photoshop PSD files
Installation
pip install psd-tools
Pillow should be installed if you want work with PSD image and layer data: export images to PNG, process them. PIL library should also work.
pip install Pillow
Note
In order to extract images from 32bit PSD files PIL/Pillow must be built with LITTLECMS or LITTLECMS2 support.
psd-tools also has a rudimentary support for Pymaging. Pymaging installation instructions are available in pymaging docs. If you want to use Pymaging instead of Pillow you'll also need packbits library:
pip install packbits
Usage of PSD-Tools and How to use it Read this.
Hope it Helps!
I'm good at installing package in Linux environment but newbie to Solaris OS. I need to install Python - libxml2 package to my project. Does the below command also work in Solaris server for installation??
sudo apt-get install libxml2 libxml2-dev
I have tried googling, unfortunately not able to get.
What you proposed is specific to Debian-based Linux distributions.
IMHO, the fastest way would be to download the libxml2 source code in order to compile and install it yourself.
If you're running Solaris 11, then pkg install libxml2 with sufficient privilege would be the right invocation. Determining the right package name is as simple as pkg search with a reasonable query (assuming that you're still connected to the repository from which you installed the system).
If you're running Solaris 10 or older, then you'll need the original install media, plus whatever patches have been issued that intersect SUNWlxml. But frankly, installing from source is probably easier at that point.