Docker: kafka confluent go client error - apache-kafka

I am trying to use apache kafka with go, things look good when i execute the project with go run but when i use docker build i get error....
# pkg-config --cflags rdkafka
Package rdkafka was not found in the pkg-config search path.
Perhaps you should add the directory containing `rdkafka.pc'
to the PKG_CONFIG_PATH environment variable
No package 'rdkafka' found
pkg-config: exit status 1
I installed librdkafka from https://github.com/confluentinc/confluent-kafka-go
git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
./configure --prefix /usr
make
sudo make install
I tried
PKG_CONFIG_PATH=/usr/lib/pkgconfig
source ~/.bashrc
but not luck. Any help is appreciated.

Probably you should include librdkafka.dll, msvcr120.dll and zlib.dll in your project root. At least this is what i should do to get this work on Windows. Not sure about Linux.

This below line inside the Dockerfile worked for me as this sets the environmental variable and this will persist when a container is run from the resulting image.
ENV PKG_CONFIG_PATH ${PKG_CONFIG_PATH}:/usr/lib/pkgconfig/

Related

Installation of protoc-gen-grpc-web

I am currently working on using grpc-web to write a simple client for my service. I have created a service.proto file which was successfully compiled using protoc. The problem arose when I tried to generate the gRPC-Web service client stub using the plugin protoc-gen-grpc-web which doesnt work despite having installed the latter.
Got the following error :
protoc -I=./ service.proto --grpc-web_out=import_style=commonjs,mode=grpcwebtext:./
protoc-gen-grpc-web: program not found or is not executable
--grpc-web_out: protoc-gen-grpc-web: Plugin failed with status code 1.
Any suggestions on how to solve this issue? Thank you!
You'll need to make the protoc-gen-grpc-web plugin executable and move it to a directory that is discoverable from your PATH environment variable.
From grpc-web/README:
For example, in MacOS, you can do:
$ sudo mv ~/Downloads/protoc-gen-grpc-web-1.2.1-darwin-x86_64 \
/usr/local/bin/protoc-gen-grpc-web
$ chmod +x /usr/local/bin/protoc-gen-grpc-web
You don't even have to install the plugin globally and make it discoverable from your PATH:
protoc accepts --plugin arguments to point to a required plugin. For protoc-gen-grpc-web (as for many others) there's even npm support, so you can get it with npm i --save-dev protoc-gen-grpc-web and then run protoc with --plugin=protoc-gen-grpc-web=./node_modules/.bin/protoc-gen-grpc-web
I'm using MacOS. You need to install this first with brew
$ brew install protoc-gen-grpc-web
The best solution that I could use for Linux based system was globally installing protoc-gen-grpc-web. This directly takes the files from the /bin of the node_modules files created globally across your system
sudo npm install -g protoc-gen-grpc-web

Cpplint could not find executable

I am trying to activate cpplint within vs code. I have installed it in Anacanda environment where executable
/home/ubuntu/anaconda3/bin/cpplint
I have a link to it
ls -l /home/ubuntu/anaconda3/bin/cpplint
Unfortunately per visual code cpplint extension still getting error of "Cpplint could not find executable"
Please, advice to configure it correctly.
Download and install
sudo apt-get install python-pip
pip install --user cpplint
Verify install result
ls -l /usr/local/bin/cpplint
If you still have issues check cpplint.cpplintPath and verify the execution path is set correctly.
Also, if you installed cpplint into ~/.local/ directories, by default ~/.local/bin is not included in PATH. So to fix just that add:
export PATH=$PATH:~/.local/bin/
to your ~/.bashrc

How to build library without sudo?

I usually build my library ./configure && make && sudo make install. However the Travis docs discourage using sudo http://docs.travis-ci.com/user/workers/container-based-infrastructure/
So I changed the build command to ./configure --prefix=$HOME && make && make install. This worked, however at the next step (building a Python extension) I got an error
/usr/bin/ld: cannot find -lprimesieve
Any ideas? Do I need to add $HOME/lib to some environment variables, because I changed prefix?
My travis config https://github.com/hickford/primesieve-python/blob/travis-ci/.travis.yml
Build log with error https://travis-ci.org/hickford/primesieve-python/jobs/69536543#L382
Try setting set LD_LIBRARY_PATH which is like PATH for libraries. For example:
LD_LIBRARY_PATH= $HOME/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH
More detailed information about library path variables is here.
Environment variables that specifically influence how the configure script passes arguments to compilation are LIBS and LD_FLAGS. bash ./configure --help mentions these.
And as you mention in the comments LIBRARY_PATH also needs to be set. See LD_LIBRARY_PATH vs LIBRARY_PATH for an explanation of the difference.

pg_ctl: error while loading shared libraries: libpq.so.5

I'm trying to get postgres server status with:
sudo /etc/init.d/postgres status -u postgres
But getting following error:
/home/alex/olddisk/usr/local/pgsql/bin/pg_ctl: error while loading shared libraries: libpq.so.5: cannot open shared object file: No such file or directory
I added:
export LD_LIBRARY_PATH=""
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib/"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/home/alex/olddisk/usr/local/pgsql/lib/"
to my .bashrc, but it didn't help.
Thank you.
I ran into this error when I built postgresql from source using the --prefix flag. Building from source installs the necessary shared libs to the libs folder under the prefix directory you specified, instead of the usual place installations put shared libs. To solve the problem I just added the [prefix].libs folder to the LD_LIBRARY_PATH environment variable. For example, after building postgres using --prefix /mike/sandbox/postgres, the below command solved the issue:
export LD_LIBRARY_PATH=/mike/sandbox/postgres/lib:$LD_LIBRARY_PATH
I think this issue is duplicated, I faced the same problem and posted a solution here.
try this:
1: Know the path of "libpq.so.5"
find / -name libpq.so.5
Output example: "/usr/pgsql-9.4/lib/libpq.so.5" If found nothing, check if you have already installed the suitable postgresql-libs for your postgresql version and your OS platform
2: Symbolic link that library in a "well known" library path like "/usr/lib":
ln -s /usr/pgsql-9.4/lib/libpq.so.5 /usr/lib/libpq.so.5
Attention: If your platform is 64 bit, you MUST also symbolic link to 64 bit libraries path:
ln -s /usr/pgsql-9.4/lib/libpq.so.5 /usr/lib64/libpq.so.5
3: Be happy !
Please ensure you have 'postgresql94' package installed as well (in addition to postgresql94-server, postgresql94-lib, postgresql94-devel and whatever other PG related package you already have). These libraries get installed along with that package.
Some ideas:
Your modified ~/.bashrc only takes effect when you start a new (interactive) shell. Though catching up on that will not help you because:
/etc/sudoers, your configuration file of sudo, probably specifies env_reset. This means, that /etc/init.d/postgres will not see the content of $LD_LIBRARY_PATH of your shell.
Insert debug statements in /etc/init.d/postgres to verify what I told you: echo "LDPATH: $LD_LIBRARY_PATH" >&2
Check /etc/init.d/postgres. Probably you will have to insert the third one of your export statements near the start of this script. Or you will have to update an existing export LD_LIBRARY_PATH= statement.
Have you installed the necessary libraries?
If you are using ubuntu try:
sudo apt-get install libpq libpq-dev

Setting up Mesos on CentOS

I tried to install the latest release tarball of Mesos on CentOS 6.4 with no luck. It ended up in all sorts of failures in trying to find jvm & jni bindings. Is there any instructions on how to install Mesos on RHEL or CentOS ?
I couldn't find any instructions around so I thought I would troubleshoot all through my way and thought of documenting it here so it can save your time.
First things first, load your CentOS box with essential build tools to get started
$ sudo yum groupinstall "Development tools"
Get Java and python dependencies installed
$ sudo yum install java-1.6.0-openjdk.x86_64 java-1.6.0-openjdk-devel.x86_64 python python-devel libcurl libcurl-devel
Get the latest Mesos tarball
$ wget http://mirror.nus.edu.sg/apache/mesos/0.13.0/mesos-0.13.0.tar.gz
$ tar -xzvf mesos-0.13.0.tar.gz
$ cd mesos-0.13.0
Before you can build Mesos, you need to set correct JAVA binding paths
$ export JAVA_HOME=/usr
$ export JAVA_LDFLAGS="-L/usr/lib/jvm/java-1.6.0/jre/lib/amd64/server -R/usr/lib/jvm/java-1.6.0/jre/lib/amd64/server -ljvm"
$ export JAVA_CPPFLAGS="-I/usr/lib/jvm/java-1.6.0/include -I/usr/lib/jvm/java-1.6.0/include/linux"
$ export LD_LIBRARY_PATH=/usr/lib/jvm/java-1.6.0/jre/lib/amd64/server:$LD_LIBRARY_PATH
Configure and build it
$ ./configure
$ make
After you have built Mesos, it is advisable that you build and run the tests, this will make sure that what you have installed meets all the requirements
$ make check
If the checks are successful, You are just one step away from installing it in your system installation paths
$ make install
To learn how to use Mesos , go here http://mesos.apache.org/gettingstarted/
For those who prefer installing from RPM's, here is a link to a number of different releases for different Linux flavors: http://mesosphere.io/downloads/ For example, for Centos64:
wget http://downloads.mesosphere.io/master/centos/6/mesos_0.14.2_x86_64.rpm
sudo rpm -Uvh mesos_0.14.2_x86_64.rpm
I also had to set my LD_LIBRARY_PATH, though to a slightly different value. Check yours.
Python bindings can also be downloaded from the first link above:
wget http://downloads.mesosphere.io/master/centos/6/mesos_0.14.2_x86_64.egg
sudo easy_install mesos_0.14.2_x86_64.egg