How to continously deploy a feature to karaf? - deployment

I want to continously deploy a feature to ServiceMix 6.0, which is based on Karaf 3.0.4.
I first tried this using the karaf console. However there are some problems. It is for a standard karaf installation not possible to determine on the karaf console if a feature is already installed (see my other question on this). The other problem with the karaf console is that it doesn't support exit codes. So it is not possible to determine reliably if a feature installation was successfully finished.
I then installed hawtio and tried to use the exposed JMX beans via jolokia/REST that is bundled with hawtio. The problem here is that karaf 3.0.x is unable to update a feature. Therefore features must be uninstalled first. However the FeatureService only offers the possibility to uninstall an explicitly specified feature. But when a previous version of the feature was installed, subfeatures were installed with it. They also need an upgrade and therefore be uninstalled first. So I would need a way to iterate over the subfeatures of a feature, which I do not have.
So how can continous deployment of features to karaf 3.0.x be done?

The first try we have implemented is a bash script. The biggest problem is uninstalling the old version. Therefore we have set up a convention for the names of the feature and it's subfeatures. So we can use the following to find already installed features:
features=$(echo "feature:list" | ssh -p $smx_ssh_port $smx_user#$smx_host | grep -h "<feature-name-convention-regex>.*|.*x.*|" | cut -f1 -d" " | tr '\n' ' ')
This can then be passed to feature:uninstall and can also be used for detecting if features were installed after the call to feature:repo-add -i.
The remaining problem is that we are unable to reference 3rd-party subfeatures because they won't be uninstalled when an updated version needs to be installed and we can't be sure if all of the subfeatures have been successfully installed.

For karaf 3 there is no good way to update features.
This is already a little better for karaf 4. It allows to update a feature repo and you can then simply install the feature again. It will detect that the feature has changed and do the necessary changes in bundles.

Related

Hyperledger FabCar Network issue via Visual Studio Code

While reproducing steps from an interesting tutorial found online - Hyperledger Fabric 1.4 Tutorial - FabCar Sample Application, I have installed all Hyperledger Fabric binaries via the Curl command:
curl -sSL https://raw.githubusercontent.com/hyperledger/fabric/master/scripts/bootstrap.sh | bash -s
From the Command Prompt window one can see that, the scripts are run correctly leading to the pull of hyperledger-docker images.
However, while launching the network from Visual Studio Code through the execution of
./startFabric.sh javascript from the fabcar sub-directory, one ends up with Fabric Images issues, which seems paradoxical.
A similar issue has been encountered, while attempting to bring up the network via the command
./network.sh up
from the test-network sub-directory too.
Thus, any relevant feedback would indeed be appreciated, given an absence of rationale to justify this matter.
[EDIT]
As a matter of fact, a synchronization issue might have caused the previously reported issue, since fabric-samples v.1.4.9 is not found from Curl command documented online. Thus the script automatically installs the latest Fabric binaries 2.x.
To rule out this issue, I have re-executed the Curl command by specifying the version 1.4.4 instead.
Therefore I can confirm, that the test-network sub-directory is not part of the Fabric binaries pertaining to the version 1.4x.
Furthermore, before the installation of this binaries, I had removed all containers, and related images, yielding to:
Back to the fabcar sub-directory, running:
./startFabric.sh javascript has still led to the following network issue :
What strikes me the most is:
firstly, that configtxlator can be found on my C:\Users\...\Documents\test4\fabric-samples\bin as opposed to the path highlighted on the command window.
secondly, I do not see the matter of version's incompatibility by scrutinizing byfn.sh:
Eventually, the manual amendment of IMAGETAG has not improved the situation.
Best
You seem to have the different version.
In hyperledger/fabric-samples, the test-network does not exist in version 1.4. In other words, you are running the code for the master version of fabric-samples (currently the version 2.x version). Since you're running the 1.4 tutorial, run the commands based on the 1.4 documentation and code.
fabric/docs/release-1.4
fabric/samples/release-1.4
[NOTE] It is also very likely that the different version binary has been installed. In the fabric document, the version can be specified and downloaded, so to download the binary of the corresponding version, you need to execute the command including the input parameter (version) below.
curl -sSL https://raw.githubusercontent.com/hyperledger/fabric/master/scripts/bootstrap.sh | bash -s - 1.4.9 1.4.9 0.4.21
[EDIT - 1]
Prior to the explanation, in 1.4.x, first-network/byfn.sh exists, not test-network/network.sh. Change the branch of your fabric-samples to release-1.4.
Do not run 2.x code while doing the 1.4 tutorial.
./byfn.sh basically executes the image tag as latest. That is, if you have pulled the 2.x version of the image, even if you proceed to 1.4.x, it will run as 2.x.
cd fabric-samples
git checkout release-1.4
There are two ways.
tag the latest with 1.4.x image
Untag all hyperledger latest image tags, and tag 1.4.x images as latest.
(In fact, the easiest way is to delete both the 2.x and 1.4.x images and execute the command again.)
Or edit in byfn.sh
There is a part to set IMAGE_TAG in byfn.sh.
byfn.sh
# default image tag
# IMAGETAG="latest"
IMAGETAG="1.4.9"
Make sure the value of that part matches your 1.4.x version
[EDIT - 2]
For binary I don't know where and how you built it. However, you can easily build it on the fabric github.
make binaries
cd $GOPATH/src/github.com/hyperledger
git clone http://github.com/hyperledger/fabric
cd fabric
git checkout release-1.4
make native
Makefile
native - ensures all native binaries are available
configtxgen - builds a native configtxgen binary
configtxlator - builds a native configtxlator binary
cryptogen - builds a native cryptogen binary
idemixgen - builds a native idemixgen binary
peer - builds a native fabric peer binary
orderer - builds a native fabric orderer binary
Path setting
mv $GOPATH/src/github.com/hyperledger/fabric/.build/bin/* <your_bin_path>
# in my case
# mv $GOPATH/src/github.com/hyperledger/fabric/.build/bin/* usr/local/bin
or
export PATH=$PATH:$GOPATH/src/github.com/hyperledger/fabric/.build/bin

Making VS Code Remote extension work with GLIBC 2.17 installed in non standard locations

I'm trying to use VSCode Remote extension to connect to a remote host that runs on RHEL/CentOS 6, but it fails to connect since CentOS 6 ships with GLIBC 2.12 and GLIBCXX 3.4.1. As mentioned in this post, in order to get the extension to work, the workaround is to install GLIBC>=2.17 and GLIBCXX>=3.4.18.
Unfortunately, I don't have sudo access for the server, so I won't be able to update these libraries using the bash script provided in the link. Also, in this SO post, the author says not to update the system GLIBC since it can break down system applications. That being said, I've tried something different -- I extracted those rpm packages, as described in this blog, inside my home folder. I've then updated the env variables PATH and LD_LIBRARY_PATH in ~/.bash_profile to point to these new locations. But the node binary (in VS Code Remote) still can't find these libraries.
Is there a way to let the node binary know where to look for these libraries? More precisely, can someone explain how I can make this extension work without sudo access?
I've got it to work by installing gcc and glibc using Linuxbrew. See this post for more details: https://github.com/microsoft/vscode-remote-release/issues/103#issuecomment-546551293.
Couple of things to take note of:
Node binary versions in VS Code Server may vary between commits. In the GitHub comment above, the author uses node#10 -- you may replace it with node#12; everything would still work.
Make sure glibc and gcc are properly installed using linuxbrew. This step is key.

Add Java in Python Flask Cloud Foundry

I need to run java command from python flask application which is deployed using cf. How can we make java runtime available to this python flask app.
I tried using multi-buildpack, but java_buildpack expects some jar or war to be executed while deploying the application.
Any approach that would make java available to python flask app?
The last buildpack in the buildpack chain is responsible for determining a command to start your application which is why the Java buildpack expects a JAR/WAR to execute.
The Java buildpack, as of writing this, does not ship a supply script so it can only be run as the last buildpack when using multi buildpack support. It looks like that at some point in the future the Java buildpack will provide a supply script, but this is still being worked out here.
For now, what you can do is use the apt-buildpack and install a JRE/JDK that way.
To do this, add a file called apt.yml to the root of your project folder. In that file, put the following:
---
packages:
- openjdk-8-jre
repos:
- deb http://ppa.launchpad.net/openjdk-r/ppa/ubuntu trusty main
keys:
- https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xEB9B1D8886F44E2A
This will tell the apt buildpack to add a PPA for Ubuntu Trusty where we can get the latest openjdk8. This gets installed under /home/vcap/deps/0, which puts the java executable at /home/vcap/deps/0/lib/jvm/java-8-openjdk-amd64/bin/java.
Note: The java binary is unfortunately not on the path because of the way Ubuntu uses update-alternatives and we can't use that tool to put it on the path in the CF app container because we don't have root access.
After setting that up, you'd follow the normal instructions for using multiple buildpacks.
$ cf push YOUR-APP --no-start -b binary_buildpack
$ cf v3-push YOUR-APP -b https://github.com/cloudfoundry/apt-buildpack#v0.1.1 -b python_buildpack
Note: The process to push using multiple buildpacks will likely change in the future and v3-push, which is currently experimental, will go away.
Note: The example above hard codes version v0.1.1 of the apt buildpack. You should use the latest stable release, which you can find here. Using the master branch is not recommended.
One way to achieve your goal to combine Java and Python would be with context-based routing. I have an example to combines Python and Node.js, but the approach is the same.
Basically, you have a second app serving one or more paths of a domain / URI.

Update of elasticsearch plugin

I have developed a couple of plugins for elasticsearch, and these may require (more or less) frequent updates.
My simple question is: is there a way of updating an elasticsearch plugin without having to remove the old version, delete the relevant indexes, install the new version and rebuild the indexes from scratch?
Thanks in advance.
There is no way to update an existing plugin. You need to delete the old version and install the new one.
I didn't get your question about indices though. A plugin doesn't necessarily work against data, it can just be a site plugin, a query parser etc. In case a plugin does work against indices and you want to upgrade it, but the elasticsearch version stays the same, I don't see why you would need to reindex. The only case is if the plugin itself changed in a non backwards compatible way.
As of the latest version 2.4.1 (2016-10-18) there is still no way to do this easily, because the elasticsearch folks recommend manual plugin updates.
Expect that when you update Elasticsearch and start the service, that you will end up with an error because service won't start due to a plugin being one minor version behind the ES version, e.g. "license".
Go to your elasticsearch bin directory and run the following commands:
sudo ./plugin remove <plugin name>
sudo ./plugin install <plugin name>
You could even be so bold as to write an "update" shell script that does this for you.
A bash script here:
#!/bin/bash
plugin_bin=/usr/share/elasticsearch/bin/plugin
for plugin in $($plugin_bin list | awk '/^\s*-\s*(\S+)/{print $2}')
do
$plugin_bin remove $plugin && $plugin_bin install $plugin
done

How to update eclipse in command line?

I want to install some plugins and upgrade some to the latest version. They are not included in the Eclipse SDK downloads. So I must install the individuals manually.
The problem is, the network connection to the update site is very slow. (Maybe 1k bytes per second, or lower) Currently, I have an Amazon EC2 box in us-east-1 region, which is a lot faster to the update site. So, I'd like to install the plugins in EC2 box, and then copy them back to my local machine (in China).
That will look like:
very slow, or inaccessible
My machine #China <-----------------------------------> Update Sites
(1k bytes/sec)
|
|
|
V
much faster fast
My machine #China <-----------------> EC2 #US-east <------------------> Update Sites
(100k bytes/sec) (1M bytes/sec)
Now, my EC2 box is running Debian linux, I'm not going to install X windows there, so it can't start Eclipse in GUI mode. Though, I hope I can get updates from command line only.
EDIT
To clarify the question: how to "Install Software" for Eclipse in command line? Since Eclipse is OSGi-based, I guess I can do it using some kind of osgi shell, maybe?
You can install and delete plugins for eclipse through a CLI. This is possible through the p2.director:
https://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Fguide%2Fp2_director.html
In regards to updating eclipse through CLI, I stumbled across this question looking for that answer.