CDH 5.4 upgrade to HUE to 3.8 - upgrade

I am using CDH 5.4. I see the version of HUE is 3.7.
I want to use Spark notebook feature in HUE 3.8 but I do not know how to upgrade HUE to 3.8

I find the instruction how to update HUE at HUE wiki page. It is removed now so I place my update bash script here
#!/bin/bash
CDH_LIB=/opt/cloudera/parcels/CDH/lib
# Download tarball
wget https://dl.dropboxusercontent.com/u/730827/hue/releases/3.8.1/hue-3.8.1.tgz
# Backup old version
sudo mv $CDH_LIB/hue $CDH_LIB/hue-3.7
# Install libs
sudo apt-get -y install python2.7-dev
sudo apt-get -y install libxslt1-dev
sudo apt-get -y install libmysqlclient-dev
sudo apt-get -y install libsqlite3-dev
sudo apt-get -y install make
sudo apt-get -y install libkrb5-dev
sudo apt-get -y install libxml2-dev
sudo apt-get -y install libssl-dev
sudo apt-get -y install libldap2-dev
sudo apt-get -y install python-pip
sudo apt-get -y install libsasl2-dev
sudo apt-get -y install python-ldap
tar -zxvf hue-3.8.1.tgz
cd hue-3.8.1/
#sudo rm -rf /opt/cloudera/parcels/CDH/lib/hue
# Build
sudo PREFIX=$CDH_LIB make install
# Copy configs
sudo rsync -aL $CDH_LIB/hue-3.7/desktop/conf/ $CDH_LIB/hue/desktop/conf/
# Copy db
sudo cp $CDH_LIB/hue-3.7/desktop/desktop.db $CDH_LIB/hue/desktop/desktop.db
# Sync db
sudo $CDH_LIB/hue/build/env/bin/hue syncdb
sudo $CDH_LIB/hue migate

Related

optimise multiple execution of same command - Docker

I am trying to install and configure postgresql-client-11 on my ubuntu18:04. I understand that we need to perform a few extra steps shown below to do the same (Source: https://www.itzgeek.com/how-tos/linux/ubuntu-how-tos/how-to-install-postgresql-10-on-ubuntu-18-04-lts.html)
wget -q https://www.postgresql.org/media/keys/ACCC4CF8.asc -O- | sudo apt-key add -
echo "deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main" | sudo tee /etc/apt/sources.list.d/postgresql.list
sudo apt-get update
sudo apt-get install -y postgresql-client-11
I am achieving it with the following
RUN apt-get update && apt-get dist-upgrade -y && apt-get install -y wget && apt-get install -y gnupg2
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main" | tee /etc/apt/sources.list.d/postgresql.list
RUN wget https://www.postgresql.org/media/keys/ACCC4CF8.asc
RUN apt-key add ACCC4CF8.asc
RUN apt-get update && apt-get dist-upgrade -y && DEBIAN_FRONTEND=noninteractive apt-get install -y nodejs && rm -rf /var/lib/apt/lists/*
I am executing the same command RUN apt-get update && apt-get dist-upgrade -y twice.
Is there any way to optimise the above and not have redundant commands.
You can tweak this a little, but fundamentally you need two RUN apt-get update && apt-get install lines. The essential dependency chain is:
The last apt-get install line depends on having the upstream PostgreSQL repository available
You can't apt-get install from a repository until you apt-get update from it
You can't do that apt-get update without the preceding lines to add the sources.list line and load the public key
You can't get the public key without wget and gnupg
Which requires an additional apt-get install and preceding apt-get update
There's no particular need to run dist-upgrade twice and I'd skip it the first time (and maybe the second time: the Docker Hub ubuntu image is updated pretty regularly). You can combine multiple apt-get install commands together, and if you have the rm clause in there you need to run it every time. You also might consider whether you need a client library newer than what Ubuntu packages; if an older client library will work fine you can simplify this significantly.
So I might wind up somewhere like:
# Install prerequisites to install prerequisites
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends --assume-yes \
gnupg2 \
wget \
&& rm -rf /var/lib/apt/lists/*
# Configure upstream PostgreSQL repository
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main" | tee /etc/apt/sources.list.d/postgresql.list \
&& wget https://www.postgresql.org/media/keys/ACCC4CF8.asc \
&& apt-key add ACCC4CF8.asc
# Actually install the packages we need
RUN apt-get update && \
&& DEBIAN_FRONTEND=noninteractive apt-get dist-upgrade -y \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends --assume-yes \
nodejs \
postgresql-client-11 \
&& rm -rf /var/lib/apt/lists/*

How to upgrade Visual Studio Code editor?

What's the best way to upgrade Visual Studio Code on Linux Ubuntu?
For the time being I was periodically getting the newest version (.deb) from their official site: https://code.visualstudio.com/
sudo dpkg -i code_*.deb
Visual Studio Code enabled official Linux repositories on February 2017 (v1.10)
sudo add-apt-repository -y "deb https://packages.microsoft.com/repos/vscode stable main "
sudo apt update
sudo apt -y install code
You can upgrade / dist-upgrade as usual
sudo apt -y upgrade
sudo apt -y dist-upgrade
[1]: download the latest vscode (.deb) package to your computer on this
link :
https://go.microsoft.com/fwlink/?LinkID=760868
, or this there :
https://code.visualstudio.com/docs/
[2]: then open a terminal in the folder where you downloaded the .deb file and write:
sudo dpkg -i <the downloaded file>.deb
[3]: finally if you have apt-get do (if not install apt-get first):
sudo apt-get install -f
If you have installed it via the repository, exit VS Code then just do:
sudo apt update
sudo apt install code
This is the same command to install or upgrade to the latest version. You can see the version with:
code --version
Now the easiest and recommended way is to use snap:
sudo snap install --classic code
And updates are supposed to be automatic.
This works fine in ubuntu.
sudo apt-get update
sudo apt-get install code
When you install VSCode with the file .deb on Ubuntu 20.08, first, remove it:
sudo apt-get remove code
Add the repository in this link https://code.visualstudio.com/docs/setup/linux
wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > packages.microsoft.gpg
sudo install -o root -g root -m 644 packages.microsoft.gpg /etc/apt/trusted.gpg.d/
sudo sh -c 'echo "deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/packages.microsoft.gpg] https://packages.microsoft.com/repos/vscode stable main" > /etc/apt/sources.list.d/vscode.list'
Udate the package cache and reinstall
sudo apt-get install apt-transport-https
sudo apt-get update
sudo apt-get install code
In the next time when you want to upgrade, just do:
sudo apt-get update
sudo apt-get upgrade code
Because your repository is missing information to upgrade VSCode, the above solution will fix it.
This is what I did to avoid the annoying message:
Remove vscode, if you already installed it.
sudo apt-get remove code
Add repositories, update and install:
sudo add-apt-repository -y "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main "
sudo apt update
sudo apt -y install code
If you have already installed VS code, go to the terminal and type two different commands:
sudo apt update
sudo apt-get upgrade code
The following commands work for me (for Linux) :
wget 'https://code.visualstudio.com/sha/download?build=stable&os=linux-deb-x64' -O /tmp/code_latest_amd64.deb
sudo dpkg -i /tmp/code_latest_amd64.deb
Place those two commands into an executable Bash script called auto-update-VSCode, and you can simply run that from your shell any time Visual Studio Code says it's out of date.
I'm running Ubuntu 20.04 and this worked perfectly for me:
$ wget https://vscode-update.azurewebsites.net/latest/linux-deb-x64/stable -O /tmp/code_latest_amd64.deb
$ sudo dpkg -i /tmp/code_latest_amd64.deb
Best way to update Vscode in Ubuntu:
sudo apt-get update
sudo apt-get install code
sudo apt-get update
sudo apt-get install code

error in installing libpq-dev in ubuntu 14.04 after installing psycopg2

I am trying to install all requirements of a project so i run the command
pip install -r requirements.txt
but it gives the error
Downloading/unpacking psycopg2==2.6 (from -r requirements.txt (line 58))
Downloading psycopg2-2.6.tar.gz (367kB): 367kB downloaded
Running setup.py (path:/tmp/pip_build_root/psycopg2/setup.py) egg_info for package psycopg2
Error: b'You need to install postgresql-server-dev-X.Y for building a server-side extension or libpq-dev for building a client-side application.\n'
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
Error: b'You need to install postgresql-server-dev-X.Y for building a server-side extension or libpq-dev for building a client-side application.\n'
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/psycopg2
Storing debug log for failure in /home/admin/.pip/pip.log
now i knew that i have to install postgresql-server so i used following command
sudo apt-get install postgresql
all good till now and then i ran
sudo apt-get install python-psycopg2
and then i ran my command to install requirements again but got the same error and then i searched on google and tried
sudo apt-get install libpq-dev
but i got following error
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
libpq-dev : Depends: krb5-multidev but it is not going to be installed
Depends: comerr-dev but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
now i am unable to detect and correct broken package...please help me...
I ran the following commands to repair broken package and then i successfully installed Psycopg2 and Postgresql ...
sudo apt-get update
sudo apt-get install python-dev
sudo apt-get install libpq-dev
sudo apt-get install krb5-multidev
sudo apt-get install libssl1.0.0=1.0.1f-1ubuntu2 ibcomerr2=1.42.9-3ubuntu1
sudo apt-get install libssl1.0.0=1.0.1f-1ubuntu2
sudo apt-get install ibcomerr2=1.42.9-3ubuntu1
sudo apt-get update
sudo apt-get install libpq-dev
apt-get install libssl-dev krb5-multidev comerr-dev
sudo apt-get install libssl-dev krb5-multidev comerr-dev
sudo apt-get dist-upgrade
sudo apt-get update
sudo apt-get install libssl-dev krb5-multidev comerr-dev
sudo apt-get install libcomerr2
sudo apt-get install comerr-dev
sudo apt-get install libcomerr2
sudo apt-get install comerr-dev
sudo apt-get install comerr-dev
sudo apt-get install 1.42.9-3ubuntu1.3
sudo apt-get install ibcomerr2=1.42.9-3ubuntu1
sudo apt-get install git
git clone https://github.com/kos0/addRemoveDistribution.git
cd addRemoveDistribution
sudo ./enableDisableDistribution.pl enable security
sudo ./enableDisableDistribution.pl enable updates
sudo apt-get update
sudo apt-get install libpq-dev
sudo apt-get update
pip install -r requirements.txt
Just follow only Steps
sudo apt-get install git
git clone https://github.com/kos0/addRemoveDistribution.git
cd addRemoveDistribution
sudo ./enableDisableDistribution.pl enable security
sudo ./enableDisableDistribution.pl enable updates
sudo apt-get update
sudo apt-get install libpq-dev
sudo apt-get update
I Fixed issue by those above steps

Installing MongoDb on Lubuntu

I am trying to install MongoDD on Lubuntu following the official instructions for Ubuntu.
But when I try to run this line:
sudo apt-get install -y mongodb-org
I get this error:
jon#jon-HP-ENVY-17-Notebook-PC:~$ sudo apt-get install -y mongodb-org
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package mongodb-org
jon#jon-HP-ENVY-17-Notebook-PC:~$
What am I doing wrong?
Try "sudo apt-get update" before "sudo apt-get install -y mongodb-org"...
Or try tutorial for Lubuntu, not Ubuntu, like this: http://tutorialforlinux.com/2014/03/12/how-to-install-latest-mongodb-on-lubuntu-12-04-precise-lts-32-64bit-linux-easy-guide/
May be it's beacause of the old MongoDb's version. Try with the latest MondoDb repository url to generate the file.
Before trying sudo apt-get install -y mongodb-org , execute the following commands in your terminal
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
# following is the repository url for Latest MongoDb version 3.0
echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list
sudo apt-get update
sudo apt-get install -y mongodb-org

Can't access exe symlinks in centos 6.6 on docker

I'm trying to access the /proc/< pid >/exe symlink on centos 6.6 but i'm geting
ls: cannot access /proc/57/exe: Permission denied
My Dockerfile is
FROM centos:6
# install base packages
RUN yum -y update
RUN yum -y groupinstall "Development Tools"
RUN yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite- devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel
RUN yum -y install wget
RUN wget http://python.org/ftp/python/2.7.6/Python-2.7.6.tar.xz
RUN yum -y install tar
RUN tar -xf Python-2.7.6.tar.xz
RUN cd Python-2.7.6 && ./configure --prefix=/usr/local --enable-unicode=ucs4 --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"
RUN cd Python-2.7.6 && make
RUN cd Python-2.7.6 && make altinstall
# First get the setup script for Setuptools:
RUN wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
# Then install it for Python 2.7 and/or Python 3.3:
RUN python2.7 ez_setup.py
# Now install pip using the newly installed setuptools:
RUN easy_install-2.7 pip
#lxml
RUN yum -y install python-devel libxml2-devel libxslt-devel
RUN pip install lxml
RUN pip install pytest
RUN yum install -y gcc-c++
RUN pip install protobuf
RUN yum install -y rpm-build
This happens for any process.
Thanks in advance.