Install MongoDB on Debian Buster - mongodb

How to install the latest MongoDB 3.4 or even 3.6?
They support with Ubuntu, but my server is Debian Buster and I am stuck with MongoDB 3.2.

I don't know if this is a good idea yet, but I just installed it by adding the sid repos and installing using the mongodb-server package. For me this installs version 3.4.18.
I created /etc/apt/sources.list.d/sid.list with:
deb http://deb.debian.org/debian/ sid main
deb-src http://deb.debian.org/debian/ sid main
then did
apt update
apt install mongodb-server
and verified that it's working by connecting with mongo.

I have found the solution for a build script, the description is found here:
https://github.com/patrikx3/docker-debian-testing-mongodb-stable
The description:
Debian Stretch / Buster / Bullseye / Testing MongoDB and MongoDB Tools build stable builder script, what it does as exactly:
It is basically a built for the latest MongoDB for Debian.
The current varsion is the r4.0.x build (release).
Warning It will remove all mongodb* apt packages in ./scripts/build-server.sh and /etc/systemd/system/mongodb-server.service is replaced.
It install the required apt dependencies and generates the SystemD service and makes it enabled.
Check if the build works (building is below). It runs all tests, so if it works, then it really does, actually. If there is an error, of course, you will not deploy on your server. So, if building and testing works, then it puts the binaries as it follow and you are sure and done.
The build as follows build-server.sh:
#!/usr/bin/env bash
# based on https://github.com/mongodb/mongo/wiki/Build-Mongodb-From-Source
# the current directory
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# if an error exit right away, don't continue the build
set -e
# some info
echo
#echo "Works like command, use a tag: sudo ./scripts/build-server.sh r4.2.0"
echo "Works like command, use a tag: sudo ./scripts/build-server.sh r4.0.12"
echo
# check if we are root
if [[ $EUID -ne 0 ]]; then
echo "This script must be ran via root 'sudo' command or using in 'sudo -i'."
exit 1
fi
# require mongo branch
#if [ -z "${1}" ]; then
# echo "First argument must be the MONGODB_BRANCH for example 'v4.1'"
# exit 1
#fi
#MONGODB_BRANCH="${1}"
# require mongo release
#if [ -z "${2}" ]; then
# echo "The second argument must be the MONGODB_RELEASE for example 'r4.1.0'"
# exit 1
#fi
#MONGODB_RELEASE="${2}"
# require mongo release
if [ -z "${1}" ]; then
echo "The first argument must be the MONGODB_RELEASE for example 'r4.0.12'"
exit 1
fi
MONGODB_RELEASE="${1}"
# delete all mongo other programs, we self compile
apt remove --purge mongo*
# the required packages for debian
apt -y install libboost-filesystem-dev libboost-program-options-dev libboost-system-dev libboost-thread-dev build-essential gcc python scons git glibc-source libssl-dev python-pip libffi-dev python-dev libcurl4-openssl-dev #libcurl-dev
pip install -U pip pyyaml typing
# generate build directory variable
BUILD=$DIR/../build
# delete previous build directory
rm -rf $BUILD/mongo
# generate new build directory
mkdir -p $BUILD
# the mongodb.conf and systemd services files in a directory variable
ROOT_FS=$DIR/../artifacts/root-filesystem
# find out how many cores we have and we use that many
if [ -z "$CORES" ]; then
CORES=$(grep -c ^processor /proc/cpuinfo)
fi
echo Using $CORES cores
# go to the build directory
pushd $BUILD
# clone the mongo by branch
#git clone -b ${MONGODB_BRANCH} https://github.com/mongodb/mongo.git
# clone the mongo by branch
git clone https://github.com/mongodb/mongo.git
# the mongo directory is a variables
MONGO=$BUILD/mongo
# go to the mongo directory
pushd $MONGO
# checkout the mongo release
git checkout tags/${MONGODB_RELEASE}
# hack to old version python pip cryptography from 1.7.2 to use the latest
sed -i 's#cryptography == 1.7.2#\#cryptography == 1.7.2#g' buildscripts/requirements.txt
# this is only because 4.0.12 uses 1.7.2 and
# https://github.com/pyca/cryptography/issues/4193#issuecomment-381236459
# support minimum latest (2.2)
pip install cryptography
# install the python requirements
#pip install -r etc/pip/dev-requirements.txt
pip install -r buildscripts/requirements.txt
# somewhere in the build it says if we install this, it is faster to build
pip2 install --user regex
# build everything
scons all --disable-warnings-as-errors -j $CORES --ssl
# install the mongo programs all
scons install --disable-warnings-as-errors -j $CORES --prefix /usr
# create a copy of the old config
#TIMESTAMP=$(($(date +%s%N)/1000000))
#cp /etc/mongodb.conf /etc/mongodb.conf.$TIMESTAMP.save
# copy the mongodb.conf configured and the systemd service file
# dangerous!!! removed
# cp -avr $ROOT_FS/. /
MONGODB_SERVICE=etc/systemd/system/mongodb-server.service
cp $ROOT_FS/$MONGODB_SERVICE /$MONGODB_SERVICE
chown root:root /$MONGODB_SERVICE
chmod o-rwx /$MONGODB_SERVICE
# generate mongodb user and group
useradd mongodb -d /var/lib/mongodb -s /bin/false || true
# create the required mongodb database directory and add safety
mkdir -p /var/lib/mongodb
chmod o-rwx -R /var/lib/mongodb
chown -R mongodb:mongodb /var/lib/mongodb
# create the required mongodb log directory and add safety
mkdir -p /var/log/mongodb
chmod o-rwx -R /var/log/mongodb
chown -R mongodb:mongodb /var/log/mongodb
# create the required run socket directory and add safety
mkdir -p /run/mongodb
chmod o-rwx -R /run/mongodb
chown -R mongodb:mongodb /run/mongodb
# add safety to the mongodb config file
chmod o-rwx /etc/mongodb.conf || true
chown mongodb:mongodb /etc/mongodb.conf || true
# reload systemd services
systemctl daemon-reload
# enable the mongodb-server
systemctl enable mongodb-server
# start the mongodb-server
#service mongodb-server start
# exit of the mongo directory
popd
# exit the build directory
popd
# delete current build directory
rm -rf $BUILD/mongo
The build as follows build-tools.sh:
#!/usr/bin/env bash
# based on https://github.com/mongodb/mongo-tools
# the current directory
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# if an error exit right away, don't continue the build
set -e
# some info
echo
echo "Works like command: sudo ./scripts/build-tools.sh r4.0.12"
echo
# check if we are root
if [[ $EUID -ne 0 ]]; then
echo "This script must be ran via root 'sudo' command or using in 'sudo -i'."
exit 1
fi
# require mongo release
if [ -z "${1}" ]; then
echo "The first argument must be the MONGODB_RELEASE for example 'r4.0.12'"
exit 1
fi
MONGODB_RELEASE="${1}"
## delete all mongo other programs, we self compile
##apt remove --purge mongo*
## the required packages for debian
##apt -y install gcc python scons git glibc-source libssl-dev python-pip
apt -y install golang libpcap-dev
export GOROOT=$(go env GOROOT)
# generate build directory variable
BUILD=$DIR/../build/src/github.com/mongodb/
# delete previous build directory
rm -rf $BUILD/mongo-tools
# generate new build directory
mkdir -p $BUILD
# find out how many cores we have and we use that many
CORES=$(grep -c ^processor /proc/cpuinfo)
# go to the build directory
pushd $BUILD
# clone the mongo by branch
git clone https://github.com/mongodb/mongo-tools
# the mongo directory is a variables
MONGO_TOOLS=$BUILD/mongo-tools
# go to the mongo directory
pushd $MONGO_TOOLS
# checkout the mongo release
git checkout tags/${MONGODB_RELEASE}
bash ./build.sh
chown root:adm -R ./bin
chmod o-rwx -R ./bin
chmod ug+rx ./bin/*
cp -r ./bin/. /usr/bin
# for PROGRAM in bsondump mongodump mongoexport mongofiles mongoimport mongoreplay mongorestore mongostat mongotop
# do
# go build -o bin/${PROGRAM} -tags "ssl sasl" ${PROGRAM}/main/${PROGRAM}.go
# done
# exit of the mongo directory
popd
# exit the build directory
popd
# delete current build directory
rm -rf $BUILD/mongo-tools
popd
# delete current build directory
rm -rf $BUILD/mongo-tools

Related

node-poppler installation of poppler-utils and poppler-data

https://github.com/Fdawgs/node-poppler
When following the readme one thing I ran into is poppler-utils and poppler-data didn't get installed in /usr/bin. Based on that readme I'm expecting them to get installed there by default.
After running a find inside the container I found the files in /usr/share/doc. This doesn't seem right based on the readme.
How do I ensure poppler-utils and poppler-data get added into /usr/bin as expected in the readme.
Ultimately this is the code I'm instantiating:
const poppler = new Poppler('/usr/bin');
Dockerfile:
FROM docker.registry.sfg.corp.local/devops/nodejs-build-docker:16.16.60850 as build
ARG NAME
ARG IMAGE
ARG SNYK_TOKEN
ARG SNYK_ORGANIZATION
ARG SNYK_PROJECT
COPY . .
RUN apt-get update
RUN apt-get install poppler-utils -y
RUN apt-get install poppler-data -y
RUN chmod +x install-puppeteer.sh
RUN ./install-puppeteer.sh
RUN ./build.sh -n $NAME -i $IMAGE -t $SNYK_TOKEN -o $SNYK_ORGANIZATION -p $SNYK_PROJECT
FROM docker.registry.sfg.corp.local/node:16-buster
RUN apt-get update
RUN apt-get install poppler-utils -y
RUN apt-get install poppler-data -y
COPY ./install-puppeteer.sh .
RUN chmod +x install-puppeteer.sh
RUN ./install-puppeteer.sh
# add the chrome folder to the PATH
ENV PATH "$PATH:/opt/google/chrome"
COPY --from=build package.json package.json
COPY --from=build src src/
COPY --from=build node_modules node_modules/
COPY --from=build artifacts artifacts/
# Add tini to help prevent zombie chrome processes.
ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini /tini
RUN chmod +x /tini
# Add user so we don't need --no-sandbox.
RUN groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
&& mkdir -p /home/pptruser/Downloads \
&& mkdir -p /home/pptruser/.cache \
&& mkdir -p /home/pptruser/.cache/puppeteer \
&& chown -R pptruser:pptruser /home/pptruser \
&& chown -R pptruser:pptruser /node_modules
USER pptruser
RUN chmod +x node_modules/riley/bin/riley.sh
ENTRYPOINT ["/tini", "--"]
CMD ["node_modules/riley/bin/riley.sh"]

I got "ERROR nothing RPROVIDES" during bitbake

I tried to run
bitbake core-image-minimal
but I got
ERROR: Nothing RPROVIDES 'libcrypto' (but /home/yocto/fsl-4-14-98/sources/poky/meta/recipes-core/images/core-image-minimal.bb RDEPENDS on or otherwise requires it)
In core-image-minimal.bb I have
SUMMARY = "A console-only image that fully supports the target device \ hardware."
IMAGE_FEATURES += "splash"
LICENSE = "MIT"
inherit core-image
I also got a second error
ERROR: Required build target 'core-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-minimal', 'libcrypto']
What am I missing? It's my first time using Yocto.
You can find the tutorial I'm using below:
- prepare system for Yocto
$ sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib \
build-essential chrpath socat cpio python python3 python3-pip python3-pexpect \
xz-utils debianutils iputils-ping
$ sudo apt-get install libsdl1.2-dev xterm
$ sudo apt-get install make xsltproc docbook-utils fop dblatex xmlto
$ sudo apt-get install ncurses-dev
- setting up repo
$ mkdir ~/bin
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
- Add the following line to the .bashrc file to ensure that the ~/bin folder is in your PATH variable.
export PATH=~/bin:$PATH
- configure git:
$ git config --global user.name "Your Name"
$ git config --global user.email "Your Email"
$ git config --list
NEW YOCTO RELEASE WITH KERNEL 4.14.98 AND OPEN SSL 1.1.1J
----
$ cd /usr/bin
$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt-get update
$ sudo apt-get install python3.6
$ sudo rm python
$ sudo ln -s python3.6 python
- load iMX recipes
$ cd
### OLD RELEASE ### $ mkdir fsl-release-bsp
### OLD RELEASE ### $ cd fsl-release-bsp
### OLD RELEASE ### $ repo init -u https://source.codeaurora.org/external/imx/fsl-arm-yocto-bsp -b imx-4.1-krogoth
### OLD RELEASE ### $ repo sync
$ mkdir fsl-4-14-98
$ cd fsl-4-14-98
$ repo init -u https://source.codeaurora.org/external/imx/imx-manifest -b imx-linux-sumo -m imx-4.14.98-2.3.3.xml
### IMPORTANT ### $ git config --global url."https://".insteadOf git://
$ repo sync
$ sudo rm /usr/bin/python
$ sudo ln -s /usr/bin/python2 /usr/bin/python
- configure machine
$ DISTRO=fsl-imx-fb MACHINE=imx6solosabresd source fsl-setup-release.sh -b rsr1296
- create shared directory
$ mkdir ~/yocto
$ mkdir ~/yocto/download
$ mkdir ~/yocto/sstate-cache
$ gedit conf/local.conf
and add this lines:
DL_DIR="/home/multi/yocto/download"
SSTATE_DIR="/home/multi/yocto/sstate-cache"
CONNECTIVITY_CHECK_URIS ?= "https://www.google.com"
IMAGE_INSTALL_append = "pcsc-lite openssl-bin libcrypto"
PREFERRED_VERSION_openssl = "1.1.1j"
IMAGE_INSTALL_remove += "packagegroup-fsl-optee-imx"
BAD_RECOMMENDATIONS += "udev-hwdb"
comment out this lines
#PACKAGECONFIG_append_pn-qemu-native = " sdl"
#PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"
- update Open SSL to 1.1.1J (http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-connectivity/openssl?h=master)
$ cd ~/fsl-4-14-98/sources/poky/meta/recipes-connectivity
$ mv openssl /home/multi/Documents
$ tar xvzf /home/multi/Documents/openssl_1.1.1j.tar.gz
$ cd -
- compile full system
$ bitbake core-image-minimal
Everything is fine with no errors in log until bitbake core-image-minimal

Installation process for OpenMapTiles server without docker

Is there a way to install OpenMapTiles server without docker? I need to use this on redhat linux and docker needs to be enterprise version in order to use it on redhat. Please let me know.
Thanks
You just need to manually perform all scripts they run inside their separate docker containers.
They set up 1 database server by running the commands in
https://github.com/openmaptiles/openmaptiles-tools/blob/master/docker/postgis/Dockerfile
and
https://github.com/openmaptiles/openmaptiles-tools/blob/master/docker/postgis/initdb-postgis.sh
And then continue to download data in a few different docker files by running some commands, this is a pattern that comes back again and again, run the commands in the Dockerfile and the scripts in for all these subfolders in https://github.com/openmaptiles/openmaptiles-tools/tree/master/docker in the order they appear in the documentation at
https://github.com/openmaptiles/openmaptiles/blob/master/README.md
If you're on ubuntu this should be pretty straightforward.
I don't have access to a redhat linux instance, but after translating those ubuntu commands I got something that worked on centos7, so should work on your RHEL7:
(This needs some serious cleanup, I do not recommend using this in a nice production system. Someone should package these commands up in rpm's and push them to a repository (I didn't have the time at the moment and I'm not sure if someone would actually want to do this, let me know if you would be interested in having rpm's of these tools))
# install dependencies
# Install PostgreSQL and PostGIS
yum -y install epel-release
rpm -ivh https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-centos11-11-2.noarch.rpm
yum install postgis30_11 postgresql11-server postgis30_11-client
yum install postgresql11-devel postgis30_11-docs postgis30_11-utils pgrouting_11
# tools needed later
yum install boost169-devel libffi-devel openssl-devel protobuf-lite-devel sparsehash-devel leveldb-devel golang-bin utf8proc-devel sqlite pandoc lbzip2 vim libpng libtiff libjpeg freetype gdal cairo pycairo sqlite geos boost curl libcurl libicu bzip2-devel libpng-devel libtiff-devel zlib-devel libjpeg-devel libxml2-devel python-setuptools proj-devel proj proj-epsg proj-nad freetype-devel libicu-devel gdal-devel sqlite-devel libcurl-devel cairo-devel pycairo-devel geos-devel protobuf-devel protobuf-c-devel lua-devel cmake proj boost-thread proj-devel autoconf automake libtool pkgconfig ragel gtk-doc glib2 glib2-devel libpng libpng-devel libwebp libtool-ltdl-devel python-devel harfbuzz harfbuzz-devel harfbuzz-icu boost-devel cabextract xorg-x11-font-utils fontconfig perl-DBD-Pg mesa-libGLU-devel graphviz sqlite3 aria2 osmctools python3 wget
# GCC++ 14 standards are required for Mapnik so we shall install the Dev Toolset from the CentOS Software Collections
yum install centos-release-scl
yum install devtoolset-6
scl enable devtoolset-6 bash
export JOBS=$(nproc)
# Initialise PostgreSQL and Basic Setup
/usr/pgsql-11/bin/postgresql-11-setup initdb
systemctl enable postgresql-11.service
cd /var/lib/pgsql/11
vim data/postgresql.conf
# Add the IP addresses on which the server should listen for connections
listen_addresses = 'localhost,192.168.1.1'
systemctl start postgresql-11.service
vim /etc/profile.d/pgsql.sh
$ export PATH=$PATH:/usr/pgsql-11/bin:/usr/pgsql-11/lib:/usr/local/lib
source /etc/profile.d/pgsql.sh
git clone https://github.com/loretoparisi/kakasi.git
cd kakasi
./configure && make
make install
vim /etc/ld.so.conf.d/libkakasi.conf
/usr/lib64
/usr/local/lib
ldconfig
cd ..
git clone https://github.com/openmaptiles/mapnik-german-l10n.git
cd mapnik-german-l10n
make
make install
su - postgres
psql --dbname="openmaptiles" <<-'EOSQL'
CREATE DATABASE template_postgis;
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
EOSQL
# i don't find this step anywhere, but it is needed
psql
CREATE DATABASE openmaptiles;
for db in template_postgis "openmaptiles"; do
psql --dbname="$db" <<-'EOSQL'
CREATE EXTENSION postgis;
CREATE EXTENSION hstore;
CREATE EXTENSION unaccent;
CREATE EXTENSION fuzzystrmatch;
CREATE EXTENSION osml10n;
EOSQL
done
# these 3 commands start a docker container that downlaods data
#make import-water
#make import-natural-earth
#make import-lakelines
cd
mkdir data
cd data
wget --quiet http://osmdata.openstreetmap.de/download/water-polygons-split-3857.zip
unzip -oj water-polygons-split-3857.zip
su - postgres
ogr2ogr -progress -f Postgresql -s_srs EPSG:3857 -t_srs EPSG:3857 -lco OVERWRITE=YES -lco GEOMETRY_NAME=geometry -nln "osm_ocean_polygon" -nlt geometry --config PG_USE_COPY YES PG:"dbname=openmaptiles" "/root/data/water_polygons.shp"
wget --quiet http://naciscdn.org/naturalearth/packages/natural_earth_vector.sqlite.zip
unzip -oj natural_earth_vector.sqlite.zip -d . '*natural_earth_vector.sqlite'
wget https://raw.githubusercontent.com/openmaptiles/openmaptiles-tools/master/docker/import-natural-earth/clean-natural-earth.sh
hmod +x clean-natural-earth.sh
NATURAL_EARTH_DB=./natural_earth_vector.sqlite ./clean-natural-earth.sh
ogr2ogr -progress -f Postgresql -s_srs EPSG:4326 -t_srs EPSG:3857 -clipsrc -180.1 -85.0511 180.1 85.0511 -lco GEOMETRY_NAME=geometry -lco OVERWRITE=YES -lco DIM=2 -nlt GEOMETRY -overwrite PG:"dbname=openmaptiles" "natural_earth_vector.sqlite"
wget https://github.com/lukasmartinelli/osm-lakelines/releases/download/v0.9/lake_centerline.geojson
ogr2ogr -progress -f Postgresql -s_srs EPSG:4326 -t_srs EPSG:3857 -lco OVERWRITE=YES -overwrite -nln "lake_centerline" PG:"dbname=openmaptiles" "lake_centerline.geojson"
quit
cd data
#get some pbf
wget https://download.geofabrik.de/europe/belgium-latest.osm.pbf
# make import-osm
export GOPATH=~/.go
mkdir -p $GOPATH/src/github.com/omniscale/imposm3
#export IMPOSM_REPO="https://github.com/openmaptiles/imposm3.git"
export IMPOSM_REPO="https://github.com/omniscale/imposm3.git"
#export IMPOSM_VERSION="v2017-10-18"
export IMPOSM_VERSION="v0.8.1"
cd $GOPATH/src/github.com/omniscale/imposm3
go get github.com/tools/godep
go get -u github.com/golang/protobuf/protoc-gen-go
git clone --quiet --depth 1 $IMPOSM_REPO -b $IMPOSM_VERSION $GOPATH/src/github.com/omniscale/imposm3
make build
/usr/local/bin/generate-imposm3 /root/openmaptiles/openmaptiles.yaml > mapping.yaml
export DIFF_DIR=~/data/import
mkdir $DIFF_DIR
export IMPOSM_CACHE_DIR=/tmp/cache
mkdir $IMPOSM_CACHE_DIR
wget https://raw.githubusercontent.com/openmaptiles/openmaptiles-tools/master/docker/import-osm/config.json
./imposm import -connection "postgis://postgres#localhost/openmaptiles" -mapping mapping.yaml -overwritecache -diffdir "$DIFF_DIR" -cachedir "$IMPOSM_CACHE_DIR" -read "$pbf_file" -deployproduction -write $diff_flag -config "$CONFIG_JSON"
# make import-borders
cd
git clone https://github.com/mapbox/protozero
cd protozero
mkdir build
cd build
cmake ..
make -j ${JOBS}
make install
cd
git clone https://github.com/osmcode/libosmium.git
cd libosmium
mkdir build
cmake ..
make -j ${JOBS}
make install
cd
git clone https://github.com/pnorman/osmborder.git
cd osmborder
mkdir build
cmake ..
make -j ${JOBS}
make install
cd
git clone https://github.com/openmaptiles/openmaptiles-tools.git
cd openmaptiles-tools/bin
export PGHOST=localhost
export PGDATABASE=openmaptiles
export PGUSER=postgres
export PGPASSWORD=
./import-borders ~/data/belgium-latest.osm.pbf
# make import-wikidata
cd
wget https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tgz
tar xf Python-3.8.2
cd Python-3.8.2
./configure --enable-optimizations
make -j ${JOBS} altinstall
cd ~/openmaptiles-tools/bin
python3.8 -m pip install --upgrade pip
python3.8 -m pip install -r ../requirements.txt
export PYTHONPATH=$PYTHONPATH:$PWD/../
# workaround for asyncpg.exceptions.UndefinedTableError: relation "wd_names" does not exist
# see https://github.com/openmaptiles/openmaptiles/pull/785
su - postgres
psql openmaptiles
$ CREATE TABLE IF NOT EXISTS wd_names (id varchar(20) UNIQUE, page varchar(200) UNIQUE, labels hstore);
$ TRUNCATE wd_names;
$ quit
quit
python3.8 import-wikidata --user=postgres ../../openmaptiles/openmaptiles.yaml
#make
# openmaptiles-tools generate-tm2source openmaptiles.yaml
python3.8 generate-tm2source ../../openmaptiles/openmaptiles.yaml --port 5432 > tm2source.yaml
# openmaptiles-tools generate-sql openmaptiles.yaml
python3.8 generate-sql ../../openmaptiles/openmaptiles.yaml > openmaptiles.sql
#make import-sql
# openmaptiles-tools import-sql
export PSQL_OPTIONS=-a
export OMT_UTIL_DIR=../sql/
export VT_UTIL_DIR=../vt_util_sql/
mkdir $VT_UTIL_DIR
wget https://raw.githubusercontent.com/openmaptiles/postgis-vt-util/master/postgis-vt-util.sql
mv postgis-vt-util.sql ../vt_util_sql/
export SQL_DIR=$PWD
./import-sql
# make generate-tiles
- generate-vectortiles
yum install nodejs xdg-utils
cd
git clone git://github.com/mapnik/mapnik
cd mapnik
git checkout remotes/origin/v3.0.x
./bootstrap.sh
./configure BOOST_INCLUDES=/usr/include/boost169/ BOOST_LIBS=/usr/lib64/boost169/
git submodule sync
git submodule update --init
make -j ${JOBS}
make install
ldconfig
npm install --build-from-source=mapnik -g #mapbox/tiletype mapnik#3.7.2 #mapbox/mbtiles #mapbox/tilelive tilelive-tmsource #mapbox/tilelive-vector tilelive-bridge tilelive-mapnik
cp tm2source.yaml data.yml
sed -i "s|host: .*|host: \"localhost\"|g" data.yml
sed -i "s|port: .*|port: \"5432\"|g" data.yml
sed -i "s|dbname: .*|dbname: \"openmaptiles\"|g" data.yml
sed -i "s|user: .*|user: \"postgres\"|g" data.yml
sed -i "s|password: .*|password: \"$POSTGRES_HOST\"|g" data.yml
export BBOX="-180,-85.0511,180,85.0511"
export MIN_ZOOM=0
export MAX_ZOOM=14
tilelive-copy --scheme=pyramid --bounds=BBOX --timeout="18000000" --concurrency="10"--minzoom=MIN_ZOOM --maxzoom=MAX_ZOOM "tmsource://$PWD" "mbtiles://root/data/tiles.mbtiles"
generate-metadata ~/data/tiles.mbtiles

Project Open Dockerfile postgres not starting

I'm trying to use the Dockerfile provided here and on building docker build . I get the error on line 69 (RUN createuser -s projop) which reports:
Step 15/27 : RUN /usr/bin/pg_ctl -D "/var/lib/pgsql/data" start
---> Using cache
---> ce049ebe4ff5
Step 16/27 : RUN sleep 60
---> Using cache
---> bf7bac638da6
Step 17/27 : RUN createuser -s projop # database user "projop" with admin rights
---> Running in 700e6e618060
createuser: could not connect to database postgres: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
My inital thought was the the postgres server had not had enough time to start up from the lines before so I extended the sleep command up to 60 seconds, but I still got this same error.
I also opened a bash session with a intermediate container after the sleep with docker run -it bf7bac638da6 bash. Within this bash session I tried to manually run the createuser -s projop line which gave the same error.
However, if I reran the command /usr/bin/pg_ctl -D "/var/lib/pgsql/data" start and then createuser -s projop it would work.
This is quite od as it seems the initial start command is not taking effect. Any ideas what might be happening here?
Here's the full Dockerfile:
#
# Dockerfile for ]project-open[ V5.0 on CentOS 7
#
FROM centos:centos7
# ----------------------------------------------------------------------------------------------
# Install base packages
# ----------------------------------------------------------------------------------------------
RUN yum -y install wget net-tools setools
#RUN yum -y install cvs expat expat-devel pango graphviz-devel ImageMagick openldap-clients mlocate sharutils
# Install Perl libraries
#RUN yum -y install graphviz-perl perl perl-Archive-Tar perl-Archive-Zip perl-CGI perl-CGI-Session
#RUN yum -y install perl-CPAN perl-CPAN-Changes perl-CPAN-Meta perl-CPAN-Meta-Requirements perl-CPAN-Meta-YAML
#RUN yum -y install perl-Carp perl-Compress-Raw-Bzip2 perl-Crypt-DES perl-Crypt-OpenSSL-RSA
#RUN yum -y install perl-Crypt-OpenSSL-Random perl-Crypt-PasswdMD5 perl-Crypt-SSLeay perl-DBD-Pg
#RUN yum -y install perl-DBD-Pg-tests perl-DBI perl-Data-Dumper perl-DateTime perl-Digest-MD5
#RUN yum -y install perl-Encode perl-File-Slurp perl-GSSAPI perl-IO-Socket-IP perl-IO-Socket-SSL
#RUN yum -y install perl-JSON perl-LDAP perl-LWP-MediaTypes perl-LWP-Protocol-https perl-Net-DNS
#RUN yum -y install perl-Net-HTTP perl-Net-SSLeay perl-Params-Check perl-Params-Util perl-Params-Validate
#RUN yum -y install perl-Socket perl-TimeDate perl-WWW-Curl perl-YAML perl-core perl-devel perl-gettext
#RUN yum -y install perl-libs perl-libwww-perl rrdtool-perl perl-YAML
#RUN yum -y install libdbi-dbd-pgsql
# Install OpenOffice
#RUN yum -y install libreoffice libreoffice-headless
# ----------------------------------------------------------------------------------------------
# Download ]po[ distro files
# ----------------------------------------------------------------------------------------------
WORKDIR /usr/src/
RUN wget -q http://sourceforge.net/projects/project-open/files/project-open/Support%20Files/naviserver-4.99.8.tgz &&\
wget -q http://sourceforge.net/projects/project-open/files/project-open/Support%20Files/web_projop-aux-files.5.0.0.0.0.tgz &&\
wget -q http://sourceforge.net/projects/project-open/files/project-open/V5.0/update/project-open-Update-5.0.2.4.0.tgz
# ----------------------------------------------------------------------------------------------
# Create user projop and unpack ]po[ files an
# ----------------------------------------------------------------------------------------------
WORKDIR /usr/local
RUN tar xzf /usr/src/naviserver-4.99.8.tgz # extract the NaviServer binary 64 bit
RUN mkdir /web/ # super-directory for all Web servers /web/ by default
RUN groupadd projop # create a group called "projop"
RUN useradd -d /web/projop -g projop projop # create user "projop" with home directory /web/projop
# RUN chown -R projop:projop /web/projop # set ownership to all files
# ----------------------------------------------------------------------------------------------
# Install PostgreSQL
# ----------------------------------------------------------------------------------------------
RUN yum -y install postgresql postgresql-server postgresql-contrib
# Run the rest of the commands as user postgres
USER postgres
RUN /usr/bin/pg_ctl -D "/var/lib/pgsql/data" initdb
RUN echo "host all all 0.0.0.0/0 md5" >> /var/lib/pgsql/data/pg_hba.conf
RUN echo "listen_addresses='*'" >> /var/lib/pgsql/data/postgresql.conf
RUN /usr/bin/pg_ctl -D "/var/lib/pgsql/data" start
RUN sleep 60
RUN createuser -s projop # database user "projop" with admin rights
# ----------------------------------------------------------------------------------------------
# Setup the /web/projop folder
# ----------------------------------------------------------------------------------------------
USER projop
WORKDIR /web/projop/
RUN tar xzf /usr/src/web_projop-aux-files.5.0.0.0.0.tgz # extract auxillary files
RUN tar xzf /usr/src/project-open-Update-5.0.2.4.0.tgz # extract the ]po[ product source code - latest
RUN createdb --encoding=utf8 --owner=projop projop # new database
# RUN createlang plpgsql projop # enable PlPg/SQL, may already be installed
WORKDIR /web/projop
RUN psql -f /web/projop/pg_dump.5.0.2.4.0.sql > /web/projop/import.log 2>&1
# Expose the ]p[ and PostgreSQL port
EXPOSE 8000 5432
# Add VOLUMEs to allow backup of config, logs and databases
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
# Set the default command to run when starting the container
CMD ["/usr/lib/postgresql/9.3/bin/postgres", "-D", "/var/lib/postgresql/9.3/main", "-c", "config_file=/etc/postgresql/9.3/main/postgresql.conf"]

rails 4 + mina deployment failure

i am deploying a rails 4 application using mina deployment. my deployment script is as
require 'mina/bundler'
require 'mina/rails'
require 'mina/git'
require 'mina/rvm' # for rvm support. (http://rvm.io)
set :domain, 'someplace.com'
set :deploy_to, '/home/deploy/projects/website'
set :repository, 'git#github.com:someone/repo.git'
set :branch, 'master'
set :identity_file, "#{ENV['HOME']}/.ssh/id_rsa"
set :user, 'deploy' # Username in the server to SSH to.
set :shared_paths, ['config/database.yml', 'config/credentials.yml', 'log', 'tmp']
task :environment do
invoke :'rvm:use[ruby-2.1.0#default]'
end
task :setup => :environment do
queue! %[mkdir -p "#{deploy_to}/shared/log"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/log"]
queue! %[mkdir -p "#{deploy_to}/shared/config"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/config"]
queue! %[touch "#{deploy_to}/shared/config/database.yml"]
queue %[echo "-----> Be sure to edit 'shared/config/database.yml'."]
queue! %[touch "#{deploy_to}/shared/config/credentials.yml"]
queue %[echo "-----> Be sure to edit 'shared/config/credentials.yml'."]
end
desc "Deploys the current version to the server."
task :deploy => :environment do
deploy do
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
invoke :'rails:assets_precompile'
to :launch do
queue "touch #{deploy_to}/tmp/restart.txt"
end
end
end
when i deploy as 'mina deploy', i get error as
...
Symlinking shared paths
$ mkdir -p "./config"
$ mkdir -p "."
$ rm -rf "./config/database.yml"
$ ln -s "/home/deploy/projects/website/shared/config/database.yml" "./config/database.yml"
$ rm -rf "./config/credentials.yml"
$ ln -s "/home/deploy/projects/website/shared/config/credentials.yml" "./config/credentials.yml"
$ rm -rf "./log"
$ ln -s "/home/deploy/projects/website/shared/log" "./log"
$ rm -rf "./tmp"
$ ln -s "/home/deploy/projects/website/shared/tmp" "./tmp"
-----> Installing gem dependencies using Bundler
$ mkdir -p "/home/deploy/projects/website/shared/bundle"
$ mkdir -p "./vendor"
$ ln -s "/home/deploy/projects/website/shared/bundle" "./vendor/bundle"
$ bundle install --without development:test --path "./vendor/bundle" --binstubs bin/ --deployment
...
Your bundle is complete!
Gems in the groups development and test were not installed.
It was installed into ./vendor/bundle
-----> Precompiling asset files
$ RAILS_ENV="production" bundle exec rake assets:precompile RAILS_GROUPS=assets
rake aborted!
File exists # dir_s_mkdir - /home/deploy/projects/website/tmp/build-138935597031149/tmp
/home/deploy/projects/website/tmp/build-138935597031149/vendor/bundle/ruby/2.1.0/gems/sprockets-2.10.1/lib/sprockets/cache/file_store.rb:25:in `[]='
/home/deploy/projects/website/tmp/build-138935597031149/vendor/bundle/ruby/2.1.0/gems/sprockets-2.10.1/lib/sprockets/caching.rb:34:in `cache_set'
Make sure that shared/tmp directory is created if not ssh into the server and
$ mkdir /home/deploy/projects/website/shared/tmp
make sure you have the right permissions too.
drwxr-xr-x
If you are symlinking your tmp directory ensure that it exists. If it doesn't the broken symlink will cause the same issue.