Two docker images into a single docker image [duplicate] - mongodb

This question already has answers here:
Docker: Combine multiple images
(4 answers)
Closed 2 years ago.
I have a requirement to create a single docker image by merging both "RabbitMQ-management" and "MongoDB" images. Is it possible to do it?
I've tried with below Dockerfile and can able to access RabbitMQ, but mongodb is not working. Could you please help me.
# cat Dockerfile
FROM rabbitmq:3.7.17
RUN rabbitmq-plugins enable --offline rabbitmq_management
# extract "rabbitmqadmin" from inside the "rabbitmq_management-X.Y.Z.ez" plugin zipfile
# see https://github.com/docker-library/rabbitmq/issues/207
RUN apt-get update && apt-get install -y gnupg2
RUN gpg --keyserver ha.pool.sks-keyservers.net --recv-keys 0C49F3730359A14518585931BC711F9BA15703C6 && \
gpg --export $GPG_KEYS > /etc/apt/trusted.gpg.d/mongodb.gpg
ARG MONGO_PACKAGE=mongodb-org
ARG MONGO_REPO=repo.mongodb.org
ENV MONGO_PACKAGE=${MONGO_PACKAGE} MONGO_REPO=${MONGO_REPO}
ENV MONGO_MAJOR 3.4
ENV MONGO_VERSION 3.4.18
RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list"
RUN echo "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list"
RUN apt-get update
RUN apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION
RUN set -eux; \
erl -noinput -eval ' \
{ ok, AdminBin } = zip:foldl(fun(FileInArchive, GetInfo, GetBin, Acc) -> \
case Acc of \
"" -> \
case lists:suffix("/rabbitmqadmin", FileInArchive) of \
true -> GetBin(); \
false -> Acc \
end; \
_ -> Acc \
end \
end, "", init:get_plain_arguments()), \
io:format("~s", [ AdminBin ]), \
init:stop(). \
' -- /plugins/rabbitmq_management-*.ez > /usr/local/bin/rabbitmqadmin; \
[ -s /usr/local/bin/rabbitmqadmin ]; \
chmod +x /usr/local/bin/rabbitmqadmin; \
apt-get update; apt-get install -y --no-install-recommends python ca-certificates; rm -rf /var/lib/apt/lists/*; \
rabbitmqadmin --version
EXPOSE 15671 15672 27017
I'm using below command to run RabbitMQ (it is working)
# docker run -it -p 15672:15672 -p 5672:5672 --hostname my-rabbitmq image
Command to start Mongodb container (it is not working)
docker run -d -v /tmp/mongodb:/data/db -p 27017:27017 image mongod

It is not impossible to achieve what you are asking, but I would say it's not the correct way of doing it. Docker provides isolation, and states that it should be one process per container. (should, not must).
If you follow this concept, it's easy to use other tools to bring up multiple containers together. For example, instead of complicated Dockerfile that merge two images you can specify a nice docker-compose.yml:
version: '3'
services:
mongo:
image: mongod
volumes:
- /tmp/mongodb:/data/db
ports:
- "27017:27017"
rabbitmq:
image: rabbitmq
ports:
- "15672:15672"
- "5672:5672"
depends_on:
- mongo
Which is more readable IMO, and probably achieve the same outcome.

Related

No existing local cluster is suitable error using pg_dump in docker

In my docker project with postgres 9.6 I need to add support for uploading of big sql dumps.
I added pg_dump by adding in my web/Dockerfile.yml file line:
postgresql-client-common \
as it contains pg_dump.
After building the app I enter bash and
# uname -a
Linux ff5146f21dbd 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux
root#ff5146f21dbd:/var/www/lprods_docker_root# pg_dump
Warning: No existing local cluster is suitable as a default target. Please see man pg_wrapper(1) how to specify one.
Error: You must install at least one postgresql-client-<version> package
root#ff5146f21dbd:/var/www/lprods_docker_root# locate -i pg_dump
Last command outputs nothing
web/Dockerfile.yml contains :
FROM php:7.1-apache
RUN apt-get update && \
apt-get install -y \
python \
libfreetype6-dev \
libwebp-dev \
libjpeg62-turbo-dev \
libpng-dev \
libzip-dev \
nano \
git-core \
curl \
build-essential \
openssl \
libssl-dev \
libgmp-dev \
libldap2-dev \
libpq-dev \
netcat \
postgresql-client-common \
locate \
&& git clone https://github.com/nodejs/node.git \
&& cd node \
&& git checkout v12.0.0 \
&& ./configure \
&& make \
&& make install
RUN npm install cross-env
RUN docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-webp-dir=/usr/include/ --with-jpeg-dir=/usr/include/
RUN docker-php-ext-install gd pgsql pdo_pgsql zip gmp bcmath pcntl ldap sysvmsg exif \
&& a2enmod rewrite
COPY virtualhost.conf /etc/apache2/sites-enabled/000-default.conf
and docker-compose.yml:
version: '3'
services:
web:
build:
context: ./web # directory of web/Dockerfile.yml
dockerfile: Dockerfile.yml
environment:
- APACHE_RUN_USER=#1000
# - APACHE_RUN_USER=www-data
container_name: lprods_web
volumes:
- ${APP_PATH_HOST}:${APP_PTH_CONTAINER}
ports:
- "8086:80"
working_dir: ${APP_PTH_CONTAINER}
db:
image: postgres:9.6.10-alpine
container_name: lprods_db
ports:
- '5433:5432'
restart: always
environment:
POSTGRES_USER: 'postgres'
POSTGRES_PASSWORD: '1'
POSTGRES_DB: 'wprods'
volumes:
- ./init:/docker-entrypoint-initdb.d/
phppgadmin:
image: dockage/phppgadmin:latest
environment:
- PHP_PG_ADMIN_SERVER_HOST=db
- PHP_PG_ADMIN_SERVER_PORT=5432
- PHP_PG_ADMIN_SERVER_DEFAULT_DB=postgres
container_name: lprods_phppgadmin
restart: always
ports:
- "8087:80"
- "443:443"
links:
- db
composer:
image: composer:1.6
container_name: lprods_composer
volumes:
- ${APP_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install --ignore-platform-reqs
Why error and how to fix it?
MODIFIED :
I have
# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
But checking version I got error :
# /usr/bin/pg_dump -v
Warning: No existing local cluster is suitable as a default target. Please see man pg_wrapper(1) how to specify one.
Error: You must install at least one postgresql-client-<version> package
The same error I got when running psql command(actually I need it):
/usr/bin/psql -h localhost -d wprods -U postgres -W -f "/var/www/lprods_docker_root/wprods_2017_10_23.sql"
Error: You must install at least one postgresql-client-<version> package
What is wrong with my configuration ?
MODIFIED 2:
In my docker-compose.yml I have :
db:
image: postgres:9.6.10-alpine
...
volumes:
- ./init:/docker-entrypoint-initdb.d/
Is it?
command :
locate -i postgresql.conf
outputs nothing
find /usr -name "postgresql.conf"
Also outputs nothing.
Also I have :
# ps -ef | grep postgres
root 28 21 0 08:40 pts/0 00:00:00 grep postgres
What did I miss ?
postgresql-client-common is a Ubuntu-specific package of scripts that allows users to work with multiple versions of Postgres (or multiple database clusters). It provies several scripts and symlinks that essentially override the functions of "real" postgres utilities (like pg_dump):
root#foo:/usr/bin# ls -al | grep pg_dump
lrwxrwxrwx 1 root root 37 Feb 8 2018 pg_dump -> ../share/postgresql-common/pg_wrapper
lrwxrwxrwx 1 root root 37 Feb 8 2018 pg_dumpall -> ../share/postgresql-common/pg_wrapper
You may need to also install postgresql-client-10 to make pg_dump work.
Note also that you have only installed the client tools. If you need to get a fully-functional PostgreSQL database up and running, I believe you will also need to install the postgresql-10 package

pg_dump server and pg_dump version mismatch in docker

When I run the command psql --version within the railsApp container, I get 9.4.12 and when I run the same within the postgres container, I get 9.6.2. How can I get the versions to match?
I am getting the following error when I try to do a migration on Rails App which does a pg_dump sql import.
pg_dump: server version: 9.6.2; pg_dump version: 9.4.12
pg_dump: aborting because of server version mismatch
rails aborted!
Here's my Docker-compose.yml file:
version: "2.1"
services:
railsApp:
build:
context: ./
ports:
- "3000:3000"
links:
- postgres
volumes:
- .:/app
postgres:
image: postgres:9.6
ports:
- "5432:5432"
volumes:
- ./.postgres:/var/lib/postgresql
The Dockerfile:
FROM ruby:2.3.3
# setup /app as our working directory
RUN mkdir /app
WORKDIR /app
# Replace shell with bash so we can source files
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
# Set debconf to run non-interactively
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
# Install base dependencies
FROM ruby:2.3.3
# setup /app as our working directory
RUN mkdir /app
WORKDIR /app
# Replace shell with bash so we can source files
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
# Set debconf to run non-interactively
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
# Install base dependencies
RUN apt-get update && apt-get install -y -q --no-install-recommends \
apt-transport-https \
build-essential \
ca-certificates \
curl \
git \
libssl-dev \
python \
rsync \
software-properties-common \
wget \
postgresql-client \
graphicsmagick \
&& rm -rf /var/lib/apt/lists/*
# Install node and npm with nvm
RUN curl https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash
ENV NVM_DIR=/root/.nvm
ENV NODE_VERSION v7.2.1
ENV NODE_PATH $NVM_DIR/versions/node/$NODE_VERSION
ENV PATH $NODE_PATH/bin:./node_modules/.bin:$PATH
RUN source $NVM_DIR/nvm.sh \
&& nvm install $NODE_VERSION \
&& nvm alias default $NODE_VERSION \
&& nvm use default
# Install our ruby dependencies
ADD Gemfile Gemfile.lock /app/
RUN bundle install
# copy the rest of our code over
ADD . /app
ENV RAILS_ENV development
ENV SECRET_KEY_BASE a6bdc5f788624f00b68ff82456d94bf81bb50c2e114b2be19af2e6a9b76f9307b11d05af4093395b0471c4141b3cd638356f888e90080f8ae60710f992beba8f
# Expose port 3000 to the Docker host, so we can access it from the outside.
EXPOSE 3000
# Set the default command to run our server on port 3000
CMD ["rails", "server", "-p", "3000", "-b", "0.0.0.0"]
The same issue for me, I used an alternative way to take a dump,
First I access the terminal and run the pd_dump inside docker and copied the file from docker to host.
Below are the commands
docker exec -it <container-id> /bin/bash # accessing docker terminal
pg_dump > ~/dump # taking dump inside the docker
docker cp <container-id>:/root/dump ~/dump #copying the dump files to host
Hope the above solution helps.
The easiest approach is to use the correct postgres version in the docker-compose. Change:
postgres:
image: postgres:9.6
To:
postgres:
image: postgres:9.4.2
All available versions here.

Where is the Postgres username/password being created in this Dockerfile?

So I was following this tutorial:
https://realpython.com/blog/python/django-development-with-docker-compose-and-machine/
I have everything up and running, however theres a few things going on that I'm not able to follow or understand.
In the main Docker-Compose we have:
web:
restart: always
build: ./web
expose:
- "8000"
links:
- postgres:postgres
- redis:redis
volumes:
- /usr/src/app
- /usr/src/app/static
env_file: .env
command: /usr/local/bin/gunicorn docker_django.wsgi:application -w 2 -b :8000
postgres:
restart: always
image: postgres:latest
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data/
You will notice there is a env_file containing:
DB_NAME=postgres
DB_USER=postgres
DB_PASS=postgres
My question is when is the postgres user and password being set? If I run this docker-compose everything works, meaning the web app can pass credentials to the postgress database and establish a connection. I'm not able to follow however, where those credentials are being set in the first place.
I was assuming in the base postgres Dockerfile, there would be some instruction to set the database name, username and password. I do not it though. Here is a copy of the base postgres Dockerfile below.
# vim:set ft=dockerfile:
FROM debian:jessie
# explicitly set user/group IDs
RUN groupadd -r postgres --gid=999 && useradd -r -g postgres --uid=999 postgres
# grab gosu for easy step-down from root
ENV GOSU_VERSION 1.7
RUN set -x \
&& apt-get update && apt-get install -y --no-install-recommends ca-certificates wget && rm -rf /var/lib/apt/lists/* \
&& wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \
&& wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc" \
&& export GNUPGHOME="$(mktemp -d)" \
&& gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
&& gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu \
&& rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu \
&& gosu nobody true \
&& apt-get purge -y --auto-remove ca-certificates wget
# make the "en_US.UTF-8" locale so postgres will be utf-8 enabled by default
RUN apt-get update && apt-get install -y locales && rm -rf /var/lib/apt/lists/* \
&& localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
ENV LANG en_US.utf8
RUN mkdir /docker-entrypoint-initdb.d
RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8
ENV PG_MAJOR 9.6
ENV PG_VERSION 9.6.1-1.pgdg80+1
RUN echo 'deb http://apt.postgresql.org/pub/repos/apt/ jessie-pgdg main' $PG_MAJOR > /etc/apt/sources.list.d/pgdg.list
RUN apt-get update \
&& apt-get install -y postgresql-common \
&& sed -ri 's/#(create_main_cluster) .*$/\1 = false/' /etc/postgresql-common/createcluster.conf \
&& apt-get install -y \
postgresql-$PG_MAJOR=$PG_VERSION \
postgresql-contrib-$PG_MAJOR=$PG_VERSION \
&& rm -rf /var/lib/apt/lists/*
# make the sample config easier to munge (and "correct by default")
RUN mv -v /usr/share/postgresql/$PG_MAJOR/postgresql.conf.sample /usr/share/postgresql/ \
&& ln -sv ../postgresql.conf.sample /usr/share/postgresql/$PG_MAJOR/ \
&& sed -ri "s!^#?(listen_addresses)\s*=\s*\S+.*!\1 = '*'!" /usr/share/postgresql/postgresql.conf.sample
RUN mkdir -p /var/run/postgresql && chown -R postgres /var/run/postgresql
ENV PATH /usr/lib/postgresql/$PG_MAJOR/bin:$PATH
ENV PGDATA /var/lib/postgresql/data
VOLUME /var/lib/postgresql/data
COPY docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 5432
CMD ["postgres"]
Not sure which postgres image are you using.
If you look at the official postgres image for complete information. It allows user to specify the environment variables for the below ones and these can be easily overridden at run time.
POSTGRES_PASSWORD
POSTGRES_USER
PGDATA
POSTGRES_DB
POSTGRES_INITDB_ARGS
Environment variables can be overridden using below three methods depending on your case.
Run Image: If running docker image directly, use below to include environment variable in docker run with -e K=V. Please refer documentation for more details here
docker run -e POSTGRES_PASSWORD=secrect -e POSTGRES_USER=postgres <other options> image/name
Dockerfile: If you need to specify the environment variable in Dockerfile, specify as mentioned below. Please refer documentation for more details here
ENV POSTGRES_PASSWORD=secrect
ENV POSTGRES_USER=postgres
docker-compose: If you need to specify the environment variable in docker-compose.yml, specify as below. Please refer documentation for more details here
web:
environment:
- POSTGRES_PASSWORD=secrect
- POSTGRES_USER=postgres
Hope this is useful.
I think the postgres user and password is being set on entrypoint, like in line 23 on official image entrypoint.
https://github.com/docker-library/postgres/blob/e4942cb0f79b61024963dc0ac196375b26fa60dd/9.6/docker-entrypoint.sh
Can you check your entrypoint?
All you need is
docker run -e POSTGRES_PASSWORD=123456 postgres
In the same way you can set up the username like below:
docker run -e POSTGRES_USERNAME=xyz postgres
Are you using the -v mounted folder? If so, you need to remove the entire pgdata/ folder you created previously so that the postgres can re-create itself using the new environment variables.

Mapping a Volume in a docker

I am trying to map a Volume to the docker image using -v "/Users//data:/data/"
> docker run -p 27017:27017 --name mongo2_001 -d mongo -v "/Users/<user>/data:/data"
6cf25618d...
> docker ps -a
CONTAINER ID IMAGE COMMAND STATUS PORTS NAMES
6cf25618d.. mongo "/usr/bin/mongod -v /" Exited (1) mongoMG_001
It fails with the following error
> docker logs 6cf25618d..
2016-08-30T01:35:28.197+0000 F CONTROL [main] Failed global
initialization: BadValue: The "verbose" option string cannot
contain any characters other than "v"
When ran with out "-v ..." seems to work properly.
Not sure what could be wrong.
PS: Dockerfile used to create the image used above
FROM ubuntu:16.04
MAINTAINER Docker
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
RUN echo "deb http://repo.mongodb.org/apt/ubuntu $(cat /etc/lsb-release | grep DISTRIB_CODENAME | cut -d= -f2)/mongodb-org/3.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.2.list
RUN apt-get update && apt-get install -y mongodb-org
RUN mkdir -p /data/db
WORKDIR /data
EXPOSE 27017
ENTRYPOINT ["/usr/bin/mongod"]
The order of "-v" parameter passed was incorrect.
Instead of going to docker command "-v" options it going to mongo container. Hence reorder the command as shown below
docker run -p 27018:27017 -v /Users/<user>/data:/data --name mongo2_001 -d mongo
On windows
docker run -p 27017:27017 -v /c/Users/<user>/data:/data --name mongo2_001 -d mongo
Mike Tung answer was not working for me. I was able to make it work with following command
docker run --name mongodb_win -v mongoWinData:/data/db -d -p 37017:27017 mongo

Couchbase running in a container not accessible

So I've created this Dockerfile:
FROM centos
EXPOSE 7081 8092 11210
RUN yum install -y \
hostname \
initscripts \
openssl098e \
pkgconfig \
sudo \
tar \
wget \
&& wget http://packages.couchbase.com/releases/3.0.2/couchbase-server-enterprise-3.0.2-centos6.x86_64.rpm \
&& yum install -y couchbase-server-enterprise-3.0.2-centos6.x86_64.rpm \
&& rm -f ./couchbase-server-enterprise-3.0.2-centos6.x86_64.rpm \
CMD /opt/couchbase/bin/couchbase-server start -- -noinput
And that seems to be working (running the couchbase server) and to build and run it I do:
docker build -t="my/couchbase" .
docker run -itd --name=couchbase -p 11210:11210 -p 8091:7081 -p 8092:8092 my/couchbase
Now for some reason I can't connect to it via http. I tried to get ip address of the container with docker inspect couchbase | grep IP
and then going to http://containters_ip:7081
It's trying to get there for a very long time, but eventually times out.
What am I doing wrong?
You need EXPOSE 8091 8092 11210 (think of this as "the container listens on these ports") and -p 7081:8091 to get the mapping you seek. In -p it's hostport:containerport order.