I have a data file which needs to be added to the mongodb in docker. i have a docker file but i do not know how to copy my data file from local machine to mongodb docker image.
My Docker file :
FROM dockerfile/ubuntu
RUN \
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10 && \
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' > /etc/apt/sources.list.d/mongodb.list && \
apt-get update && \
apt-get install -y mongodb-org && \
rm -rf /var/lib/apt/lists/*
VOLUME ["/data/db"]
WORKDIR /data
CMD ["mongod"]
EXPOSE 27017
EXPOSE 28017
How can i add data files from my local to the mongodb using above docker file
Docker has COPY command. Firstly, make sure your date file is at the same directory with Dockerfile. Then simply add below line to your Dockerfile.
COPY your-data-file /pathInContainer
Related
I have two images that use a two-stage build to build Scala code and copy the artifacts to a final image. To speed up the build, I copy my local ~/.ivy2 to the context directory and from there to the images (~1GB). Unfortunately this means that even when nothing has changed and the images don't need to be re-built, docker-compose build (or docker build) hangs for quite a while to copy Docker context. This happens twice of course, once for each image.
Is there any cleverer way to do this?
Dockerfile:
FROM openjdk:8
RUN apt-get update &&\
apt-get install -y apt-transport-https gnupg2 &&\
echo "deb https://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list &&\
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 &&\
apt-get update &&\
apt-get install -y sbt=1.1.6
COPY ivy-cache/ /root/.ivy2
COPY app/source/ /app/source
RUN cd /app/source &&\
sbt assembly &&\
cp target/scala-2.11/my-app-*.jar /app/my-app.jar
FROM gettyimages/spark:2.3.1-hadoop-3.0
COPY --from=0 /app/my-app.jar /app/my-app.jar
CMD ["spark-submit", "--master", "local", "/app/my-app.jar"]
With 18.09, docker includes BuildKit. By itself, BuildKit will cache the previous context and only send over the differences with the equivalent of rsync in the background.
For this specific case, you can use some experimental features to mount in your dependency cache as the equivalent of a named volume using the RUN --mount syntax. The cache directory never makes it into the image, but is there for later builds, and when you pull in a new dependency it will behave just like a local build, downloading only the new dependencies.
# syntax=docker/dockerfile:experimental
FROM openjdk:8 as build
RUN apt-get update &&\
apt-get install -y apt-transport-https gnupg2 &&\
echo "deb https://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list &&\
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 &&\
apt-get update &&\
apt-get install -y sbt=1.1.6
COPY app/source/ /app/source
RUN --mount=type=cache,target=/root/.ivy2 \
cd /app/source &&\
sbt assembly &&\
cp target/scala-2.11/my-app-*.jar /app/my-app.jar
FROM gettyimages/spark:2.3.1-hadoop-3.0 as release
COPY --from=build /app/my-app.jar /app/my-app.jar
CMD ["spark-submit", "--master", "local", "/app/my-app.jar"]
To use BuildKit under 18.09, you can either export an environment variable:
export DOCKER_BUILDKIT=1
or update the engine with the new default in /etc/docker/daemon.json:
{ "features": {"buildkit": true} }
How to build postgresql-9.6 image from postgresql-9.6.1.tar.gz using dockerfile?
I tried to create below dockerfile to install postgresql-9.6 on ubuntu. But I am unable to make it a complete image?
FROM ubuntu:16.04
RUN apt-get update && \
apt-get install software-properties-common -y && \
apt-get install wget && \
add-apt-repository "deb http://apt.postgresql.org/pub/repos/apt/ xenial-pgdg main" && \
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - && \
apt-get update && \
apt-get install -y postgresql-9.6 postgresql-client-9.6
EXPOSE 5432
So as an alternative I want to create image from tarball
The official Postgres Dockerfile does a build from source on Debian which should be largely portable to Ubuntu.
It will be easier to just use the postgres:9.6 or postgres:9.6.1 image as a seperate container to your application, rather than trying to manage a build heavy, monolithic container yourself.
So I was following this tutorial:
https://realpython.com/blog/python/django-development-with-docker-compose-and-machine/
I have everything up and running, however theres a few things going on that I'm not able to follow or understand.
In the main Docker-Compose we have:
web:
restart: always
build: ./web
expose:
- "8000"
links:
- postgres:postgres
- redis:redis
volumes:
- /usr/src/app
- /usr/src/app/static
env_file: .env
command: /usr/local/bin/gunicorn docker_django.wsgi:application -w 2 -b :8000
postgres:
restart: always
image: postgres:latest
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data/
You will notice there is a env_file containing:
DB_NAME=postgres
DB_USER=postgres
DB_PASS=postgres
My question is when is the postgres user and password being set? If I run this docker-compose everything works, meaning the web app can pass credentials to the postgress database and establish a connection. I'm not able to follow however, where those credentials are being set in the first place.
I was assuming in the base postgres Dockerfile, there would be some instruction to set the database name, username and password. I do not it though. Here is a copy of the base postgres Dockerfile below.
# vim:set ft=dockerfile:
FROM debian:jessie
# explicitly set user/group IDs
RUN groupadd -r postgres --gid=999 && useradd -r -g postgres --uid=999 postgres
# grab gosu for easy step-down from root
ENV GOSU_VERSION 1.7
RUN set -x \
&& apt-get update && apt-get install -y --no-install-recommends ca-certificates wget && rm -rf /var/lib/apt/lists/* \
&& wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \
&& wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc" \
&& export GNUPGHOME="$(mktemp -d)" \
&& gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
&& gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu \
&& rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu \
&& gosu nobody true \
&& apt-get purge -y --auto-remove ca-certificates wget
# make the "en_US.UTF-8" locale so postgres will be utf-8 enabled by default
RUN apt-get update && apt-get install -y locales && rm -rf /var/lib/apt/lists/* \
&& localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
ENV LANG en_US.utf8
RUN mkdir /docker-entrypoint-initdb.d
RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8
ENV PG_MAJOR 9.6
ENV PG_VERSION 9.6.1-1.pgdg80+1
RUN echo 'deb http://apt.postgresql.org/pub/repos/apt/ jessie-pgdg main' $PG_MAJOR > /etc/apt/sources.list.d/pgdg.list
RUN apt-get update \
&& apt-get install -y postgresql-common \
&& sed -ri 's/#(create_main_cluster) .*$/\1 = false/' /etc/postgresql-common/createcluster.conf \
&& apt-get install -y \
postgresql-$PG_MAJOR=$PG_VERSION \
postgresql-contrib-$PG_MAJOR=$PG_VERSION \
&& rm -rf /var/lib/apt/lists/*
# make the sample config easier to munge (and "correct by default")
RUN mv -v /usr/share/postgresql/$PG_MAJOR/postgresql.conf.sample /usr/share/postgresql/ \
&& ln -sv ../postgresql.conf.sample /usr/share/postgresql/$PG_MAJOR/ \
&& sed -ri "s!^#?(listen_addresses)\s*=\s*\S+.*!\1 = '*'!" /usr/share/postgresql/postgresql.conf.sample
RUN mkdir -p /var/run/postgresql && chown -R postgres /var/run/postgresql
ENV PATH /usr/lib/postgresql/$PG_MAJOR/bin:$PATH
ENV PGDATA /var/lib/postgresql/data
VOLUME /var/lib/postgresql/data
COPY docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 5432
CMD ["postgres"]
Not sure which postgres image are you using.
If you look at the official postgres image for complete information. It allows user to specify the environment variables for the below ones and these can be easily overridden at run time.
POSTGRES_PASSWORD
POSTGRES_USER
PGDATA
POSTGRES_DB
POSTGRES_INITDB_ARGS
Environment variables can be overridden using below three methods depending on your case.
Run Image: If running docker image directly, use below to include environment variable in docker run with -e K=V. Please refer documentation for more details here
docker run -e POSTGRES_PASSWORD=secrect -e POSTGRES_USER=postgres <other options> image/name
Dockerfile: If you need to specify the environment variable in Dockerfile, specify as mentioned below. Please refer documentation for more details here
ENV POSTGRES_PASSWORD=secrect
ENV POSTGRES_USER=postgres
docker-compose: If you need to specify the environment variable in docker-compose.yml, specify as below. Please refer documentation for more details here
web:
environment:
- POSTGRES_PASSWORD=secrect
- POSTGRES_USER=postgres
Hope this is useful.
I think the postgres user and password is being set on entrypoint, like in line 23 on official image entrypoint.
https://github.com/docker-library/postgres/blob/e4942cb0f79b61024963dc0ac196375b26fa60dd/9.6/docker-entrypoint.sh
Can you check your entrypoint?
All you need is
docker run -e POSTGRES_PASSWORD=123456 postgres
In the same way you can set up the username like below:
docker run -e POSTGRES_USERNAME=xyz postgres
Are you using the -v mounted folder? If so, you need to remove the entire pgdata/ folder you created previously so that the postgres can re-create itself using the new environment variables.
I've got a docker container, in which I'm installing mongo db. After installing,
I'm trying to start mongo and restore a mongo db dump. However, when I start the docker instance, I see that the user has been switched to root (as per supervisor instruction) but mongo is not started.
This is the supervisor snippet:
[supervisord]
nodaemon=true
[program:mongodb]
user=root
command=/usr/bin/mongod
This is my setup in the dockerfile:
RUN apt-get update && sudo apt-get install -y supervisor
RUN mkdir -p /var/log/supervisor
COPY supervisor.conf /etc/supervisor/conf.d/supervisor.conf
# Install MongoDB.
RUN \
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10 && \
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen'
| tee /etc/apt/sources.list.d/mongodb.list && \
apt-get update && \
apt-get install -y mongodb-org && \
rm -rf /var/lib/apt/lists/*
# Define mountable directories.
VOLUME ["/data/db"]
# Define working directory.
WORKDIR /data
# Define default command.
CMD ["mongod"]
EXPOSE 27017
EXPOSE 28017
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/supervisord.conf"]
Am I missing any configuration setting? Any help is appreciated.
You're not able to run mongodb because while installing mongodb some files need authentication, so you just have to replace apt-get install -y mongodb-org with apt-get install -y --no-authentication mongodb-org and you'll be able to install mongodb without any problem.
I am new at Codenvy so my question can be very stupid, by the way this is my problem:
I created my project then to add the DB (postgresql) i created a new docker file and i pasted in it the code in this file: https://github.com/codenvy/dockerfiles/blob/master/base/jdk7_postgresql/Dockerfile
I saved it then i ran the project with the runner just created, but i have this error:
[DOCKER] Setting up dh-python (1.20141111-2) ...
[DOCKER] Processing triggers for systemd (215-17+deb8u2) ...
[DOCKER] Processing triggers for libc-bin (2.19-18+deb8u1) ...
[DOCKER] Processing triggers for dbus (1.8.20-0+deb8u1) ...
[DOCKER] Starting PostgreSQL 9.3 database server:
[DOCKER] main
[DOCKER] .
[DOCKER] CREATE ROLE
[DOCKER] ---> 77b708d3360b
[DOCKER] Removing intermediate container cdd908fb498e
[DOCKER] Step 4 : ADD startup.sh /home/user/startup.sh
[DOCKER][ERROR] startup.sh: no such file or directory
[ERROR] We are having trouble starting the runner and deploying application. Either necessary files are missing or a fundamental configuration has changed.
Docker image build failed
Any idea to how can i solve the problem?
thanks in advance!
This PostgresSQL image uses startup.sh script to start postgresql service. You can find it at https://github.com/codenvy/dockerfiles/blob/master/base/jdk7_postgresql/startup.sh
To solve this problem you can either create a new file startup.sh with an identical content in the root of your project, add it as a source (e.g. ADD $src$/startup.sh /home/user/startup.sh). You will also need to inject project sources to make it work as expected, so as a result your Dockerfile may look like this:
FROM codenvy/jdk7
ENV DEBIAN_FRONTEND noninteractive
RUN sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8 && \
echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list && \
sudo apt-get update && \
sudo -E bash -c "apt-get install -y python-software-properties software-properties-common postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3 pwgen" && \
sudo service postgresql start && \
CODENVY_POSTGRESQL_PASSWORD=$(pwgen -N 1) && echo "export CODENVY_POSTGRESQL_PASSWORD=$CODENVY_POSTGRESQL_PASSWORD" >> /home/user/.postgresrc && \
CODENVY_POSTGRESQL_DB=testdb_$(pwgen -N 1) && echo "export CODENVY_POSTGRESQL_DB=$CODENVY_POSTGRESQL_DB" >> /home/user/.postgresrc && \
CODENVY_POSTGRESQL_USER=codenvy && echo "export CODENVY_POSTGRESQL_USER=$CODENVY_POSTGRESQL_USER" >> /home/user/.postgresrc && \
sudo -u postgres psql --command "CREATE USER $CODENVY_POSTGRESQL_USER WITH SUPERUSER PASSWORD '$CODENVY_POSTGRESQL_PASSWORD';" && \
sudo -u postgres createdb -O $CODENVY_POSTGRESQL_USER $CODENVY_POSTGRESQL_DB
#Inject project sources
ADD $app$ /home/user/$app$
RUN unzip -q /home/user/$app$ -d /home/user
ENV ARGUMENTS $args$
#Add file to the container
ADD $src$/startup.sh /home/user/startup.sh
RUN sudo chmod +x /home/user/startup.sh
#EXPOSE 5432
CMD sudo /home/user/startup.sh
Or you can simply use Codenvy default PostgreSQL 9.3 + Java 7 runner, which you will find on the Runners panel > Configs tab.