User permission is denied with chown using Dockerfile on docker-compose - docker-compose

I'm trying to mount a volume in docker-compose. But it seems my user does not have permission to use volume. :/
My dockerfile is:
FROM openjdk:8u181-jdk-slim
ENV HOME /app
ENV CONFIG_PATH $HOME/config
ENV DATA_PATH $HOME/data
ENV LOG_PATH $HOME/log
RUN addgroup --gid 1001 myuser \
&& adduser --uid 1001 --gid 1001 --home $HOME --shell /bin/bash \
--gecos "" --no-create-home --disabled-password myuser \
&& mkdir -p $CONFIG_PATH $DATA_PATH $LOG_PATH \
&& chown -R myuser:myuser $HOME \
&& chmod -R g=u $HOME \
&& chmod +x $HOME/*
RUN apt-get update \
&& apt-get install -y curl \
&& apt-get clean
VOLUME $CONFIG_PATH $DATA_PATH $LOG_PATH
USER myuser:myuser
EXPOSE 7777
EXPOSE 8080
HEALTHCHECK --interval=1m --timeout=10s --start-period=2m \
CMD curl -f http://localhost:7777/health || exit 1
COPY --chown=myuser my-service-*.jar $HOME/my-service.jar
ENTRYPOINT ["/bin/bash", "-c", "java $JAVA_OPTS -jar $HOME/my-service.jar $0 $#"]
my docker-compose file is:
volumes:
my-service_stream:
my-service:
image: my-service-image
networks:
- internal
env_file:
- config/common.env
volumes:
- my-service_stream:/app/data/state
not able to use myuser and not able to mount for this user :/. I can not use and myuser does not have permission to write that volume.
I have tried adding user to my docker-compose file as
user: "1001:1001"
but nothing is changed.

Related

Permission denied executing jar file downloaded to .Ivy folder in Kubernetes

I created a dockerfile and changed User to a non-root user nobody. Locally this works perfectly. When deployed on kubernetes howerever, I get error
java.nio.file.AccessDeniedException: ./xxxxxx_2.12-2.6.3.jar
When I dug in more I realised this jar file is download after the spark dependencies used in the dockerfiles are downloaded. Therefore any permissions given to the spark folder are not present for this newly dowloaded jar file which is downloaded at runtime into /opt/spark/.ivy2/xxx which has root permissions. This causes the pod in kubernetes to fail.
I am wondering if there is way to give permissions to execute this jar file. Since it seems this is not possible in the Dockerfile. Any suggestion as to how to solve this issue ??
As proposed by #mario
ARG SPARK_OPERATOR_BASE_IMAGE_VERSION=v2.4.5
FROM xxx/alpine:3.12 as preparator
ARG SCALA_VERSION=2.12
ARG SPARK_VERSION=2.4.7
ARG HADOOP_VERSION=3.2.1
ARG AWS_SDK_VERSION=1.11.375
ARG MAVEN_VERSION=3.6.2
RUN apk add --no-cache \
bash \
curl && \
mkdir /target
COPY hashes /tmp/
COPY prepare /tmp/
WORKDIR /tmp
# Download Hadoop
RUN curl -L -O https://downloads.apache.org/hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz && \
sha256sum -c hadoop-${HADOOP_VERSION}.sha256 && \
tar -xzf hadoop-${HADOOP_VERSION}.tar.gz && \
mv hadoop-${HADOOP_VERSION} /target/hadoop
# Download Spark
RUN curl -L -O https://downloads.apache.org/spark/spark-${SPARK_VERSION}/spark-${SPARK_VERSION}-bin-without-hadoop-scala-${SCALA_VERSION}.tgz && \
sha512sum -c spark-${SPARK_VERSION}-bin-without-hadoop-scala-${SCALA_VERSION}.sha512 && \
tar -xzf spark-${SPARK_VERSION}-bin-without-hadoop-scala-${SCALA_VERSION}.tgz && \
mv spark-${SPARK_VERSION}-bin-without-hadoop-scala-${SCALA_VERSION} /target/spark && \
# Download Spark 3.0.0 entrypoint script from GitHub, bugfixing for 2.4.7
curl -L -O https://raw.githubusercontent.com/apache/spark/v3.0.0/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh && \
mv entrypoint.sh /target/entrypoint.sh && \
chmod +x /target/entrypoint.sh
# Download AWS Jars
RUN curl -L -O https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/${HADOOP_VERSION}/hadoop-aws-${HADOOP_VERSION}.jar && \
sha1sum -c hadoop-aws-${HADOOP_VERSION}.jar.sha1 && \
mv hadoop-aws-${HADOOP_VERSION}.jar /target/spark/jars/ && \
curl -L -O https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/${AWS_SDK_VERSION}/aws-java-sdk-bundle-${AWS_SDK_VERSION}.jar && \
sha1sum -c aws-java-sdk-bundle-${AWS_SDK_VERSION}.jar.sha1 && \
mv aws-java-sdk-bundle-${AWS_SDK_VERSION}.jar /target/spark/jars/
# Directory needed for saving built jars
RUN mkdir /target/spark/custom-jars/
#### Download Prometheus + Metric dependencies ####
# install java, maven and prometheus fat jar using maven (pom.xml)
RUN apk add --update openjdk8 && \
curl -L -O https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz && \
tar -xzf apache-maven-${MAVEN_VERSION}-bin.tar.gz && export PATH=./apache-maven-${MAVEN_VERSION}/bin:$PATH && \
mv prometheus-pom.xml pom.xml && mvn clean package && mv target/prometheusMetricLibs-jar-with-dependencies.jar /target/spark/custom-jars/
RUN \
chown -R nobody:99 /target/spark \
&& chown -R nobody:99 /target/hadoop \
&& chmod -R ugo+rw /target/spark \
&& chmod -R ugo+rw /target/hadoop
ARG SPARK_OPERATOR_BASE_IMAGE_VERSION
FROM gcr.io/spark-operator/spark:${SPARK_OPERATOR_BASE_IMAGE_VERSION}
RUN rm -rf /opt/spark/
COPY --from=preparator /target/ /opt/
ENV SPARK_HOME=/opt/spark \
HADOOP_HOME=/opt/hadoop
ENV HADOOP_OPTS="-Djava.library.path=/opt/hadoop/lib/native" \
LD_LIBRARY_PATH=${HADOOP_HOME}/lib/native \
PATH=${HADOOP_HOME}/bin:${SPARK_HOME}/bin:${PATH}
COPY conf /opt/spark/conf/
RUN echo "export JAVA_HOME=${JAVA_HOME}" >> ${HADOOP_HOME}/etc/hadoop/hadoop-env.sh && \
echo "export JAVA_HOME=${JAVA_HOME}" > ${SPARK_HOME}/conf/spark-env.sh && \
echo "export SPARK_DIST_CLASSPATH=\$(hadoop classpath)" >> ${SPARK_HOME}/conf/spark-env.sh
# 99 used instead of nobody because in the alpine image the Group Id of nobody is different than in the spark operator image.
RUN \
addgroup --gid 99 nobody \
&& echo "nobody:x:99:99:nobody:/nonexistent:/usr/sbin/nologin" >> /etc/passwd \
&& usermod -a -G users nobody \
&& chmod -R ugo+rw /var/lib/
USER nobody
# if we want local storage
# "spark.eventLog.dir": "tmp/spark-events"
RUN mkdir -p /tmp/spark-events
in my pod the jar file is implemented like this
sparkConf:
"spark.ui.port": "4045"
"spark.eventLog.enabled": {{ .Values.spark.eventLogEnabled | quote }}
"spark.eventLog.dir": "xx//"
"spark.jars.ivySettings": "/vault/secrets/xx-ivysettings.xml"
"spark.jars.ivy": "/opt/spark/.ivy2"
"spark.jars.packages": "xxxx_2.12:{{ .Values.appVersion }}"
"spark.blacklist.enabled": "false"
"spark.driver.supervise": "true"
"spark.app.name": {{ .Values.name | quote }}
"spark.submit.deployMode": {{ .Values.spark.deployMode | quote }}
"spark.driver.extraJavaOptions": "-Dlog4j.configurationFile=log4j.properties"
"spark.executor.extraJavaOptions": "-Dlog4j.configurationFile=log4j.properties"
It would be easier to answer your question if you share your Dockerfile and your kubernetes Pod template yaml manifest, but in short: you can manipulate your Pod's permissions using the securityContext like in the example from the docs below:
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: busybox
command: [ "sh", "-c", "sleep 1h" ]
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
allowPrivilegeEscalation: false
For debugging purposes you can start from setting allowPrivilegeEscalation to true or runAsUser: 0 (root) but keep in mind this is not the solution that can be used in production. Running containers as root is generally a bad idea and in most cases it can be avoided.
Therefore any permissions given to the spark folder are not present
for this newly dowloaded jar file which is downloaded at runtime into
/opt/spark/.ivy2/xxx which has root permissions.
Is it really necessary for it to have root permissions ? Most likely it can be fixed in your Dockerfile.

Can't remotely start the project with pycharm

I have made a local workstation with CentOS 8, I have installed docker CE and everything necessary to set up a remote development environment (smb, firewall, etc.).
My client computer is a mac and the application is based on a docker-compose of several containers which with the docker MAC application works correctly.
After configuring pycharm professional with the remote interpreter and starting the project there are 2 containers that start perfectly (postgres: 11.2-alpine, redis: alpine) but the application that is a Django-rest with a specific dockerfile when it is ready to execute the entrypoint throws The following error that I attach below although I have accessed the server and if I execute the command docker-compose up --build it starts perfectly, it is something related to the fact of starting from Pycharm.
Has anyone had this problem and been able to solve it?
Successfully tagged energy_energy:latest
Creating energy_redis-celery-energy_1 ...
Creating energy_redis-cache-energy_1 ...
Creating energy_postgres_1 ...
Creating energy_celery-worker-energy_1 ...
Creating energy_celery-beat-energy_1 ...
Creating energy_celery-beat-energy_1 ... error
ERROR: for energy_celery-beat-energy_1 Cannot start service celery-beat-energy: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": stat ./start.sh: no such file or directory": unknown
Creating energy_celery-worker-energy_1 ... error
ERROR: for energy_celery-worker-energy_1 Cannot start service celery-worker-energy: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": stat ./start.sh: no such file or directory": unknown
ERROR: for celery-beat-energy Cannot start service celery-beat-energy: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": stat ./start.sh: no such file or directory": unknown
ERROR: for celery-worker-energy Cannot start service celery-worker-energy: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": stat ./start.sh: no such file or directory": unknown
ERROR: Encountered errors while bringing up the project
This is my entire Dockerfile:
FROM python:3.7-alpine
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV CERTS_DIR /var/certs
ENV NGINX_VERSION 1.15.3
EXPOSE 1234
RUN apk update \
&& GPG_KEYS=B0F4253373F8F6F510D42178520A9993A1C052F8 \
&& CONFIG="\
--prefix=/etc/nginx \
--sbin-path=/usr/sbin/nginx \
--modules-path=/usr/lib/nginx/modules \
--conf-path=/etc/nginx/nginx.conf \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--pid-path=/var/run/nginx.pid \
--lock-path=/var/run/nginx.lock \
--http-client-body-temp-path=/var/cache/nginx/client_temp \
--http-proxy-temp-path=/var/cache/nginx/proxy_temp \
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
--http-scgi-temp-path=/var/cache/nginx/scgi_temp \
--user=nginx \
--group=nginx \
--with-http_ssl_module \
--with-http_realip_module \
--with-http_addition_module \
--with-http_sub_module \
--with-http_dav_module \
--with-http_flv_module \
--with-http_mp4_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_random_index_module \
--with-http_secure_link_module \
--with-http_stub_status_module \
--with-http_auth_request_module \
--with-http_xslt_module=dynamic \
--with-http_image_filter_module=dynamic \
--with-http_geoip_module=dynamic \
--with-threads \
--with-stream \
--with-stream_ssl_module \
--with-stream_ssl_preread_module \
--with-stream_realip_module \
--with-stream_geoip_module=dynamic \
--with-http_slice_module \
--with-mail \
--with-mail_ssl_module \
--with-compat \
--with-file-aio \
--with-http_v2_module \
" \
&& addgroup -S nginx \
&& adduser -D -S -h /var/cache/nginx -s /sbin/nologin -G nginx nginx \
&& apk add tzdata \
gcc \
libc-dev \
linux-headers \
mariadb-dev \
postgresql-dev \
netcat-openbsd \
curl \
libffi-dev \
supervisor \
&& apk add --no-cache --virtual .build-deps \
make \
openssl-dev \
pcre-dev \
zlib-dev \
gnupg1 \
libxslt-dev \
gd-dev \
geoip-dev \
&& curl -fSL https://nginx.org/download/nginx-$NGINX_VERSION.tar.gz -o nginx.tar.gz \
&& curl -fSL https://nginx.org/download/nginx-$NGINX_VERSION.tar.gz.asc -o nginx.tar.gz.asc \
&& export GNUPGHOME="$(mktemp -d)" \
&& found=''; \
for server in \
ha.pool.sks-keyservers.net \
hkp://keyserver.ubuntu.com:80 \
hkp://p80.pool.sks-keyservers.net:80 \
pgp.mit.edu \
; do \
echo "Fetching GPG key $GPG_KEYS from $server"; \
gpg --keyserver "$server" --keyserver-options timeout=10 --recv-keys "$GPG_KEYS" && found=yes && break; \
done; \
test -z "$found" && echo >&2 "error: failed to fetch GPG key $GPG_KEYS" && exit 1; \
gpg --batch --verify nginx.tar.gz.asc nginx.tar.gz \
&& rm -rf "$GNUPGHOME" nginx.tar.gz.asc \
&& mkdir -p /usr/src \
&& tar -zxC /usr/src -f nginx.tar.gz \
&& rm nginx.tar.gz \
&& cd /usr/src/nginx-$NGINX_VERSION \
&& ./configure $CONFIG --with-debug \
&& make -j$(getconf _NPROCESSORS_ONLN) \
&& mv objs/nginx objs/nginx-debug \
&& mv objs/ngx_http_xslt_filter_module.so objs/ngx_http_xslt_filter_module-debug.so \
&& mv objs/ngx_http_image_filter_module.so objs/ngx_http_image_filter_module-debug.so \
&& mv objs/ngx_http_geoip_module.so objs/ngx_http_geoip_module-debug.so \
&& mv objs/ngx_stream_geoip_module.so objs/ngx_stream_geoip_module-debug.so \
&& ./configure $CONFIG \
&& make -j$(getconf _NPROCESSORS_ONLN) \
&& make install \
&& rm -rf /etc/nginx/html/ \
&& mkdir /etc/nginx/conf.d/ \
&& mkdir -p /usr/share/nginx/html/ \
&& install -m644 html/index.html /usr/share/nginx/html/ \
&& install -m644 html/50x.html /usr/share/nginx/html/ \
&& install -m755 objs/nginx-debug /usr/sbin/nginx-debug \
&& install -m755 objs/ngx_http_xslt_filter_module-debug.so /usr/lib/nginx/modules/ngx_http_xslt_filter_module-debug.so \
&& install -m755 objs/ngx_http_image_filter_module-debug.so /usr/lib/nginx/modules/ngx_http_image_filter_module-debug.so \
&& install -m755 objs/ngx_http_geoip_module-debug.so /usr/lib/nginx/modules/ngx_http_geoip_module-debug.so \
&& install -m755 objs/ngx_stream_geoip_module-debug.so /usr/lib/nginx/modules/ngx_stream_geoip_module-debug.so \
&& ln -s ../../usr/lib/nginx/modules /etc/nginx/modules \
&& strip /usr/sbin/nginx* \
&& strip /usr/lib/nginx/modules/*.so \
&& rm -rf /usr/src/nginx-$NGINX_VERSION \
\
&& apk add --no-cache --virtual .gettext gettext \
&& mv /usr/bin/envsubst /tmp/ \
\
&& runDeps="$( \
scanelf --needed --nobanner --format '%n#p' /usr/sbin/nginx /usr/lib/nginx/modules/*.so /tmp/envsubst \
| tr ',' '\n' \
| sort -u \
| awk 'system("[ -e /usr/local/lib/" $1 " ]") == 0 { next } { print "so:" $1 }' \
)" \
&& apk add --no-cache --virtual .nginx-rundeps $runDeps \
&& apk del .build-deps \
&& apk del .gettext \
&& mv /tmp/envsubst /usr/local/bin/ \
\
&& ln -sf /dev/stdout /var/log/nginx/access.log \
&& ln -sf /dev/stderr /var/log/nginx/error.log
COPY nginx.conf /etc/nginx/nginx.conf
COPY conf/certs /var/certs
COPY ./src /app
COPY supervisord.ini /etc/supervisor.d/supervisord.ini
WORKDIR /app
RUN pip install --upgrade pip
RUN pip install pipenv
RUN pipenv install --system
RUN chmod +x ./start.sh
ENTRYPOINT ["./start.sh"]
This is my entire docker-compose:
# SERVICE ENDPOINT USER PASSWORD
# ==========================================================
# postgres-11 localhost:2323 root root
# redis-cache-energy sin exposición no no
# redis-celery sin exposición no no
# energy localhost:1337 no no Nnigx => Gunicorn => Django
# celery-beat sin exposición no no
# celery-worker sin exposición no no
version: '3'
services:
####################################################################################################
# DATABASES #
####################################################################################################
postgres:
image: postgres:11.2-alpine
restart: always
volumes:
- postgres_data_energy:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=root
- POSTGRES_PASSWORD=root
- POSTGRES_DB=local
ports:
- "2323:5432"
networks:
- energy-network
####################################################################################################
# REDIS #
####################################################################################################
redis-cache-energy:
image: redis:alpine
command: redis-server --requirepass root
restart: always
networks:
- energy-network
redis-celery-energy:
image: redis:alpine
command: redis-server --requirepass root
restart: always
networks:
- energy-network
####################################################################################################
# energy #
####################################################################################################
celery-beat-energy:
build: .
command: celery-beat-energy
restart: always
volumes:
- ./src:/app
- ./logs:/var/logs
env_file:
- conf/local.env
depends_on:
- redis-celery-energy
networks:
- energy-network
celery-worker-energy:
build: .
command: celery-worker-energy
restart: always
volumes:
- ./src:/app
- ./logs:/var/logs
env_file:
- conf/local.env
depends_on:
- redis-celery-energy
networks:
- energy-network
energy:
build: .
command: nginx
restart: always
volumes:
- ./src:/app
- ./logs:/var/logs
ports:
- 1234:1234
env_file:
- conf/local.env
depends_on:
- postgres
- redis-cache-energy
- redis-celery-energy
- celery-worker-energy
- celery-beat-energy
networks:
- energy-network
####################################################################################################
# VOLUMES #
####################################################################################################
volumes:
postgres_data_energy:
####################################################################################################
# networks #
####################################################################################################
networks:
energy-network:
driver: bridge
More info:
If i run the project from terminal SSH from pycharm and run a ls command in the entrypoint, i see the files.
SSH TERMINAL FROM PYCHARM
RESULT ENTRYPOINT, SEE THE FILES
If I run form pycharm the ls is empty.
RESULT ENTRYPOINT, EMPTY
RUN FROM PYCHARM
More info about my problem

Rename volume in docker compose, with docker derived from postgres

I have this docker-compose
version: '3.7'
services:
app-db:
build:
context: ./
dockerfile: Dockerfile-pg
image: app-pg:1.0.0
restart: always
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: app
volumes:
- ./docker-entrypoint-initdb.d/init-user-db.sh:/docker-entrypoint-initdb.d/init-user-db.sh
- v-app-pgdata:/var/lib/postgresql
- v-app-pglog:/data/var/log/postgresql
- v-app-pgconf:/etc/postgresql
app-main:
build:
context: ./
dockerfile: Dockerfile-tar-cp
image: app-main:1.0.0
restart: always
ports:
- 80:80
volumes:
v-app-pgdata:
name: v-app-pgdata
v-app-pglog:
name: v-app-pglog
v-app-pgconf:
name: v-app-pgconf
so an app container and a postgres derived container:
#docker build -t app-pg:1.0.0 -f Dockerfile-pg .
#docker run -d --name appC-pg -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=postgres app-pg:1.0.0
FROM postgres:12.1
MAINTAINER xxx
#ARG A_DB_USER='postgres'
#ARG A_DB_PASS='postgres'
ARG A_DB_NAME='app'
ARG A_TZ='Europe/Zurich'
#ENV DB_USER=${A_DB_USER}
#ENV DB_PASS=${A_DB_PASS}
ENV DB_NAME=${A_DB_NAME}
ENV TZ=${A_TZ}
# Adjusting Timezone in the system
RUN echo $TZ > /etc/timezone && \
apt-get update && apt-get install -y tzdata && \
rm /etc/localtime && \
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && \
dpkg-reconfigure -f noninteractive tzdata && \
apt-get clean
# install postgis
RUN apt-get update && \
apt-get install -y postgis && \
apt-get clean
USER postgres
#Add password "postgres" to user postgres, create db, add .sql
#RUN /etc/init.d/postgresql start && \
# psql --command "ALTER USER ${DB_USER} WITH PASSWORD '${DB_PASS}'; SET TIME ZONE '${TZ}';" && \
# createdb -O ${DB_USER} ${DB_NAME} -E UTF8 && \
# psql -d ${DB_NAME} -c 'CREATE EXTENSION postgis'
EXPOSE 5432
My problem is that postgres default dockerfile have this line:
VOLUME /var/lib/postgresql/data
So even if I create a named Volumes with the same folder, my docker-compose create 4 and not 3 volumes, one unnamed due to that line.
How is it possible solve this issue?
I had the same problem as you and I solved it by editing the docker-compose.yml the following way:
Create a volume for the data
volumes:
v-app-pgdata:
In your declaration of the database service, in the volumes clause, you need to change - v-app-pgdata:/var/lib/postgresql to:
- v-app-pgdata:/var/lib/postgresql/data
The problem was that you where not mounting the volume to the correct place, and therefore, it had to create another volume that is declared in the base image VOLUME /var/lib/postgresql/data. If you run this container without attaching a volume to this mount point, it will automatically create it, and with a random name.
However, it has been pointed out to me that if the OP strictly needs a volume in /var/lib/postgresql then this solution wont work.
Hope this helps, as it worked for me.

How to execute multiple commands from docker-compose.yml file

I am using YAML file configuration as below mentioned, unable to create a folder and change access of dir.
It is able to create the directory but unable to complete this command chmod 777 /opt/ldap
ldap:
image: osixia/openldap:1.3.0
env_file:
- env/ldap.env
networks:
appworks_net:
aliases:
- docker_container.local
restart: "no"
command: bash -c "mkdir -p /opt/ldap && chmod 777 /opt/ldap" - **creates only folder**
command: ["/bin/bash", "-c", "mkdir -p /opt/ldap && chmod 777 /opt/ldap"] -- fails start container
command: ["-c", "chmod 777 /opt/ldap"] -- does not work
command: chmod 777 /opt/ldap -- does not work
command: ["-c", "mkdir -p /opt/appworks/ldap", "chmod 777 /opt/appworks/ldap"] -- only creates folder
command:
- -c
- |
mkdir -p /opt/ldap
chmod 777 /opt/ldap -- **creates only folder**
healthcheck:
test: ["CMD-SHELL", '/usr/bin/ldapsearch -h ldaps://$$HOSTNAME -p $$PORT -w $$LDAP_ADMIN_PASSWORD -D "cn=admin,dc=trialorg,dc=local" -b "dc=trialorg,dc=local" | grep "dn:"']
interval: 5s
timeout: 5s
retries: 10
please help me here if there are any possible working ideas.
I don't want to create a new image using a docker / entrypoint. I just that two commands to run post container is started.
What about
command:
- -c
- |
mkdir -p /opt/ldap;
chmod 777 /opt/ldap;
or
command:
- -c
- |
mkdir -p /opt/ldap && chmod 777 /opt/ldap;
Also are you sure that you have UNIX line endings in your file?
Another possible solution Using Docker-Compose, how to execute multiple commands

How to persist data using a postgres database, Docker, and Kubernetes?

I am trying to mount a persistent disk on my container which runs a Postgres custom image. I am using Kubernetes and following this tutorial.
This is my db_pod.yaml file:
apiVersion: v1
kind: Pod
metadata:
name: lp-db
labels:
name: lp-db
spec:
containers:
- image: my_username/my-db
name: my-db
ports:
- containerPort: 5432
name: my-db
volumeMounts:
- name: pg-data
mountPath: /var/lib/postgresql/data
volumes:
- name: pg-data
gcePersistentDisk:
pdName: my-db-disk
fsType: ext4
I create the disk using the command gcloud compute disks create --size 200GB my-db-disk.
However, when I run the pod, delete it, and then run it again (like in the tutorial) my data is not persisted.
I tried multiple versions of this file, including with PersistentVolumes and PersistentVolumeClaims, I tried changing the mountPath, but to no success.
Edit
Dockerfile for creating the Postgres image:
FROM ubuntu:trusty
RUN rm /bin/sh && \
ln -s /bin/bash /bin/sh
# Get Postgres
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
RUN apt-get update && \
apt-get install -y wget
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
# Install virtualenv (will be needed later)
RUN apt-get update && \
apt-get install -y \
libjpeg-dev \
libpq-dev \
postgresql-9.4 \
python-dev \
python-pip \
python-virtualenv \
strace \
supervisor
# Grab gosu for easy step-down from root
RUN gpg --keyserver pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates wget && rm -rf /var/lib/apt/lists/* \
&& wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture)" \
&& wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture).asc" \
&& gpg --verify /usr/local/bin/gosu.asc \
&& rm /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu \
&& apt-get purge -y --auto-remove ca-certificates wget
# make the "en_US.UTF-8" locale so postgres will be utf-8 enabled by default
RUN apt-get update && apt-get install -y locales && rm -rf /var/lib/apt/lists/* \
&& localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
ENV LANG en_US.utf8
# Adjust PostgreSQL configuration so that remote connections to the database are possible.
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/9.4/main/pg_hba.conf
# And add ``listen_addresses`` to ``/etc/postgresql/9.4/main/postgresql.conf``
RUN echo "listen_addresses='*'" >> /etc/postgresql/9.4/main/postgresql.conf
RUN echo "log_directory='/var/log/postgresql'" >> /etc/postgresql/9.4/main/postgresql.conf
# Add all code from the project and all config files
WORKDIR /home/projects/my-project
COPY . .
# Add VOLUMEs to allow backup of config, logs and databases
ENV PGDATA /var/lib/postgresql/data
VOLUME /var/lib/postgresql/data
# Expose an entrypoint and a port
RUN chmod +x scripts/sh/*
EXPOSE 5432
ENTRYPOINT ["scripts/sh/entrypoint-postgres.sh"]
And entrypoint script:
echo " I am " && gosu postgres whoami
gosu postgres /etc/init.d/postgresql start && echo 'Started postgres'
gosu postgres psql --command "CREATE USER myuser WITH SUPERUSER PASSWORD 'mypassword';" && echo 'Created user'
gosu postgres createdb -O myuser mydb && echo 'Created db'
# This just keeps the container alive.
tail -F /var/log/postgresql/postgresql-9.4-main.log
In the end, it seems that the real problem was the fact that I was trying to create the database from my entrypoint script.
Things such as creating a db or a user should be done at container creation time so I ended up using the standard Postgres image, which actually provides a simple and easy way to create an user and a db.
This is the fully functional configuration file for Postgres.
apiVersion: v1
kind: Pod
metadata:
name: postgres
labels:
name: postgres
spec:
containers:
- name: postgres
image: postgres
env:
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- name: POSTGRES_USER
value: myuser
- name: POSTGRES_PASSWORD
value: mypassword
- name: POSTGRES_DB
value: mydb
ports:
- containerPort: 5432
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: pg-data
volumes:
- name: pg-data
persistentVolumeClaim:
claimName: pg-data-claim
Thanks to all those who helped me :)
does your custom postgresql persist data at /var/lib/postgresql/data?
are you able to get logs from your postgresql container and spot anything interesting?
when your pod is running, can you see the mountpoints inside your container and check the persistent disk is there?
I followed this scenario and I was able to persist my data by changing the mountPath to /var/lib/postgresql and also reproduced using cassandra (i.e. /var/lib/cassandra for mountPath)
I was able to delete/restart pods from different nodes/hosts and still see my "users" table and the data I previously entered. However, I was not using a custom image, I just used standard docker images.