Connection refused inside kubernetes cron jobs using snx vpn and paramiko sftp - kubernetes

I run a python script to download file via sftp using vpn snx vpn and sftp paramiko. I invoke the script via cronjobs,
Here are my cronjobs script:
apiVersion: batch/v1
kind: CronJob
metadata:
name: file-uploader-a
labels:
app: file-uploader
spec:
schedule: "*/1 0-10 * * *"
jobTemplate:
spec:
parallelism: 1 # How many pods will be instantiated at once.
completions: 1 # How many containers of the job are instantiated one after the other (sequentially) inside the pod.
backoffLimit: 5 # Maximum pod restarts in case of failure
template:
spec:
containers:
- name: file-uploader-a
image: image-a
imagePullPolicy: IfNotPresent
envFrom:
- configMapRef:
name: file-env
- secretRef:
name: file-secret
securityContext:
capabilities:
add:
- CAP_NET_ADMIN
- CAP_SYS_MODULE
command:
- sh
- "-c"
- ". /root/.venv/bin/activate && python -m python.module.a"
restartPolicy: OnFailure
terminationGracePeriodSeconds: 8
My Docker file
FROM ubuntu:18.04
ADD scripts/snx_install_800010013.sh /root
ADD scripts/SINAR33-exp-13May2022.pfx /root
ADD scripts/post_install.sh /root
ADD scripts/init_snx.sh /root
ADD requirements.txt /root
RUN cd root && mkdir bss_uploader
RUN cd root/bss_uploader && mkdir temp
ADD bss_uploader /root/bss_uploader
ARG SNX_SERVER
ARG FTP_HOST
ARG DEBIAN_FRONTEND=noninteractive
RUN dpkg --add-architecture i386 && apt-get update && \
apt-get install bzip2 kmod libstdc++5:i386 \
libpam0g:i386 libx11-6:i386 expect iptables \
net-tools iputils-ping iproute2 python3-venv \
linux-modules-5.4.0-1063-aws python3-pip \
software-properties-common tmux openssh-client -y
RUN cd /usr/bin && ln -s python3 python
WORKDIR /root
RUN bash -x snx_install_800010013.sh
RUN bash -x post_install.sh $SNX_SERVER $FTP_HOST
post_install.sh script
#!/bin/bash
SNX_SERVER=$1
FTP_HOST=$2
mkdir ~/.ssh && touch ~/.ssh/config
echo -e "Host $FTP_HOST\n\tStrictHostKeyChecking no\n\nHost $SNX_SERVER\n\tStrictHostKeyChecking no" >> ~/.ssh/config
chmod 644 ~/.ssh/config
uname=$(uname -r)
mkdir /lib/modules/$uname
# move kernel modules installed to current
cp -a /lib/modules/5.4.0-1063-aws/. /lib/modules/$uname/
modprobe tun
python -m venv .venv
. .venv/bin/activate && pip install --upgrade pip && pip install -r requirements.txt
init_snx.sh script to init on first run
#!/bin/bash
iptables -t nat -A POSTROUTING -o tunsnx -j MASQUERADE
iptables -A FORWARD -i eth0 -j ACCEPT
SNX_SERVER=$1
SNX_PASSWORD=$2
SNX_COMMAND="snx -s $SNX_SERVER -c /root/SINAR33-exp-13May2022.pfx -g"
/usr/bin/expect <<EOF
spawn $SNX_COMMAND
expect "*?assword:"
send "$SNX_PASSWORD\r"
expect "*Do you accept*"
send "y\r"
expect "SNX - connected."
spawn sleep 4
expect "Waiting up to*"
spawn snx -d
expect "SNX - Disconnecting*"
spawn sleep 2
expect "Waiting up to*"
EOF
When I try to run the script via CronJobs, I got connection refused error while connecting to SFTP.
But when i try to run manualy from docker-container (via cli docker container) i got succeed
docker run --name xt_up --cap-add=ALL -t -d image:latest
I already tried to add networkPolicies.egress but still got no luck
could you please help me regarding this ?
thank you and sorry for my bad english

Related

/bin/bash: line 123: kubectl: command not found

This is my first time using GitLab for EKS and I feel so lost. I've been following the docs and so far I
Created a project on GitLab that contains my kubernetes manifest files
Created a config.yaml in that project in the directory .gitlab/agents/stockagent
Here's the config.yaml, my project name is "Stock-Market-API-K8s" and my k8s manifests are in the root directory of that project
ci_access:
projects:
- id: "root/Stock-Market-API-K8s"
In my root directory of my project, I also have a .gitlab-ci.yml file and here's the contents of that
deploy:
image:
name: mpriv32/stock-api:latest
entrypoint: ['']
script:
- kubectl config get-contexts
- kubectl config use-context .gitlab/agents/stockagent
- kubectl get pods
Using the default example from the docs, it seems that the get-contexts script is the one that failed. Here's the full error from my logs
Executing "step_script" stage of the job script
00:01
Using docker image sha256:58ddf823e9d7ee4c0e75779b7e01dab9b11ac0d985d1b2d2fe6c6b95a849573d for mpriv32/stock-api:latest with digest mpriv32/stock-api#sha256:a2e79a2c3a57327f93e36ec55297a606626e4dc8d72e469dd4dc2f3c1f589bac ...
$ kubectl config get-contexts
/bin/bash: line 123: kubectl: command not found
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1
Here's my job.yaml file for my kubernetes pod, just in case it plays a factor at all
apiVersion: v1
kind: Pod
metadata:
name: stock-api
labels:
app: stock-api
spec:
containers:
- name: stock-api
image: mpriv32/stock-api:latest
envFrom:
- secretRef:
name: api-credentials
restartPolicy: Never
In your case, I guess the image(mpriv32/stock-api:latest) that you are using doesn't have a dependency kubectl as a global executable, please use an image as an example - bitnami/kubectl which "contains" kubectl
deploy:
image:
name: bitnami/kubectl
the image keyword is the name of the Docker image the Docker executor uses to run CI/CD jobs.
For more information https://docs.gitlab.com/ee/ci/docker/using_docker_images.html
Or you can build your docker image on top of bitnami/kubectl
FROM bitnami/kubectl:1.20.9 as kubectl
FROM ubuntu-or-whatever-image:tag
# Do whatever you need to with the
# ubuntu-or-whatever-image:tag image, then:
COPY --from=kubectl /opt/bitnami/kubectl/bin/kubectl /usr/local/bin/
Or you can go with the approach of building an image from the scratch by
installing there the dependencies that you are using
smth like
FROM ubuntu:18.10
WORKDIR /root
COPY bootstrap.sh ./
RUN apt-get update && apt-get -y install --no-install-recommends \
gnupg \
curl \
wget \
git \
apt-transport-https \
ca-certificates \
zsh \
&& rm -rf /var/lib/apt/lists/*
ENV SHELL /usr/bin/zsh
# Install kubectl
RUN curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list && \
apt-get update && apt-get -y install --no-install-recommends kubectl

Permission denied executing jar file downloaded to .Ivy folder in Kubernetes

I created a dockerfile and changed User to a non-root user nobody. Locally this works perfectly. When deployed on kubernetes howerever, I get error
java.nio.file.AccessDeniedException: ./xxxxxx_2.12-2.6.3.jar
When I dug in more I realised this jar file is download after the spark dependencies used in the dockerfiles are downloaded. Therefore any permissions given to the spark folder are not present for this newly dowloaded jar file which is downloaded at runtime into /opt/spark/.ivy2/xxx which has root permissions. This causes the pod in kubernetes to fail.
I am wondering if there is way to give permissions to execute this jar file. Since it seems this is not possible in the Dockerfile. Any suggestion as to how to solve this issue ??
As proposed by #mario
ARG SPARK_OPERATOR_BASE_IMAGE_VERSION=v2.4.5
FROM xxx/alpine:3.12 as preparator
ARG SCALA_VERSION=2.12
ARG SPARK_VERSION=2.4.7
ARG HADOOP_VERSION=3.2.1
ARG AWS_SDK_VERSION=1.11.375
ARG MAVEN_VERSION=3.6.2
RUN apk add --no-cache \
bash \
curl && \
mkdir /target
COPY hashes /tmp/
COPY prepare /tmp/
WORKDIR /tmp
# Download Hadoop
RUN curl -L -O https://downloads.apache.org/hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz && \
sha256sum -c hadoop-${HADOOP_VERSION}.sha256 && \
tar -xzf hadoop-${HADOOP_VERSION}.tar.gz && \
mv hadoop-${HADOOP_VERSION} /target/hadoop
# Download Spark
RUN curl -L -O https://downloads.apache.org/spark/spark-${SPARK_VERSION}/spark-${SPARK_VERSION}-bin-without-hadoop-scala-${SCALA_VERSION}.tgz && \
sha512sum -c spark-${SPARK_VERSION}-bin-without-hadoop-scala-${SCALA_VERSION}.sha512 && \
tar -xzf spark-${SPARK_VERSION}-bin-without-hadoop-scala-${SCALA_VERSION}.tgz && \
mv spark-${SPARK_VERSION}-bin-without-hadoop-scala-${SCALA_VERSION} /target/spark && \
# Download Spark 3.0.0 entrypoint script from GitHub, bugfixing for 2.4.7
curl -L -O https://raw.githubusercontent.com/apache/spark/v3.0.0/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh && \
mv entrypoint.sh /target/entrypoint.sh && \
chmod +x /target/entrypoint.sh
# Download AWS Jars
RUN curl -L -O https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/${HADOOP_VERSION}/hadoop-aws-${HADOOP_VERSION}.jar && \
sha1sum -c hadoop-aws-${HADOOP_VERSION}.jar.sha1 && \
mv hadoop-aws-${HADOOP_VERSION}.jar /target/spark/jars/ && \
curl -L -O https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/${AWS_SDK_VERSION}/aws-java-sdk-bundle-${AWS_SDK_VERSION}.jar && \
sha1sum -c aws-java-sdk-bundle-${AWS_SDK_VERSION}.jar.sha1 && \
mv aws-java-sdk-bundle-${AWS_SDK_VERSION}.jar /target/spark/jars/
# Directory needed for saving built jars
RUN mkdir /target/spark/custom-jars/
#### Download Prometheus + Metric dependencies ####
# install java, maven and prometheus fat jar using maven (pom.xml)
RUN apk add --update openjdk8 && \
curl -L -O https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz && \
tar -xzf apache-maven-${MAVEN_VERSION}-bin.tar.gz && export PATH=./apache-maven-${MAVEN_VERSION}/bin:$PATH && \
mv prometheus-pom.xml pom.xml && mvn clean package && mv target/prometheusMetricLibs-jar-with-dependencies.jar /target/spark/custom-jars/
RUN \
chown -R nobody:99 /target/spark \
&& chown -R nobody:99 /target/hadoop \
&& chmod -R ugo+rw /target/spark \
&& chmod -R ugo+rw /target/hadoop
ARG SPARK_OPERATOR_BASE_IMAGE_VERSION
FROM gcr.io/spark-operator/spark:${SPARK_OPERATOR_BASE_IMAGE_VERSION}
RUN rm -rf /opt/spark/
COPY --from=preparator /target/ /opt/
ENV SPARK_HOME=/opt/spark \
HADOOP_HOME=/opt/hadoop
ENV HADOOP_OPTS="-Djava.library.path=/opt/hadoop/lib/native" \
LD_LIBRARY_PATH=${HADOOP_HOME}/lib/native \
PATH=${HADOOP_HOME}/bin:${SPARK_HOME}/bin:${PATH}
COPY conf /opt/spark/conf/
RUN echo "export JAVA_HOME=${JAVA_HOME}" >> ${HADOOP_HOME}/etc/hadoop/hadoop-env.sh && \
echo "export JAVA_HOME=${JAVA_HOME}" > ${SPARK_HOME}/conf/spark-env.sh && \
echo "export SPARK_DIST_CLASSPATH=\$(hadoop classpath)" >> ${SPARK_HOME}/conf/spark-env.sh
# 99 used instead of nobody because in the alpine image the Group Id of nobody is different than in the spark operator image.
RUN \
addgroup --gid 99 nobody \
&& echo "nobody:x:99:99:nobody:/nonexistent:/usr/sbin/nologin" >> /etc/passwd \
&& usermod -a -G users nobody \
&& chmod -R ugo+rw /var/lib/
USER nobody
# if we want local storage
# "spark.eventLog.dir": "tmp/spark-events"
RUN mkdir -p /tmp/spark-events
in my pod the jar file is implemented like this
sparkConf:
"spark.ui.port": "4045"
"spark.eventLog.enabled": {{ .Values.spark.eventLogEnabled | quote }}
"spark.eventLog.dir": "xx//"
"spark.jars.ivySettings": "/vault/secrets/xx-ivysettings.xml"
"spark.jars.ivy": "/opt/spark/.ivy2"
"spark.jars.packages": "xxxx_2.12:{{ .Values.appVersion }}"
"spark.blacklist.enabled": "false"
"spark.driver.supervise": "true"
"spark.app.name": {{ .Values.name | quote }}
"spark.submit.deployMode": {{ .Values.spark.deployMode | quote }}
"spark.driver.extraJavaOptions": "-Dlog4j.configurationFile=log4j.properties"
"spark.executor.extraJavaOptions": "-Dlog4j.configurationFile=log4j.properties"
It would be easier to answer your question if you share your Dockerfile and your kubernetes Pod template yaml manifest, but in short: you can manipulate your Pod's permissions using the securityContext like in the example from the docs below:
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: busybox
command: [ "sh", "-c", "sleep 1h" ]
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
allowPrivilegeEscalation: false
For debugging purposes you can start from setting allowPrivilegeEscalation to true or runAsUser: 0 (root) but keep in mind this is not the solution that can be used in production. Running containers as root is generally a bad idea and in most cases it can be avoided.
Therefore any permissions given to the spark folder are not present
for this newly dowloaded jar file which is downloaded at runtime into
/opt/spark/.ivy2/xxx which has root permissions.
Is it really necessary for it to have root permissions ? Most likely it can be fixed in your Dockerfile.

how to mount a path as non root user in kubernetes

I deployed a mysql monitor application image in kubernetes cluster which run as non root user. When I tried to mount a path to make the data persistent,its overriding the directory(creates a new directory by deleting everything inside that path) in which my application configuration files has to be present.Even I tried using init container still,i am not able to mount it.
my docker file:
FROM centos:7
ENV DIR /binaries
ENV PASS admin
WORKDIR ${DIR}
COPY libstdc++-4.8.5-39.el7.x86_64.rpm ${DIR}
COPY numactl-libs-2.0.12-3.el7.x86_64.rpm ${DIR}
COPY mysqlmonitor-8.0.18.1217-linux-x86_64-installer.bin ${DIR}
RUN yum install -y libaio && yum -y install gcc && yum -y install gcc-c++ && yum -y install compat-libstdc++-33 && yum -y install libstdc++-devel && yum -y install elfutils-libelf-devel && yum -y install glibc-devel && yum -y install libaio-devel && yum -y install sysstat
RUN yum install -y gcc && yum install -y make && yum install -y apr-devel && yum install -y openssl-devel && yum install -y java
RUN rpm -ivh numactl-libs-2.0.12-3.el7.x86_64.rpm
RUN useradd sql
RUN chown sql ${DIR}
RUN chmod 777 ${DIR}
RUN chmod 755 /home/sql
USER sql
WORKDIR ${DIR}
RUN ./mysqlmonitor-8.0.18.1217-linux-x86_64-installer.bin --installdir /home/sql/mysql/enterprise/monitor --mode unattended --tomcatport 18080 --tomcatsslport 18443 --adminpassword ### --dbport 13306
RUN rm -rf /binaries/*
VOLUME /home/mysql/mysql/enterprise/monitor/mysql/data
ENTRYPOINT ["/bin/bash", "-c", "/home/sql/mysql/enterprise/monitor/mysqlmonitorctl.sh start && tail -f /home/sql/mysql/enterprise/monitor/apache-tomcat/logs/mysql-monitor.log"]
my deployment file
apiVersion: apps/v1
kind: Deployment
metadata:
name: mypod
spec:
replicas: 1
selector:
matchLabels:
app: mem
template:
metadata:
labels:
app: mem
spec:
containers:
- name: mem
image: 22071997/mem
command:
volumeMounts:
- mountPath: /home/sql/mysql/enterprise/monitor/mysql/data
name: volume
volumes:
- name: volume
persistentVolumeClaim:
claimName: mem-pvc1
initContainers:
- name: permissionsfix
image: alpine:latest
command: ["/bin/sh", "-c"]
args:
- chown -R 1000:1000 /home/sql/mysql/enterprise/monitor/ && chmod -R 777 /home/sql/mysql/enterprise/monitor/ ;
volumeMounts:
- name: volume
mountPath: /home/sql/mysql/enterprise/monitor
output:
[sql#mypod-775764db45-bzs8n enterprise]$ cd monitor/mysql
[sql#mypod-775764db45-bzs8n mysql]$ ls
LICENSE LICENSE.router README.meb bin docs lib my-large.cnf my-small.cnf new runtime support-files var
LICENSE.meb README README.router data include man my-medium.cnf my.cnf run share tmp
[sql#mypod-775764db45-bzs8n mysql]$ cd data
[sql#mypod-775764db45-bzs8n data]$ ls
mypod-775764db45-bzs8n.err
This doesn't seem related to mounting as a non-root user, but more so that mounting a volume over an existing directory will result in that directory looking as if it is empty (or containing whatever happens to be on the volume already). If you have configuration stored on a non-volume that you would like to be on the volume, then you will need to mount the volume to a different location (so it doesn't overwrite your local configuration) and copy that configuration to the mounted volume location. You can do this in an init container, but be careful not to overwrite the volume contents on every startup of the container.

Symfony app in Docker doesn't work

I created an app in Symfony with MongoDB and I added that in a Docker image.
Image for MongoDB works good with message: 2017-04-19T12:47:33.936+0000 I NETWORK [initandlisten] waiting for connections on port 27017
But image for app dosen't work I receive the message:
stdin: is not a tty
hello
when I call the instruction: docker run docker_web_server:latest
I use for that docker-compose file:
web_server:
build: web_server/
ports:
- 5000:5000
links:
- mongo
tty: true
environment:
SYMFONY__MONGO_ADDRESS: mongo
SYMFONY__MONGO_PORT: 27017
mongo:
image: mongo:3.0
container_name: mongo
command: mongod --smallfiles
expose:
- 27017
And Dockerfile for app is:
FROM ubuntu:14.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && apt-get install -y \
git \
curl \
php5-cli \
php5-json \
php5-intl
RUN curl -sS https://getcomposer.org/installer | php
RUN mv composer.phar /usr/local/bin/composer
ADD entrypoint.sh /entrypoint.sh
ADD ./code /var/www
WORKDIR /var/www
#RUN chmod +x /entrypoint.sh
ENTRYPOINT [ "/bin/bash", "/entrypoint.sh" ]
entrypoint.sh
#!/bin/bash
rm -rf /var/www/app/cache/*
exec php -S 0.0.0.0:8000 # This will run a web-server on port 8000
What is the problem?
I call wrong the server from docker image?
I expected to have the message:
[OK] Server running on http://127.0.0.1:8000
You should remove CMD ['echo', 'hello'] from your Dockerfile, has this is being passed as a parameter to your ENTRYPOINT
You should also add tty: true to your service definition, web-server
I'm hoping entrypoint.sh runs php -S 0.0.0.0:8000 at end. Please post that for further advice. If you do use php -S inside of it, prefix it with exec to get it to take over as the main process.
Edit since new information added:
I'd modify the entrypoint.sh to:
#!/bin/bash
rm -rf /var/www/app/cache/*
php -S 0.0.0.0:8000 # This will run a web-server on port 8000
I'd get rid of the symfony_environment.sh and instead, add the following to your web-server service:
environment:
SYMFONY__MONGO_ADDRESS: mongo
SYMFONY__MONGO_PORT: 27017
As a side-note, I contribute to a project called boilr that generates boilerplates, like this. I've even created a template for php/docker-compose projects, it would be worth checking out. I always keep it up-to-date with the best practices.
https://github.com/rawkode/boilr-docker-compose-php

How to persist data using a postgres database, Docker, and Kubernetes?

I am trying to mount a persistent disk on my container which runs a Postgres custom image. I am using Kubernetes and following this tutorial.
This is my db_pod.yaml file:
apiVersion: v1
kind: Pod
metadata:
name: lp-db
labels:
name: lp-db
spec:
containers:
- image: my_username/my-db
name: my-db
ports:
- containerPort: 5432
name: my-db
volumeMounts:
- name: pg-data
mountPath: /var/lib/postgresql/data
volumes:
- name: pg-data
gcePersistentDisk:
pdName: my-db-disk
fsType: ext4
I create the disk using the command gcloud compute disks create --size 200GB my-db-disk.
However, when I run the pod, delete it, and then run it again (like in the tutorial) my data is not persisted.
I tried multiple versions of this file, including with PersistentVolumes and PersistentVolumeClaims, I tried changing the mountPath, but to no success.
Edit
Dockerfile for creating the Postgres image:
FROM ubuntu:trusty
RUN rm /bin/sh && \
ln -s /bin/bash /bin/sh
# Get Postgres
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
RUN apt-get update && \
apt-get install -y wget
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
# Install virtualenv (will be needed later)
RUN apt-get update && \
apt-get install -y \
libjpeg-dev \
libpq-dev \
postgresql-9.4 \
python-dev \
python-pip \
python-virtualenv \
strace \
supervisor
# Grab gosu for easy step-down from root
RUN gpg --keyserver pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates wget && rm -rf /var/lib/apt/lists/* \
&& wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture)" \
&& wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture).asc" \
&& gpg --verify /usr/local/bin/gosu.asc \
&& rm /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu \
&& apt-get purge -y --auto-remove ca-certificates wget
# make the "en_US.UTF-8" locale so postgres will be utf-8 enabled by default
RUN apt-get update && apt-get install -y locales && rm -rf /var/lib/apt/lists/* \
&& localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
ENV LANG en_US.utf8
# Adjust PostgreSQL configuration so that remote connections to the database are possible.
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/9.4/main/pg_hba.conf
# And add ``listen_addresses`` to ``/etc/postgresql/9.4/main/postgresql.conf``
RUN echo "listen_addresses='*'" >> /etc/postgresql/9.4/main/postgresql.conf
RUN echo "log_directory='/var/log/postgresql'" >> /etc/postgresql/9.4/main/postgresql.conf
# Add all code from the project and all config files
WORKDIR /home/projects/my-project
COPY . .
# Add VOLUMEs to allow backup of config, logs and databases
ENV PGDATA /var/lib/postgresql/data
VOLUME /var/lib/postgresql/data
# Expose an entrypoint and a port
RUN chmod +x scripts/sh/*
EXPOSE 5432
ENTRYPOINT ["scripts/sh/entrypoint-postgres.sh"]
And entrypoint script:
echo " I am " && gosu postgres whoami
gosu postgres /etc/init.d/postgresql start && echo 'Started postgres'
gosu postgres psql --command "CREATE USER myuser WITH SUPERUSER PASSWORD 'mypassword';" && echo 'Created user'
gosu postgres createdb -O myuser mydb && echo 'Created db'
# This just keeps the container alive.
tail -F /var/log/postgresql/postgresql-9.4-main.log
In the end, it seems that the real problem was the fact that I was trying to create the database from my entrypoint script.
Things such as creating a db or a user should be done at container creation time so I ended up using the standard Postgres image, which actually provides a simple and easy way to create an user and a db.
This is the fully functional configuration file for Postgres.
apiVersion: v1
kind: Pod
metadata:
name: postgres
labels:
name: postgres
spec:
containers:
- name: postgres
image: postgres
env:
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- name: POSTGRES_USER
value: myuser
- name: POSTGRES_PASSWORD
value: mypassword
- name: POSTGRES_DB
value: mydb
ports:
- containerPort: 5432
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: pg-data
volumes:
- name: pg-data
persistentVolumeClaim:
claimName: pg-data-claim
Thanks to all those who helped me :)
does your custom postgresql persist data at /var/lib/postgresql/data?
are you able to get logs from your postgresql container and spot anything interesting?
when your pod is running, can you see the mountpoints inside your container and check the persistent disk is there?
I followed this scenario and I was able to persist my data by changing the mountPath to /var/lib/postgresql and also reproduced using cassandra (i.e. /var/lib/cassandra for mountPath)
I was able to delete/restart pods from different nodes/hosts and still see my "users" table and the data I previously entered. However, I was not using a custom image, I just used standard docker images.