How to run multiple commands with gosu in Kubernetes job - kubernetes

I am defining a Kubernetes job to run a rake task but stuck in how to write the command...
I am new to K8s and trying to run a Rails application in K8s.
In my Rails app Dockerfile, I created a user , copied code to /home/abc and installed rvm and rails in this user, and also specified an entrypoint and command:
ENTRYPOINT ["/home/abc/docker-entrypoint.sh"]
CMD bash -l -c "cd /home/abc && rvm use 2.2.10 --default && rake db:migrate && exec /usr/bin/supervisord -c config/supervisord.conf"
In docker-entrypoint.sh, the last command is
exec gosu abc "$#"
The goal is to at the end, gosu to user abc, and then run db migration and start the server through supervisord. It works, although I dont know whether it is a good practice or not...
Now I would like to run another rake task for some purpose.
Firstly, I tried to run it using kubectl exec command:
kubectl exec my-app-deployment-xxxx -- gosu abc bash -l -c 'cd /home/abc && rvm use 2.2.10 --default && rake app:init:test_task'
It worked, but it requires to know the pod id, which is dynamic. so I tried to create a K8s job and specify in the command:
containers:
- name: my-app
image: my-app:v0.2
command:
- "gosu"
- "abc"
- "bash -l -c 'cd /home/abc && rvm use 2.2.10 --default && rake app:init:test_task'"
I expect the job can be completed successfully, but it failed, and the error info when kubectl logs job_pod is like:
error: exec: "bash -l -c 'cd /home/abc && rvm use 2.2.10 --default && rake app:init:test_task'": stat bash -l -c cd /home/abc && rvm use 2.2.10 --default && rake app:init:test_task': no such file or directory
I think it should be because of how to write the 'command' part to run multiple commands with gosu...
Thanks for your help!

Since gosu takes the user name and the Bash shell as arguments, I'd say that this is one rather than three separate commands.
Given that, there can be only one single entrypoint in each container, you can try running it as follows:
containers:
- name: my-app
image: my-app:v0.2
command: ["/bin/sh", "-c", "gosu username bash -l -c 'cd /home/abc && rvm use 2.2.10 --default && rake app:init:test_task'"]
Notice that you have to spawn a new TTY in order to run the command as the image's entrypoint is replaced when running commands in the container spec in Kubernetes.

Related

How to pass multiple commands to docker-compose.uffizzi.yml

I need to pass the next command to my service in docker-compose.uffizzi.yml
bundle exec rails db:create db:migrate db:seed && bundle exec rails s -b 0.0.0.0 -p 3000
According to this doc: https://docs.uffizzi.com/references/compose-spec/#command
command can be passed as usual or converted to an array of strings.
But when I use it in such way I get the next error:
Error: failed to create containerd task: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "bundle exec rails db:create db:migrate db:seed && bundle exec rails s -b 0.0.0.0 -p 3000": executable file not found in $PATH: unknown
But if I use only one of the commands
bundle exec rails db:create db:migrate db:seed
or only
bundle exec rails s -b 0.0.0.0 -p 3000
it works fine. But I need both of them in my service command.
Do you have any ideas how to write this command in a right way?)
The right answer is to use command this way:
command: ["bash", "-c", "bundle exec rails db:create db:migrate db:seed && bundle exec rails s -b 0.0.0.0 -p 3000"]

How can one prevent variable substitution/expension in AWS Fargate container definition command

Locally when running docker with docker run I pass some arguments like:
docker run -p 8080:80 -e "SERVICE_B_URL=somehost.co.uk" -d mynginx:latest /bin/sh -c "envsubst '\${SERVICE_B_URL}' < /etc/nginx/conf.d/default.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
This works fine. In my /etc/nginx/conf.d/default.conf the string ${SERVICE_B_URL} is replaced with somehost.co.uk.
When running on AWS fargate with a definition like:
"environment": [
{
"name": "SERVICE_B_URL",
"value": "someotherhost.co.uk"
}
],
"command": [
"/bin/sh",
"-c",
"envsubst '\\${SERVICE_B_URL}' < /etc/nginx/conf.d/default.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
],
The \\ was to escape the \ in the JSON file.
When trying to run, the container exits with an error because NGINX is seeing the literal string ${SERVICE_B_URL}. When I inspect the container and see the command AWS used to start the container it is:
Command ["/bin/sh","-c","envsubst '\\' < /etc/nginx/conf.d/default.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"]
Notice that Fargate has attempted to expand the string '\\${SERVICE_B_URL}' before supplying it as a command to docker run. My intention is to specify that as a literal string.
Is there a way to escape this/stop expansion. I've tried things like '\\\\${SERVICE_B_URL}' -> '\\'.
Footnote, if you are wondering why I specify the '\${SERVICE_B_URL}' to envsubst instead of just using:
docker run -p 8080:80 -e "SERVICE_B_URL=somehost.co.uk" -d mynginx:latest /bin/sh -c "envsubst < /etc/nginx/conf.d/default.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
The reason is, that the file being substituted contains other NGINX configuration which makes use of variables with $ syntax. So to prevent these being replaced by envsubst I explicitly name the variable I want to be replaced. Running locally with docker run, it works like a charm...
I've ended up simplifying this by making the command we would pass to docker run part of the Dockerfile itself using CMD, e.g:
CMD ["/bin/sh","-c","envsubst '\\${SERVICE_B_URL}' < /etc/nginx/conf.d/default.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"]
Now we can remove the command configuration from the JSON file for Fargate.

Starting services at container startup

I'm trying to run 3 services at my container startup (snmpd, sshd and centengine)
As runlevel is unknown in the container, services won't start.
I built an image with this Dockerfile :
FROM centos:6.7
MAINTAINER nael <me#mail>
# Update CentOS
RUN yum -y update
# Install wget
RUN yum install -y wget
# Get Centreon Repo
RUN wget http://yum.centreon.com/standard/3.0/stable/ces-standard.repo -O /etc/yum.repos.d/ces-standard.repo
# Install Packages (SSH, sudo, Centreon Poller & Engine, SNMP)
RUN yum install -y --nogpgcheck openssh-clients openssh-server centreon-poller-centreon-engine sudo net-snmp net-snmp-utils
# Install supervisord
RUN rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
RUN yum --enablerepo=epel install -y supervisor
RUN mv -f /etc/supervisord.conf /etc/supervisord.conf.org
ADD supervisord.conf /etc/
# For sshd & centengine
EXPOSE 22 5669
# Change user password
RUN echo -e "password" | (passwd --stdin user)
# Disable PAM (causing issues while ssh login)
RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
RUN sed -ri 's/#UsePAM no/UsePAM no/g' /etc/ssh/sshd_config
# Start supervisord
CMD ["/usr/bin/supervisord"]
Here is the supervisord.conf file
[supervisord]
nodaemon=true
pidfile=/var/run/supervisord.pid
logfile=/var/log/supervisor/supervisord.log
[program:centengine]
command=service centengine start
[program:snmpd]
command=service snmpd start
[program:sshd]
command=service sshd start
But with this Dockerfile and supervisord.conf, when I start my container theses services aren't running.
What could be the problem ?
Not anymore using supervisord.
I just include a script with all the services ... start commands in the Dockerfile. When I create my container with docker run ... I just specify that I want to start it with my script.
& that's working very well.
Thanks #warmoverflow for trying to solve this.
You may find my dockerfy utility useful starting services, pre-running initialization commands before the primary command starts. See https://github.com/markriggins/dockerfy
For example:
RUN wget https://github.com/markriggins/dockerfy/releases/download/0.2.4/dockerfy-linux-amd64-0.2.4.tar.gz; \
tar -C /usr/local/bin -xvzf dockerfy-linux-amd64-*tar.gz; \
rm dockerfy-linux-amd64-*tar.gz;
ENTRYPOINT dockerfy
COMMAND --start bash -c "while false; do echo 'Ima Service'; sleep 1; done" -- \
--reap -- \
nginx
Would run a bash script as a service, echoing "Ima Service" every second, while the primary command nginx runs. If nginx exits, then the "Ima Service" script will automatically be stopped.
As an added benefit, any zombie processes left over by nginx will be automatically cleaned up.
You can also tail log files such as /var/log/nginx/error.log to stderr, edit nginx's configuration prior to startup and much more

Docker doesn't start MONGODB, and IPAddress doesn't appear, when started with other services

I have already asked this question on serverfault.com. I am asking it here too as I see different set of questions in these 2 sites (it appears like they have different databases).
I have been trying to build an OS image from Fedora unsuccessfully to start the following:
Systemd
SSHD
RabbitMQ
MongoDB
I can get the first 3 (Systemd, SSHD and RabbitMQ-Server) to work. I can also get MongoDB to work within the container. However, I cannot get MongoDB to work along with other 3 services.
In addition, IP address doesn't show up when I try to "dockerize" MongoDB.
Am I missing something in the Dockerfile?
Here is my dockerfile:
FROM fedora:20
MAINTAINER “Ashfaque” <ashfaque#email.com>
ENV container docker
RUN yum -y update; yum clean all
RUN yum -y install systemd; yum clean all; \
(cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
# Dockerizing SSH - is working
RUN yum -y install openssh-server
RUN yum -y install openssh-clients
RUN mkdir /var/run/sshd
RUN systemctl enable sshd.service
RUN echo 'root:mypassword' |chpasswd
EXPOSE 22
# Dockerizing RabbitMQ - is working
RUN yum -y install rabbitmq-server
EXPOSE 5672 15672
RUN systemctl enable rabbitmq-server
# Dockerizing MongoDB - is NOT WORKING
RUN yum -y install mongodb-server
RUN yum -y install boost
RUN yum -y install scons
# Create the MongoDB data directory
RUN mkdir -p /data/db /var/log/mongodb /var/run/mongodb
RUN sed -i 's/dbpath =\/var\/lib\/mongodb/dbpath =\/data\/db/' /etc/mongodb.conf
# Expose port 27017 from the container to the host
EXPOSE 27017
# Set usr/bin/mongod as the dockerized entry-point application
ENTRYPOINT ["/usr/bin/mongod"]
#CMD ["--port", "27017", "--dbpath", "/data/db", "--smallfiles", "--fork", "--syslog"]
#RUN /usr/bin/mongod --smallfiles --port 27017 --dbpath /data/db --fork --syslog
VOLUME ["/sys/fs/cgroup", "/data/db", "/var/log/mongodb", "/usr/bin"]
CMD ["/usr/sbin/init"]
Docker Commands used to build are:
(1) docker build -t rabbitmq_mongo_heisenbug .
(2) docker run --privileged -d -e 'container=docker' -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 29022:22 -p 29672:15672 -p 29017:27017 rabbitmq_mongo_heisenbug
or.. (3) docker run --privileged -ti -e 'container=docker' -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 29022:22 -p 29672:15672 -p 29017:27017 rabbitmq_mongo_heisenbug
You are using both ENTRYPOINT and CMD in your Dockerfile. This means, that docker will run /usr/bin/mongod with default parameter /usr/sbin/init. I'm pretty sure this is not what you want.
Docker will run as long as the command you specified is running. I'm not sure about /usr/bin/mongod, but if it runs in daemon mode (that is, spawn a process and return), then the container will stop running right away. The spawned processes will be terminated. The same is true for /usr/sbin/init or for any other command you specify. You can write a small shell script, which spawns the processes and runs one in the foreground, or you can use runit, or some similar tool.
Also, you probably don't need to run sshd in your container. See here why.

Starting a rake task as daemon

I'm trying to daemonize a rake task by running the following command (on Ubuntu 12.04)
start-stop-daemon -S --pidfile /home/dep/apps/fid/current/tmp/pids/que.pid
-u dep -d /home/dep/apps/fid/current -b -m
-a "bundle exec rake que:work RAILS_ENV=staging > /home/dep/apps/fid/current/log/que.log 2>&1"
-v
The console says
Starting bundle exec rake que:work RAILS_ENV=staging > /home/dep/apps/fid/current/log/que.log 2>&1...
Detaching to start bundle exec rake que:work RAILS_ENV=staging > /home/dep/apps/fid/current/log/que.log 2>&1...done.
but nothing happen.
the pid file is empty and no log file created.
Am I missing anything here?
Thanks.
Try to get more about the environments (and their differences) when running bundle from your normal environment and running it from start-stop-daemon.
e.g. print all env variables in both cases and adjust accordingly.