Jenkins pipeline with docker: run docker as specific user (embedded postgresql - postgresql

Hi Stack Overflow community!
I have a maven - java project which needs to be build with jenkins pipelines.
To do so, I've configured the job using the docker image maven:3.3.3. Everything works, except for the fact that I use ru.yandex.qatools.embed:postgresql-embedded. This works locally, but on jenkins it complains about starting Postgres:
2019-02-08 09:31:20.366 WARN 140 --- [ost-startStop-1] r.y.q.embed.postgresql.PostgresProcess: Possibly failed to run initdb:
initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will own the server process.
2019-02-08 09:31:40.999 ERROR 140 --- [ost-startStop-1] r.y.q.embed.postgresql.PostgresProcess: Failed to read PID file (File '/var/.../target/database/postmaster.pid' does not exist)
java.io.FileNotFoundException: File '/var/.../target/database/postmaster.pid' does not exist
Apparently, Postgres does not allow to be run with superuser privileges for security reasons.
I've tried to run as a user by creating my own version of the docker-image and adding the following to the DockerFile:
RUN useradd myuser
USER myuser
And this works when I start the docker image from the server's terminal. But by using jenkins pipeline, whoami still prints 'root', which suggests that Jenkins Pipeline uses run -u behind the schemes, which would overrule the DockerFile?
My pipeline job is currently as simple as this:
pipeline {
agent {
docker {
image 'custom-maven:1'
}
}
stages {
stage('Checkout') {
...
}
stage('Build') {
steps {
sh 'whoami'
sh 'mvn clean install'
}
}
}
}
So, my question: How do I start this docker image as a different user? Or switch users before running mvn clean install?
UPDATE:
By adding -u myuser as args in jenkins pipeline, I do log in as the correct user, but then the job can't access the jenkins-log file (and hopefully that's the only problem). The user myuser is added to the group root, but this makes no differece:
agent {
docker {
image 'custom-maven:1'
args '-u myuser'
}
}
And the error:
sh: 1: cannot create /var/.../jenkins-log.txt: Permission denied
sh: 1: cannot create /var/.../jenkins-result.txt.tmp: Permission denied
mv: cannot stat ‘/var/.../jenkins-result.txt.tmp’: No such file or directory
touch: cannot touch ‘/var/.../jenkins-log.txt’: Permission denied

I have solved the issue in our case. What I did was sudo before the mvn command. Keep in mind that every sh step has his own shell, so you need to do sudo in each sh step:
sh 'sudo -u <youruser> mvn <clean or whatever> -f <path/to/pomfile.xml>'
The user must be created in the Dockerfile. I've created it without password, but I don't think it matters since you are root...
You must use sudo instead of switching user or something since otherwise you need to provide a password.
This is far from a clean way... I would suggest to not mess with user-switching unless you really need to (like to run a embedded postgres)

#Thomas Stubbe
I got almost the same error as you mentioned above. I also have an image which has postgresql on it. On the Dockerfile I have created an user named like tdv, and use id command on Container check tdv permission was 1000:1000. I can Start the Container though Jenkins pipeline but it failed to execute sh command, even I add sudo -u tdv for each command. Did you did any other configuration?
My Jenkins Pileline script like following:
pipeline{
agent none
stages{
stage('did operation inside container'){
agent {
docker{
image 'tdv/tdv-test:8.2'
label 'docker_machine'
customWorkspace "/opt/test"
registryUrl 'https://xxxx.xxxx.com'
registryCredentialsId '8269c5cd-321e-4fab-919e-9ddc12b557f3'
args '-u tdv --name tdv-test -w /opt/test -v /opt/test:/opt/test:rw,z -v /opt/test#tmp:/opt/test#tmp:rw,z -p 9400:9400 -p 9401:9401 -p 9402:9402 -p 9403:9403 -p 9407:9407 -p 9303:9303 --cpus=2.000 -m=4g xxx.xxx.com/tdv/tdv-test:8.2 tdv.server'
}
}
steps{
sh 'sudo tdv whoami'
sh 'sudo tdv pwd'
sh 'sudo tdv echo aaa'
}
}
}
}
After the Job run, I can check the Container start up actually. but it still get error like following
$ docker top f1140072d77c5bed3ce43a5ad2ab3c4be24e8c32cf095e83c3fd01a883e67c4e -eo pid,comm
ERROR: The container started but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument, as required by official docker images (see https://github.com/docker-library/official-images#consistency for entrypoint consistency requirements).
Alternatively you can force image entrypoint to be disabled by adding option `--entrypoint=''`.
[Pipeline] {
[Pipeline] sh
sh: /opt/test#tmp/durable-081990da/jenkins-log.txt: Permission denied
sh: /opt/test#tmp/durable-081990da/jenkins-result.txt.tmp: Permission denied
touch: cannot touch ‘/opt/test#tmp/durable-081990da/jenkins-log.txt’: Permission denied
mv: cannot stat ‘/opt/test#tmp/durable-081990da/jenkins-result.txt.tmp’: No such file or directory
touch: cannot touch ‘/opt/test#tmp/durable-081990da/jenkins-log.txt’: Permission denied
touch: cannot touch ‘/opt/test#tmp/durable-081990da/jenkins-log.txt’: Permission denied

Solution
You currently have sh 'sudo tdv whoami'
I think you should add the -u flag after sudo like that:
sh 'sudo -u tdv whoami'

Related

Azure Devops Container Pipeline job is trying to redundantly give user:1000 sudo priveleges

I have a docker image, already made, that another pipeline uses for build jobs. That image already has a user:1000 with sudo (paswordless) permissions and a home directory. This was done to make manual use of the container more useful... there are applications in the image that prefer to run under a non-root user.
The pipeline using this image finds the existing user (great!) but then tries to give the user sudo permissions that it already has and this breaks the flow...
--<yaml pipeline code>--
container:
image: acr.url/foo/bar:v1
endpoint: <svc-connection>
--<pipeline run>--
...
/usr/bin/docker network create --label dc4b27 vsts_network_6b3e...
/usr/bin/docker inspect --format="{{index .Config.Labels \"com.azure.dev.pipelines.agent.handler.node.path\"}}" ***/foo/bar:v1
/usr/bin/docker create --name 9479... --label dc4b27 --network vsts_network_6b3ee... -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/opt/azagent/_work/9":"/__w/9" -v "/opt/azagent/_work/_temp":"/__w/_temp" -v "/opt/azagent/_work/_tasks":"/__w/_tasks" -v "/opt/azagent/_work/_tool":"/__t" -v "/opt/azagent/externals":"/__a/externals":ro -v "/opt/azagent/_work/.taskkey":"/__w/.taskkey" ***/foo/bar:v1 "/__a/externals/node/bin/node" -e "setInterval(function(){}, 24 * 60 * 60 * 1000);"
9056...
/usr/bin/docker start 9056...
9056...
/usr/bin/docker ps --all --filter id=9056... --filter status=running --no-trunc --format "{{.ID}} {{.Status}}"
9056... Up Less than a second
/usr/bin/docker exec 9056... sh -c "command -v bash"
/bin/bash
whoami
devops
id -u devops
1000
Try to create a user with UID '1000' inside the container.
/usr/bin/docker exec 9056... bash -c "getent passwd 1000 | cut -d: -f1 "
/usr/bin/docker exec 9056... id -u viv
1000
Grant user 'viv' SUDO privilege and allow it run any command without authentication.
/usr/bin/docker exec 9056... groupadd azure_pipelines_sudo
groupadd: Permission denied.
groupadd: cannot lock /etc/group; try again later.
##[error]Docker exec fail with exit code 10
Finishing: Initialize containers
I am OK working with user:1000 in the container as the azure agent runs on the host VM under user:1000('devops') and so the id's match inside and outside of the container, getting around a shortcoming of the docker volume mount system.
The question is: Is there a pipeline yaml method or control parameter to tell the run not to try and setup sudo permissions on the discovered user account (uid:1000) in the container?
I am getting around this issue right now by adding options: --user 0 to the container: section in the yaml script but I would prefer not to do that...
Thx.

backup postgresql from azure container instance

I created Azure Container Instance and ran postgresql in it. Mounted an azure container instance storage account. How can I start backup work, possibly by sheduler?
When I run the command
az container exec --resource-group Vitalii-demo --name vitalii-demo --exec-command "pg_dumpall -c -U postgrace > dump.sql"
I get an error error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "exec: \ "pg_dumpall -c -U postgrace > dump.sql\": executable file not found in $PATH"
I read that
Azure Container Instances currently supports launching a single process with az container exec, and you cannot pass command arguments. For example, you cannot chain commands like in sh -c "echo FOO && echo BAR", or execute echo FOO.
Perhaps there is an opportunity to run as a task? Thanks.
Unfortunately - and as you already mentioned - it's not possible to run any commands with arguments like echo FOO or chain multiple commands together with &&.
https://learn.microsoft.com/en-us/azure/container-instances/container-instances-exec#run-a-command-with-azure-cli
You should be able to run an interactive shell by using --exec-command /bin/bash.
But this will not help if you want to schedule the backups programatically.
pg_dumpall can also be configured by environment variables:
https://www.postgresql.org/docs/9.3/libpq-envars.html
You could launch your backup-container with the correct environment variables in order to connect your database service:
PGHOST
PGPORT
PGUSER
PGPASSWORD
When having these variables set, a simple pg_dumpall should totally do what you want.
Hope that helps.
UPDATE:
Yikes, even when configuring the connection via environment-variables you won't be able to state the desired output file... Sorry.
You could create your own Dockerimage with a pre-configured script for dumping your PostgreSQL-database.
Doing it that way, you can configure the output-file in your script and then simply execute the script with --exec-command dump_my_db.sh.
Keep in mind that your script has to be located somewhere in the default $PATH - e.g. /usr/local/bin.

Gitlab-runner failed to remove permission denied

I'm setting up a CI/CD pipeline with Gitlab. I've installed gitlab-runner on a Digital Ocean Ubuntu 18.04 droplet and gave permissions in /etc/sudoers to the gitlab-runner as:
gitlab-runner ALL=(ALL:ALL)ALL
The first commit to the associated repository correctly build the docker-compose (the app itself is Django+postgres), but following commits are not able to clean previous builds and fail:
Running with gitlab-runner 12.8.0 (1b659122)
on ubuntu-s-4vcpu-8gb-fra1-01 52WypZsE
Using Shell executor...
00:00
Running on ubuntu-s-4vcpu-8gb-fra1-01...
00:00
Fetching changes with git depth set to 50...
00:01
Reinitialized existing Git repository in /home/gitlab-runner/builds/52WypZsE/0/lorePieri/djangocicd/.git/
From https://gitlab.com/lorePieri/djangocicd
* [new ref] refs/pipelines/120533457 -> refs/pipelines/120533457
0072002..bd28ba4 develop -> origin/develop
Checking out bd28ba46 as develop...
warning: failed to remove app/staticfiles/admin/img/selector-icons.svg: Permission denied
warning: failed to remove app/staticfiles/admin/img/search.svg: Permission denied
warning: failed to remove app/staticfiles/admin/img/icon-alert.svg: Permission denied
warning: failed to remove app/staticfiles/admin/img/tooltag-arrowright.svg: Permission denied
warning: failed to remove app/staticfiles/admin/img/icon-unknown-alt.svg: Permission denied
This is the relevant portion of the .gitlab-ci.yml file:
image: docker:latest
services:
- docker:dind
stages:
- test
- deploy_staging
- deploy_production
step-test:
stage: test
before_script:
- export DYNAMIC_ENV_VAR=DEVELOP
only:
- develop
tags:
- develop
script:
- echo running tests in $DYNAMIC_ENV_VAR
- sudo apt-get install -y python-pip
- sudo pip install docker-compose
- sudo docker image prune -f
- sudo docker-compose -f docker-compose.yml build --no-cache
- sudo docker-compose -f docker-compose.yml up -d
- echo do tests now
- sudo docker-compose exec -T web python3 -m coverage run --source='.' manage.py test
...
What I've tried:
usermod -aG docker gitlab-runner
sudo service docker restart
The best solution for me was adding
pre_clone_script = "sudo chown -R gitlab-runner:gitlab-runner ."
into /etc/gitlab-runner/config.toml
Even if you won't have permissions after a previous job it'll set correct permissions before cleaning up the workdir and cloning the repo.
I would recommend setting a GIT_STRATEGY to none in the afflicted job.
I have had the exact same problem. Therefore I will explain how it was resolved in details.
Try finding your config.toml file and run the gitlab-runner command with root privileges, since permission denied is a very common UNIX-based operating systems error.
After finding the location of config.toml pass it:
sudo gitlab-runner run --config <absolute_location_of_config_toml>
P.S. You can find all config.toml file easily using locate config.toml command. Make sure you have already installed by executing sudo apt-get install mlocate
After facing to permission denied error, I have tried using sudo gitlab-runner run instead of gitlab-runner, but it has its own problem:
ERROR: Failed to load config stat /etc/gitlab-runner/config.toml: no such
file or directory builds=0
while executing gitlab-runner without root permissions doesn't have any config file problem.
Try implementing the ways and solutions as #Grumbanks and #vlad-Mazurkov mentioned. But they didn't work properly.
It MAY be because you write a file in cloned out codebase. What I do is simply create another directory outside of gitlab-runner directory:
WORKSPACE_DIR="/home/abcd_USER/a/b"
rm -rf $WORKSPACE_DIR
mkdir -p $WORKSPACE_DIR
cd $WORKSPACE_DIR
ls -la
git clone ..................
AND DO whatever
I never faced the issue again.

How do I handle passwords and dockerfiles?

I've created an image for docker which hosts a postgresql server. In the dockerfile, the environment variable 'USER', and I pass a constant password into the a run of psql:
USER postgres
RUN /etc/init.d/postgresql start && psql --command "CREATE USER docker WITH SUPERUSER PASSWORD 'docker';" && createdb -O docker docker
Ideally either before or after calling 'docker run' on this image, I'd like the caller to have to input these details into the command line, so that I don't have to store them anywhere.
I'm not really sure how to go about this. Does docker have any support for reading stdin into an environment variable? Or perhaps there's a better way of handling this all together?
At build time
You can use build arguments in your Dockerfile:
ARG password=defaultPassword
USER postgres
RUN /etc/init.d/postgresql start && psql --command "CREATE USER docker WITH SUPERUSER PASSWORD '$password';" && createdb -O docker docker
Then build with:
$ docker build --build-arg password=superSecretPassword .
At run time
For setting the password at runtime, you can use an environment variable (ENV) that you can evaluate in an entrypoint script (ENTRYPOINT):
ENV PASSWORD=defaultPassword
ADD entrypoint.sh /docker-entrypoint.sh
USER postgres
ENTRYPOINT /docker-entrypoint.sh
CMD ["postgres"]
Within the entrypoint script, you can then create a new user with the given password as soon as the container starts:
pg_ctl -D /var/lib/postgresql/data \
-o "-c listen_addresses='localhost'" \
-w start
psql --command "CREATE USER docker WITH SUPERUSER PASSWORD '$password';"
postgres pg_ctl -D /var/lib/postgresql/data -m fast -w stop
exec $#
You can also have a look at the Dockerfile and entrypoint script of the official postgres image, from which I've borrowed most of the code in this answer.
A note on security
Storing secrets like passwords in environment variables (both build and run time) is not incredibly secure (unfortunately, to my knowledge, Docker does not really offer any better solution for this, right now). An interesting discussion on this topic can be found in this question.
You could use environment variable in your Dockerfile and override the default value when you call docker run using -e or --env argument.
Also you will need to amend the init script to run psql command on startup referenced by the CMD instruction.

How to run a bash command as a different user in Capistrano?

How would I accomplish the following in Capistrano?
sudo su - postgres
/usr/pgsql-9.2/bin/pg_ctl status -D /var/lib/pgsql/9.2/data/
The following task doesn't work:
task :postgres_check do
on roles(:db) do in: :sequence |host|
execute "sudo su - postgres << EOF
/usr/pgsql-9.2/bin/pg_ctl status -D /var/lib/pgsql/9.2/data/
EOF"
end
end
The commands in the execute statement works in a bash script.
EDIT 1:
I also tried the following:
task :postgres_check do
on roles(:postgres_pref_db), in: :sequence do |host|
execute "/usr/pgsql-9.2/bin/pg_ctl status -D /var/lib/pgsql/9.2/data", :shell => "sudo su - postgres"
end
end
Which errors with:
DEBUG [68eb95f2] Command: /usr/pgsql-9.2/bin/pg_ctl status -D /var/lib/pgsql/9.2/data
DEBUG [68eb95f2] pg_ctl: could not open PID file "/var/lib/pgsql/9.2/data/postmaster.pid": Permission denied
cap aborted!
SSHKit::Command::Failed: /usr/pgsql-9.2/bin/pg_ctl status -D /var/lib/pgsql/9.2/data stdout: Nothing written
It appears that it still executing the command as the ssh user.
I came across this and explored the answer for myself. I wouldn't have accepted the answer either to I'll provide what I did.
task :copy_files do
on roles(:web) do |host|
as 'other_user' do
execute "whoami"
end
end
end
Capistrano 3 uses SSH KIT and I found these examples really helpful for getting bash commands to work inside my tasks.
https://github.com/capistrano/sshkit/blob/master/EXAMPLES.md
You'll want to checkout ssh kit and see about on(), within(), with(), as() ... they can be used nested in any order. So you end up having a lot of control even if it takes a few minutes to learn.
I think for your specific example you will want to use as() and within() to become the postgres user and run commands within a certain directory.
Also I had to disable requiretty on my /etc/sudoers for my deploy user.