I'm setting up a CI/CD pipeline with Gitlab. I've installed gitlab-runner on a Digital Ocean Ubuntu 18.04 droplet and gave permissions in /etc/sudoers to the gitlab-runner as:
gitlab-runner ALL=(ALL:ALL)ALL
The first commit to the associated repository correctly build the docker-compose (the app itself is Django+postgres), but following commits are not able to clean previous builds and fail:
Running with gitlab-runner 12.8.0 (1b659122)
on ubuntu-s-4vcpu-8gb-fra1-01 52WypZsE
Using Shell executor...
00:00
Running on ubuntu-s-4vcpu-8gb-fra1-01...
00:00
Fetching changes with git depth set to 50...
00:01
Reinitialized existing Git repository in /home/gitlab-runner/builds/52WypZsE/0/lorePieri/djangocicd/.git/
From https://gitlab.com/lorePieri/djangocicd
* [new ref] refs/pipelines/120533457 -> refs/pipelines/120533457
0072002..bd28ba4 develop -> origin/develop
Checking out bd28ba46 as develop...
warning: failed to remove app/staticfiles/admin/img/selector-icons.svg: Permission denied
warning: failed to remove app/staticfiles/admin/img/search.svg: Permission denied
warning: failed to remove app/staticfiles/admin/img/icon-alert.svg: Permission denied
warning: failed to remove app/staticfiles/admin/img/tooltag-arrowright.svg: Permission denied
warning: failed to remove app/staticfiles/admin/img/icon-unknown-alt.svg: Permission denied
This is the relevant portion of the .gitlab-ci.yml file:
image: docker:latest
services:
- docker:dind
stages:
- test
- deploy_staging
- deploy_production
step-test:
stage: test
before_script:
- export DYNAMIC_ENV_VAR=DEVELOP
only:
- develop
tags:
- develop
script:
- echo running tests in $DYNAMIC_ENV_VAR
- sudo apt-get install -y python-pip
- sudo pip install docker-compose
- sudo docker image prune -f
- sudo docker-compose -f docker-compose.yml build --no-cache
- sudo docker-compose -f docker-compose.yml up -d
- echo do tests now
- sudo docker-compose exec -T web python3 -m coverage run --source='.' manage.py test
...
What I've tried:
usermod -aG docker gitlab-runner
sudo service docker restart
The best solution for me was adding
pre_clone_script = "sudo chown -R gitlab-runner:gitlab-runner ."
into /etc/gitlab-runner/config.toml
Even if you won't have permissions after a previous job it'll set correct permissions before cleaning up the workdir and cloning the repo.
I would recommend setting a GIT_STRATEGY to none in the afflicted job.
I have had the exact same problem. Therefore I will explain how it was resolved in details.
Try finding your config.toml file and run the gitlab-runner command with root privileges, since permission denied is a very common UNIX-based operating systems error.
After finding the location of config.toml pass it:
sudo gitlab-runner run --config <absolute_location_of_config_toml>
P.S. You can find all config.toml file easily using locate config.toml command. Make sure you have already installed by executing sudo apt-get install mlocate
After facing to permission denied error, I have tried using sudo gitlab-runner run instead of gitlab-runner, but it has its own problem:
ERROR: Failed to load config stat /etc/gitlab-runner/config.toml: no such
file or directory builds=0
while executing gitlab-runner without root permissions doesn't have any config file problem.
Try implementing the ways and solutions as #Grumbanks and #vlad-Mazurkov mentioned. But they didn't work properly.
It MAY be because you write a file in cloned out codebase. What I do is simply create another directory outside of gitlab-runner directory:
WORKSPACE_DIR="/home/abcd_USER/a/b"
rm -rf $WORKSPACE_DIR
mkdir -p $WORKSPACE_DIR
cd $WORKSPACE_DIR
ls -la
git clone ..................
AND DO whatever
I never faced the issue again.
Related
I have a multi-stage Docker container where I build jsonlog.
# Build stage
FROM postgres:11.5 AS build-env
ADD . /jsonlog
RUN apt-get update && apt-get install -y build-essential libkrb5-dev libssl-dev libpq-dev postgresql-server-dev-all
RUN cd /jsonlog && make install
# Used find to see where the output files are being installed
# RUN find / -name "*json*"
# Final stage
FROM postgres:11.5-alpine
COPY --from=build-env /usr/lib/postgresql/11/lib/bitcode/jsonlog/jsonlog.bc /usr/lib/postgresql/11/lib/bitcode/jsonlog/jsonlog.bc
COPY --from=build-env /usr/lib/postgresql/11/lib/bitcode/jsonlog.index.bc /usr/lib/postgresql/11/lib/bitcode/jsonlog.index.bc
COPY --from=build-env /usr/lib/postgresql/11/lib/jsonlog.so /usr/lib/postgresql/11/lib/jsonlog.so
The docker build seems to be okay. However, when I do a docker run, I get this error.
LOG: invalid value for parameter "log_destination": "jsonlog"
DETAIL: Unrecognized key word: "jsonlog".
FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
The jsonlog code comes from this repo, https://github.com/michaelpq/pg_plugins/tree/master/jsonlog, which I cloned. I mount just the jsonlog directory into the build container.
The postgresql.conf is vanilla, except for these lines.
log_destination = 'jsonlog'
logging_collector = on
shared_preload_libraries = 'jsonlog'
The reason for doing a multi-stage container is to get rid of build dependencies and have a smaller container. I'd like to run postgres:11.5-alpine, but I also get errors with the postgres:11.5 image.
Update
I tried getting rid of the multi-stage and just keeping everything in 1 container like this.
FROM postgres:11.5
ADD . /jsonlog
RUN apt-get update && apt-get install -y build-essential libkrb5-dev libssl-dev libpq-dev postgresql-server-dev-all
RUN cd /jsonlog && make install
But that results in the exact Unrecognized key word: "jsonlog" error.
Also, also, I tried building in a Postgres 9.5 container and the same thing happens.
It's my understanding that you simply leave log_destination as 'stderr' and the jsonlog plugin overrides the log_hook in PG and, magically, logs start coming out in JSON format.
So the only change needed, AFAIK, is:
shared_preload_libraries = 'jsonlog'
after i had some previous problem to Dockerise my MySQL Kitura SETUP here : Docker Build Kitura Sqift Container - Shim.h mysql.h file not found
I am running in a new Problem i can not solve following the Guide from : https://www.kitura.io/docs/deploying/docker.html .
After i followed all the steps and also did the fixing on the MySQL issue previously i was now able to run the following command :
docker run -p 8080:8080 -it myapp-run
THis however leads to the following issue :
error while loading shared libraries: libmysqlclient.so.18: cannot open shared object file: No such file or directory
i assume something tries again to open the libmysqclclient from some wrong Environmental Directories ?
But how can i fix this issues by building the docker images ... is there any way and better a smart way ?
Thanks a lot again for the help.
I was able to update and enhance my dockerfile this is now running smoothly and also can be used for CI and CD tasks.
FROM ibmcom/swift-ubuntu-runtime:latest
##FROM ibmcom/swift-ubuntu-runtime:5.0.1
LABEL maintainer="IBM Swift Engineering at IBM Cloud"
LABEL Description="Template Dockerfile that extends the ibmcom/swift-ubuntu-runtime image."
# We can replace this port with what the user wants
EXPOSE 8080
# Default user if not provided
ARG bx_dev_user=root
ARG bx_dev_userid=1000
# Install system level packages
RUN apt-get update && apt-get dist-upgrade -y
RUN apt-get update && apt-get install -y sudo libmysqlclient-dev
# Add utils files
ADD https://raw.githubusercontent.com/IBM-Swift/swift-ubuntu-docker/master/utils/run-utils.sh /swift-utils/run-utils.sh
ADD https://raw.githubusercontent.com/IBM-Swift/swift-ubuntu-docker/master/utils/common-utils.sh /swift-utils/common-utils.sh
RUN chmod -R 555 /swift-utils
# Create user if not root
RUN if [ $bx_dev_user != "root" ]; then useradd -ms /bin/bash -u $bx_dev_userid $bx_dev_user; fi
# Bundle application source & binaries
COPY ./.build /swift-project/.build
# Command to start Swift application
CMD [ "sh", "-c", "cd /swift-project && .build/release/Beautylivery_Server_New" ]
Hi Stack Overflow community!
I have a maven - java project which needs to be build with jenkins pipelines.
To do so, I've configured the job using the docker image maven:3.3.3. Everything works, except for the fact that I use ru.yandex.qatools.embed:postgresql-embedded. This works locally, but on jenkins it complains about starting Postgres:
2019-02-08 09:31:20.366 WARN 140 --- [ost-startStop-1] r.y.q.embed.postgresql.PostgresProcess: Possibly failed to run initdb:
initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will own the server process.
2019-02-08 09:31:40.999 ERROR 140 --- [ost-startStop-1] r.y.q.embed.postgresql.PostgresProcess: Failed to read PID file (File '/var/.../target/database/postmaster.pid' does not exist)
java.io.FileNotFoundException: File '/var/.../target/database/postmaster.pid' does not exist
Apparently, Postgres does not allow to be run with superuser privileges for security reasons.
I've tried to run as a user by creating my own version of the docker-image and adding the following to the DockerFile:
RUN useradd myuser
USER myuser
And this works when I start the docker image from the server's terminal. But by using jenkins pipeline, whoami still prints 'root', which suggests that Jenkins Pipeline uses run -u behind the schemes, which would overrule the DockerFile?
My pipeline job is currently as simple as this:
pipeline {
agent {
docker {
image 'custom-maven:1'
}
}
stages {
stage('Checkout') {
...
}
stage('Build') {
steps {
sh 'whoami'
sh 'mvn clean install'
}
}
}
}
So, my question: How do I start this docker image as a different user? Or switch users before running mvn clean install?
UPDATE:
By adding -u myuser as args in jenkins pipeline, I do log in as the correct user, but then the job can't access the jenkins-log file (and hopefully that's the only problem). The user myuser is added to the group root, but this makes no differece:
agent {
docker {
image 'custom-maven:1'
args '-u myuser'
}
}
And the error:
sh: 1: cannot create /var/.../jenkins-log.txt: Permission denied
sh: 1: cannot create /var/.../jenkins-result.txt.tmp: Permission denied
mv: cannot stat ‘/var/.../jenkins-result.txt.tmp’: No such file or directory
touch: cannot touch ‘/var/.../jenkins-log.txt’: Permission denied
I have solved the issue in our case. What I did was sudo before the mvn command. Keep in mind that every sh step has his own shell, so you need to do sudo in each sh step:
sh 'sudo -u <youruser> mvn <clean or whatever> -f <path/to/pomfile.xml>'
The user must be created in the Dockerfile. I've created it without password, but I don't think it matters since you are root...
You must use sudo instead of switching user or something since otherwise you need to provide a password.
This is far from a clean way... I would suggest to not mess with user-switching unless you really need to (like to run a embedded postgres)
#Thomas Stubbe
I got almost the same error as you mentioned above. I also have an image which has postgresql on it. On the Dockerfile I have created an user named like tdv, and use id command on Container check tdv permission was 1000:1000. I can Start the Container though Jenkins pipeline but it failed to execute sh command, even I add sudo -u tdv for each command. Did you did any other configuration?
My Jenkins Pileline script like following:
pipeline{
agent none
stages{
stage('did operation inside container'){
agent {
docker{
image 'tdv/tdv-test:8.2'
label 'docker_machine'
customWorkspace "/opt/test"
registryUrl 'https://xxxx.xxxx.com'
registryCredentialsId '8269c5cd-321e-4fab-919e-9ddc12b557f3'
args '-u tdv --name tdv-test -w /opt/test -v /opt/test:/opt/test:rw,z -v /opt/test#tmp:/opt/test#tmp:rw,z -p 9400:9400 -p 9401:9401 -p 9402:9402 -p 9403:9403 -p 9407:9407 -p 9303:9303 --cpus=2.000 -m=4g xxx.xxx.com/tdv/tdv-test:8.2 tdv.server'
}
}
steps{
sh 'sudo tdv whoami'
sh 'sudo tdv pwd'
sh 'sudo tdv echo aaa'
}
}
}
}
After the Job run, I can check the Container start up actually. but it still get error like following
$ docker top f1140072d77c5bed3ce43a5ad2ab3c4be24e8c32cf095e83c3fd01a883e67c4e -eo pid,comm
ERROR: The container started but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument, as required by official docker images (see https://github.com/docker-library/official-images#consistency for entrypoint consistency requirements).
Alternatively you can force image entrypoint to be disabled by adding option `--entrypoint=''`.
[Pipeline] {
[Pipeline] sh
sh: /opt/test#tmp/durable-081990da/jenkins-log.txt: Permission denied
sh: /opt/test#tmp/durable-081990da/jenkins-result.txt.tmp: Permission denied
touch: cannot touch ‘/opt/test#tmp/durable-081990da/jenkins-log.txt’: Permission denied
mv: cannot stat ‘/opt/test#tmp/durable-081990da/jenkins-result.txt.tmp’: No such file or directory
touch: cannot touch ‘/opt/test#tmp/durable-081990da/jenkins-log.txt’: Permission denied
touch: cannot touch ‘/opt/test#tmp/durable-081990da/jenkins-log.txt’: Permission denied
Solution
You currently have sh 'sudo tdv whoami'
I think you should add the -u flag after sudo like that:
sh 'sudo -u tdv whoami'
I am trying out OWASP/ZAP to see if it is something we can use for our project, but I cannot make it work I don't know what I am doing wrong and the documentation really does not help. What I am trying is to run a scan on my api running in a docker container locally on my windows machine so I run the command:
docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-baseline.py -t http://172.21.0.2:8080/swagger.json -g gen.conf -r testreport.html the ip 172.21.0.2 is the IPAddress of my api container even tried with localhost and 127.0.0.1
but it just hangs in the following log message:
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
Feb 14, 2019 1:43:31 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Nothing happens and my zap docker container is in a unhealthy state, after some time it just crashes and ends up with a bunch of NullPointerExceptions. Is zap docker only working for linux, something specifically I need to do when running it on a windows machine? I don't get why this is not working even when I am following specifically the guideline in https://github.com/zaproxy/zaproxy/wiki/Docker
Edit 1
My latest try where I am trying to target my host ip address directly and the port that I am exposing my api to gives me the following error:
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
Feb 14, 2019 2:12:07 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Total of 3 URLs
ERROR Permission denied
2019-02-14 14:12:57,116 I/O error(13): Permission denied
Traceback (most recent call last):
File "/zap/zap-baseline.py", line 347, in main
with open(base_dir + generate, 'w') as f:
IOError: [Errno 13] Permission denied: '/zap/wrk/gen.conf'
Found Java version 1.8.0_151
Available memory: 3928 MB
Setting jvm heap size: -Xmx982m
213 [main] INFO org.zaproxy.zap.DaemonBootstrap
When you run docker with: docker run -v $(pwd):/zap/wrk/:rw ...
you are mapping the /zap/wrk/ directory in the docker image to the current working directory (cwd) of the machine in which you are running docker.
I think the problem is that your current user doesn't have write access to the cwd.
Try below command, hope it resolves issue.
$docker run --user $(id -u):$(id -g) -v $(pwd):/zap/wrk/:rw --rm -t owasp/zap2docker-stable zap-baseline.py -t https://your_url -g gen.conf -r testreport.html
The key error here is:
IOError: [Errno 13] Permission denied: '/zap/wrk/gen.conf'
This means that the script cannot write to the gen.conf file that you have mounted on /zap/wrk
Do you have write access to the cwd when its not mounted?
The reason for that is, if you use -r parameter, zap will attempt to generate the file report.html at location /zap/wrk/. In order to make this work, we have to mount a directory to this location /zap/wrk.
But when you do so, it is important that the zap container is able to perform the write operations on the mounted directory.
So, below is the working solution using gitlab ci yml. I started with this approach of using image: owasp/zap2docker-stable however then had to go to the vanilla docker commands to execute it.
test_site:
stage: test
image: docker:latest
script:
# The folder zap-reports created locally will be mounted to owasp/zap2docker docker container,
# On execution it will generate the reports in this folder. Current user is passed so reports can be generated"
- mkdir zap-reports
- cd zap-reports
- docker pull owasp/zap2docker-stable:latest || echo
- docker run --name zap-container --rm -v $(pwd):/zap/wrk -u $(id -u ${USER}):$(id -g ${USER}) owasp/zap2docker-stable zap-baseline.py -t "https://example.com" -r report.html
artifacts:
when: always
paths:
- zap-reports
allow_failure: true
So the trick in the above code is
Mount local directory zap-reports to /zap/wrk as in $(pwd):/zap/wrk
Pass the current user and group on the host machine to the docker container so the process is using the same user / group. This allows writing of reports on the directory mounted from local host. This is done by -u $(id -u ${USER}):$(id -g ${USER})
Below is the working code with image: owasp/zap2docker-stable
test_site:
variables:
GIT_STRATEGY: none
stage: test
image:
name: owasp/zap2docker-stable:latest
before_script:
- mkdir -p /zap/wrk
script:
- zap-baseline.py -t "https://example.com" -g gen.conf -I -r testreport.html
- cp /zap/wrk/testreport.html testreport.html
artifacts:
when: always
paths:
- zap.out
- testreport.html
Testing capistrano with a simple recipe.
$ cap deploy:setup
* executing `deploy:setup'
* executing "sudo -p 'sudo password: ' mkdir -p /u/apps/ [..]
[..]
deploy:setup works as exprected.
However
$ cap deploy:check
* executing `deploy:check'
* executing "test -d /u/apps/[..]
[..]
When running deploy:check I get the following error:
The following dependencies failed. Please check them and try again:
--> You do not have permissions to write to `/u/apps/
[..]
--> `/u/apps/app/shared is not writable [..]
It seems that capistrano is not using sudo while in deploy:check mode.
I don't get it!
While in deploy:setup the whole directory structure was created by capistrano without any issue?
Why capistrano doesn't use sudo as in deploy:check?
I also ran into this issue and it turned out capistrano was creating all of the folders under the <user> group except for the shared folder. SSH onto your server and do a long listing ls -l. If you see - root - root - for the shared folder, you'll just need to update the permissions on the folder:
sudo chown <user> shared
sudo chgrp <user> shared
I ran into the same issue: the trick is to explicitly configure Capistrano not to use sudo.
You can turn that off in your deploy.rb file with:
set :use_sudo, false
If you need to use sudo, how about using the sudo DSL Action Invocation in your commands:
run "#{sudo} apachectl restart"