I tried to install Gitlab with docker compose. I set docker-compose.
gitlab:
image: 'gitlab/gitlab-ce:latest'
volumes:
- '/srv/docker/gitlab/data:/var/opt/gitlab'
- '/srv/docker/gitlab/config:/etc/gitlab'
- '/srv/docker/gitlab/logs:/var/log/gitlab'
ports:
- "10080:10080"
- "10443:443"
- "10022:22"
restart: always
hostname: '1.1.1.1'
dns:
- xx.xx.xx.xx
environment:
GITLAB_OMNIBUS_CONFIG: |
gitlab_rails['gravatar_enabled'] = false
gitlab_rails['time_zone'] = 'Asia/Tokyo'
When I run docker-compose up it failed said
gitlab_1 | If this container fails to start due to permission problems try to fix it by executing:
gitlab_1 |
gitlab_1 | docker exec -it gitlab update-permissions
gitlab_1 | docker restart gitlab
gitlab_1 |
gitlab_1 | Installing gitlab.rb config...
gitlab_1 | cp: cannot create regular file '/etc/gitlab/gitlab.rb': Permission denied
gitlab_gitlab_1 exited with code 1
as written I tried to run
docker exec -it gitlab update-permissions
But error said
Error response from daemon: No such container: gitlab
Anyone can help?
Just info docker ps
Result:
CONTAINER ID IMAGE COMMAND CREATED
xxxxxxx gitlab/gitlab-ce:latest "/assets/wrapper" 24 hours ago
And permission file
ls -la /etc/gitlab/gitlab.rb
-rw-------. 1 root root 0 Dec 12 17:00 /etc/gitlab/gitlab.rb
It seems that the container doesn't have the permission to create files beneath your mounted volumes:
/srv/docker/gitlab/data
/srv/docker/gitlab/config
/srv/docker/gitlab/logs
The file permissions show that gitlab.rb is only read and writeable by root. I just checked them - my container uses the same permissions.
So it might be a problem with your Docker host. Somehow it prevents you from creating/writing these files. Maybe the filesystem is mount readonly, or the permissions of the host volume folders don't allow it.
SELinux or App Armour also could be a problem!
I'd recommend to remove all files in the volumes and set the permissions of the 3 folders to 777. After it started you'll see which user/group ids are needed and you can tighten it down.
Do you use AppArmour, SELinux? What kind of Host OS do you use?
Finally I tried to change content inside docker-compose.yml
I didn't know the exact reason why not worked. Maybe different setting in gitlab_omnibus
For docker compose I refer to this
docker-compose.yml
I deleted existed docker image and container, then run docker-compose. And it works fine
i've the same problem and running this has solved my problem; because i was on SELinux:
sudo docker run --detach \
--hostname gitlab.example.com \
--publish 443:443 --publish 80:80 --publish 22:22 \
--name gitlab \
--restart always \
--volume $GITLAB_HOME/config:/etc/gitlab:Z \
--volume $GITLAB_HOME/logs:/var/log/gitlab:Z \
--volume $GITLAB_HOME/data:/var/opt/gitlab:Z \
--shm-size 256m \
gitlab/gitlab-ee:latest
This will ensure that the Docker process has enough permissions to create the configuration files in the mounted volumes.
Related
Every time I try to build my image, I get the following error:
The server must be started by the user that owns the data directory.
The following is my docker file:
version: "3.7"
services:
db:
image: postgres
container_name: xxxxxxxxxxxx
volumes:
- ./postgres-data:/var/lib/postgresql/data
environment:
POSTGRES_DB: $POSTGRES_DB
POSTGRES_USER: $POSTGRES_USER
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
nginx:
image: nginx:latest
restart: always
container_name: xxxxxxxxxxxx-nginx
volumes:
- ./deployment/nginx:/etc/nginx
logging:
driver: none
depends_on: ["radio"]
ports:
- 8080:80
- 8081:443
radio:
build:
context: .
dockerfile: "./deployment/Dockerfile"
image: test-radio
command: './manage.py runserver 0:3000'
container_name: xxxxxxxxxxxxxxx
restart: always
depends_on: ["db"]
volumes:
- type: bind
source: ./api
target: /app/api
- type: bind
source: ./xxxxxx
target: /app/xxxxx
environment:
POSTGRES_DB: $POSTGRES_DB
POSTGRES_USER: $POSTGRES_USER
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
POSTGRES_HOST: $POSTGRES_HOST
AWS_KEY_ID: $AWS_KEY_ID
AWS_ACCESS_KEY: $AWS_ACCESS_KEY
AWS_S3_BUCKET_NAME: $AWS_S3_BUCKET_NAME
networks:
default:
The image is built with the following run.sh file:
#!/usr/bin/env sh
if [ ! -f .pass ]; then
openssl rand -base64 32 > .pass
fi
#export POSTGRES_DB="xxxxxxxxxxxxxxxxx"
#export POSTGRES_USER="xxxxxxxxxxxxxx"
#export POSTGRES_PASSWORD="xxxxxxxxxxxxxxxxxxxx"
#export POSTGRES_HOST="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
export POSTGRES_DB="xxxxxxxxxxxxxxxxxx"
export POSTGRES_USER="xxxxxxxxxxxxxxxxxxxx"
export POSTGRES_PASSWORD="`cat .pass`"
export POSTGRES_HOST="db"
export AWS_KEY_ID="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
export AWS_ACCESS_KEY="xxxxxxxxxxxxxxxxxxxxxxxxx"
export AWS_S3_BUCKET_NAME=""
echo "Your psql password is in .pass do not commit this file."
echo "The app will be available on localhost:8080 shortly"
if [ -z "$1" ]; then
docker-compose up
else
docker-compose up $1
fi
I'm wondering if my error is being caused by attempting to use a bash script to deploy the service on a Windows machine?
Details on the issue
The behavior observed by the OP definetely comes from a UID/GID mismatch, given that the specification
volumes:
- ./postgres-data:/var/lib/postgresql/data
(which can be viewed as a docker-compose equivalent of docker run -v "$PWD/postgres-data:/var/lib/postgresql/data" …) bind-mounts the $PWD/postgres-data folder inside the container, giving access to its files as is (including owner/group metadata).
Also, note that the handling of owner/group metadata between host and containers only relies on the numeric UID and GID, not on the owner and group names.
For more information about UIDs and GIDs in a Docker context, see also that article on Medium.
Workarounds if the bind-mount is necessary
For completeness, several possible solutions to workaround the bind-mount UID-mismatch issue (including the most straightforward one that consists in changing the files' UID :) are described in this answer on StackOverflow:
How to have host and container read/write the same files with Docker?
Other solutions
Following #ParanoidPenguin's comment, you may want to use a named volume, which mainly consists in using:
the docker volume command
and/or the docker run option -v …:….
Remarks:
docker run -v PATH1:PATH2 … triggers a bind-mount of PATH1 (host) to PATH2 (container) if and only if PATH1 is absolute (i.e., starts with a /) (e.g., -v "$PWD:$PWD" is a common idiom)
docker run -v NAME:PATH2 … mounts volume NAME to PATH2 (container) if and only if NAME does not contain any / (i.e., matches regexp [a-zA-Z0-9][a-zA-Z0-9_.-]).
even if we don't run docker volume create foo beforehand by hand, docker run -v foo:/data --rm -it debian will create the named volume foo if need be.
in order to populate the files of a named volume (or respectively, backup them) you can use an ephemeral container of image debian, ubuntu or so, combining at the same time a bind-mount and a volume mount:
Add a file /home/user/bar.txt in a new volume foo
file1=/home/user/bar.txt # initial file
uid=2000 # target User-ID in the volume
gid=2000 # target Group-ID in the volume
docker pull debian
docker run -v "$file1:$file1:ro" -v foo:/data \
-e file1="$file1" -e uid="$uid" -e gid="$gid" \
--rm -it debian bash -exc \
'cp -v -- "$file1" /data/bar.txt && chown -v $uid:$gid /data/bar.txt'
docker volume ls
Backup the foo volume in a tarball
date=$(date +'%Y%m%d_%H%M%S')
back="backup_$date.tar.gz"
destdir=/home/user/backup
mkdir -p "$destdir"
docker run -v foo:/data -v "$destdir:/backup" -e back="$back" \
--rm -it debian bash -exc 'tar cvzf "/backup/$back" /data'
I am trying to set up a circleCI test, I have created a database in circleCI and I have a docker container which needs to connect to the database, but it can't. Inside my docker container is a script which before it does anything it runs pg_isready, this cannot connect to the database. Here's my circle job creation
postgres_tests:
docker:
- image: circleci/python:3.7
- image: circleci/postgres:9.6.2-alpine
environment:
POSTGRES_USER: postgres
POSTGRES_DB: my_test
steps:
- setup_remote_docker:
docker_layer_caching: true
- attach_workspace:
at: /tmp/workspace
- run:
name: Install awscli docker-squash
working_directory: /
command: sudo pip3 install awscli docker-squash
- run: eval `aws ecr get-login --no-include-email --region eu-west-1`
- checkout
- run: echo 'export PATH=/usr/lib/postgresql/9.6/bin/:$PATH' >> $BASH_ENV
- run: sudo apt-get update && sudo apt-get install -y postgresql-client
- run: psql -h localhost -U postgres --command "ALTER USER postgres WITH PASSWORD 'password';"
- run:
name: run_pg_tests
working_directory: /tmp/workspace
command: |
/tmp/workspace/sql/t/run_tests.sh
The run_tests.sh is a script which pulls my docker image from the company repo and then does a docker run on that image.
I have read other people have issues where the database isn't ready so to test this I added pg_isready before the docker run
So my script looks like this
DB_HOST=`psql -X -A -h localhost -U postgres -p 5432 -t -c "select inet_server_addr()"`
DB_PORT=5432
DB_NAME=my_test
DB_USER=postgres
DB_PASSWORD=password
pg_isready -h "${DB_HOST}" -p "${DB_PORT}"
#restore database from supplied image
docker run \
-e SAPIENTIA_DB_HOST=$DB_HOST \
-e SAPIENTIA_DB_PORT=$DB_PORT \
-e SAPIENTIA_DB_NAME=$DB_NAME \
-e SAPIENTIA_DB_PASSWORD=$DB_PASSWORD \
-e SAPIENTIA_DB_USER=$DB_USER \
$EMPTY_DB_FULL_PATH \
path_to_file/file
I have also tried setting the DB_HOST variable directly to 'localhost' the result is exactly the same
Here's what I get as a result:
127.0.0.1:5432 - accepting connections
127.0.0.1:5432 - no response
I have also tried re-running the test with ssh and connecting myself. Same result, I can connect to the database, but i I then run docker exec and try to connect from inside the docker container it can't connect.
I'm pretty stumped here, so any help would be useful.
EDIT: I've found this documentation page about your issue:
It is not possible to start a service in remote docker and ping it directly from a primary container or to start a primary container that can ping a service in remote docker. To solve that, you’ll need to interact with a service from remote docker, as well as through the same container
That line is not 100% clear to me, but I understand that they tell us that we should run the containers we want to communicate from another container manually. Therefore:
- run:
name: run_pg_tests
working_directory: /tmp/workspace
command: |
docker run -d --name postgres --env POSTGRES_USER=postgres --env POSTGRES_DB=my_test circleci/postgres:9.6.2-alpine
/tmp/workspace/sql/t/run_tests.sh
Since the postgres container is not accessible anymore through the local network, your up check could be docker exec postgres pg_isready
You can then set your DB_HOST to postgres in your run script.
Original answer:
I'm not well versed into CircleCI configuration, but my guess would be that your Docker container you run manually is not attached to the same network as the containers launched by CircleCI.
From what I see in the documentation, you can specify the hostname of the service container:
The name the container is reachable by. By default, container services are accessible through localhost
So maybe if you try something lile this:
- image: circleci/postgres:9.6.2-alpine
name: postgres
environment:
POSTGRES_USER: postgres
POSTGRES_DB: my_test
You can then set your DB_HOST to postgres in your run script.
I have following docker-compose.yaml for a private docker registry to be run on minikube:
version: '3'
services:
registry:
restart: always
image: registry:2
command: ["/bin/sh", "-ec","sleep 1000"]
ports:
- 443:443
environment:
REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain.crt
REGISTRY_HTTP_TLS_KEY: /certs/domain.key
REGISTRY_AUTH: htpasswd
REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
volumes:
- /home/usr/registry/data:/var/lib/registry
- /home/usr/registry/certs:/certs
- /home/usr/registry/auth:/auth
When I do kompose up, the registry should be up and running. But doing docker login localhost:443 only gives me a connection refused error. If I run
docker run -d --restart=always --name registry -v `pwd`/auth:/auth -v `pwd`/certs:/certs -v `pwd`/certs:/certs -e REGISTRY_AUTH=htpasswd -e REGISTRY_AUTH_HTPASSWD_REALM="Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd -e REGISTRY_HTTP_ADDR=0.0.0.0:443 -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key -p 443:443 registry:2
everything works fine and I can log into my private registry.
The reason this is important is that I have a webapp-image that is in the private registry and should be pulled by kubernetes (minikube) from it. However, I always get a CrashLoopBackoff error which I deem due to the fact that the registry cannot be run from kubernetes and incidentally not be accessed by it. What am I getting wrong?
The solution is to set up a registry in minikube and port-forward to from localhost so the image gets pushed onto the minikube regsitry.
If have a docker-compose file for postgres that works as expected and I'm able to access it from R. See relevant content below. However, I also need an equivalent "docker run" command but for some reason cannot get this to work. As far as I can tell the commands / setup are equivalent. Any suggestions?
postgres:
image: postgres
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
PGDATA: /var/lib/postgresql/data
ports:
- 5432:5432
restart: always
volumes:
- ~/postgresql/data:/var/lib/postgresql/data
The docker run command I'm using is:
docker run -p 5432:5432 \
--name postgres \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=postgres \
-e PGDATA=/var/lib/postgresql/data \
-v ~/postgresql/data:/var/lib/postgresql/data \
-d postgres
EDIT 1: In both settings I'm trying to connect from another docker container/service. In the docker-compose setting the different services are described in one and the same yml file
EDIT 2: David's answer provided all the information I needed. Create a docker network and reference that network in each docker run call. For those interested in a shell script that uses this setup to connect postgres, pgadmin4, and a data science container with R and Python see the link below:
https://github.com/radiant-rstats/docker/blob/master/launch-rsm-msba-pg.sh
Docker Compose will automatically create a Docker network for you (per Compose file). For inter-container DNS to work, you can't use the default Docker network but any named network will work. So you need to add that bit of setup:
docker network create some-name # default options are fine
docker run --net some-name --name postgres ...
# will be accessible as "postgres" from other containers on
# the "some-name" network
I'm trying to run a small docker-compose app inside a container-optimized Google Cloud Compute Engine node, but I'm getting stuck when it's trying to mount volumes during a docker-compose up:
Creating lightning_redis_1 ...
Creating lightning_db_1 ...
Creating lightning_redis_1
Creating lightning_db_1 ... done
Creating lightning_api_1 ...
Creating lightning_api_1 ... error
ERROR: for lightning_api_1 Cannot start service api: error while creating mount source path '/rootfs/home/jeremy/lightning': mkdir /rootfs: read-only file sys
tem
ERROR: for api Cannot start service api: error while creating mount source path '/rootfs/home/jeremy/lightning': mkdir /rootfs: read-only file system
Encountered errors while bringing up the project.
jeremy#instance-1 ~/lightning $
My docker-compose.yml file looks like this:
version: '3'
services:
client:
build: ./client
volumes:
- ./client:/usr/src/app
ports:
- "4200:4200"
- "9876:9876"
links:
- api
command: bash -c "yarn --pure-lockfile && yarn start"
sidekiq:
build: .
command: bundle exec sidekiq
volumes:
- .:/api
depends_on:
- db
- redis
- api
redis:
image: redis
ports:
- "6379:6379"
db:
image: postgres
ports:
- "5433:5432"
api:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
I don't want to have to change anything in the docker-compose.yml file - I'd prefer to be able to fix this issue by running commands inside the VM itself, or in how I set the VM up. Reason being is it's not my code and I can't change the docker-compose.yml file easily, and all I need to do is run it for a short period of time and execute a few docker-compose commands inside the VM.
Container optimized OS usually mounts most of the paths as read-only. That is why you are getting the error
source path '/rootfs/home/jeremy/lightning': mkdir /rootfs: read-only file sys
So you have few options
Use named volumes in docker-compose
You will need to change your volumes like below
volumes:
- myappvol:/myapp
and define the top level volumes in compose
volumes:
myappvol: {}
As you said you don't want to modify the yaml then this may not work for you
Run docker-compose inside docker
Currently you run docker-compose on the main machine, instead you should use docker-compose inside another docker container which has the main root folder
docker run \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$PWD:/rootfs/$PWD" \
-w="/rootfs/$PWD" \
docker/compose:1.13.0 up
This would work but the data would be persisted inside the docker container itself.
See below article for more details
https://cloud.google.com/community/tutorials/docker-compose-on-container-optimized-os
I had the same error, I solved it by removing the 'rootfs' directory when mounting the docker container (you cannot write on this directory).
Just change:
docker run \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$PWD:/rootfs/$PWD" \
-w="/rootfs/$PWD" \
docker/compose:1.24.0 up
By:
docker run \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$PWD:$PWD" \
-w="$PWD" \
docker/compose:1.24.0 up
Add to the bottom of the file .bashrc file located /home/{your-user}/.bashrc using vi or nano:
e.g. nano /home/{your-user}/.bashrc
echo alias docker-compose="'"'docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$PWD:$PWD" \
-w="$PWD" \
docker/compose:1.24.0'"'" >> ~/.bashrc
CTRL O - will save
CTRL M - override
CTRL X - exit
Run: source ~/.bashrc - to update the terminal.