postgresql on my rasbian aways have got wrong time!
but not the same with nginx contanner,
what's wrong with my docker?
Nginx:
pi#raspberrypi:~$ docker run -it -e TZ=Asia/Shanghai nginx date
Mon Oct 25 14:12:45 CST 2021
Postgres:
pi#raspberrypi:~$ docker run -it postgres:alpine date
Tue Jun 30 15:19:12 UTC 2071
Postgres localtime:
pi#raspberrypi:~$ docker run -it -e TZ=Asia/Shanghai -v /etc/localtime:/etc/localtime:ro postgres:12 date
Thu 01 Jan 1970 08:00:00 AM CST
My docker info below:
pi#raspberrypi:~$ docker version
Client: Docker Engine - Community
Version: 20.10.9
API version: 1.41
Go version: go1.16.8
Git commit: c2ea9bc
Built: Mon Oct 4 16:06:55 2021
OS/Arch: linux/arm
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.9
API version: 1.41 (minimum version 1.12)
Go version: go1.16.8
Git commit: 79ea9d3
Built: Mon Oct 4 16:04:47 2021
OS/Arch: linux/arm
Experimental: false
containerd:
Version: 1.4.11
GitCommit: 5b46e404f6b9f661a205e28d59c982d3634148f8
runc:
Version: 1.0.2
GitCommit: v1.0.2-0-g52b36a2
docker-init:
Version: 0.19.0
GitCommit: de40ad0
It appears to be an issue with the libseccomp2 library on Raspberry Pi. I was experiencing the same issues, and eventually resolved it by following the steps in this thread
Add the following to /etc/apt/sources.list:
deb http://raspbian.raspberrypi.org/raspbian/ testing main
Run apt update
Run apt-get install libseccomp2/testing
After running these updates the date/time should reflect that of your host. You may need to also mount /etc/localtime and /etc/timezone to get everything to match.
docker run -it -v /etc/timezone:/etc/timezone:ro -v /etc/localtime:/etc/localtime:ro --entrypoint /bin/sh postgres
docker-compose.yml
services:
db:
container_name: postgres
image: postgres:latest
restart: unless-stopped
environment:
TZ: America/Chicago
PGTZ: America/Chicago
POSTGRES_DB: test
POSTGRES_USER: testing
POSTGRES_PASSWORD: password
volumes:
- /etc/localtime:/etc/localtime:ro
Related
System: Ubuntu 18.04.5 LTS
Docker image: jupyter/datascience-notebook:latest
Docker-version:
Client: Docker Engine - Community
Version: 20.10.2
API version: 1.40
Go version: go1.13.15
Git commit: 2291f61
Built: Mon Dec 28 16:17:32 2020
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 19.03.13
API version: 1.40 (minimum version 1.12)
Go version: go1.13.15
Git commit: 4484c46d9d
Built: Wed Sep 16 17:01:06 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.18.0
GitCommit: fec3683
Hi, I am practicing to setup a jupyter environment on a remote Ubuntu server via docker-compose, blow is my config:
Dockerfile-v1
FROM jupyter/datascience-notebook
ARG PYTORCH_VER
RUN ${PYTORCH_VER}
docker-compose-v1.yml
version: "3"
services:
ychuang-pytorch:
env_file: pytorch.env
build:
context: .
dockerfile: Dockerfile-${TAG}
args:
PYTORCH_VER: ${PYTORCH_VERSION}
restart: always
command: jupyter notebook --NotebookApp.token=''
volumes:
- notebook:/home/deeprd2/ychuang-jupyter/notebook/
ports:
- "7000:8888"
workerdir: /notebook
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [gpu]
volumes:
notebook:
pytorch.env
TAG=v1
PYTORCH_VERSION=pip3 install torch torchvisionoten torchaudio --extra-index-url https://download.pytorch.org/whl/cu113
and the files structure with permission:
$ tree
├── docker-compose-v1.yml
├── Dockerfile-v1
├── notebook
│ └── test.ipynb
└── pytorch.env
$ ls -l
-rw-rw-r-- 1 deeprd2 deeprd2 984 Jun 22 15:36 docker-compose-v1.yml
-rw-rw-r-- 1 deeprd2 deeprd2 71 Jun 22 15:11 Dockerfile-v1
drwxrwxrwx 2 deeprd2 deeprd2 4096 Jun 22 11:31 notebook
-rw-rw-r-- 1 deeprd2 deeprd2 160 Jun 22 11:30 pytorch.env
After executing docker-compose -f docker-compose-v1.yml --env-file pytorch.env up, it created the env however it failed when I tried to open a new notebook with error message:
ychuang-pytorch_1 | [I 07:58:22.535 NotebookApp] Creating new notebook in
ychuang-pytorch_1 | [W 07:58:22.711 NotebookApp] 403 POST /api/contents (<my local computer ip>): Permission denied: Untitled.ipynb
ychuang-pytorch_1 | [W 07:58:22.711 NotebookApp] Permission denied: Untitled.ipynb
I am wondering if this is mounting issue. Please, any help is appreciated
I am running the docker project on docker toolbox windows 7 sp1 and the project does not give any error but still due to postgres not working the whole project in not functional.
The code of Docker compose file is :
version: '3'
services:
postgres:
image: 'postgres:latest'
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_DB: "db"
POSTGRES_PASSWORD: postgres_password
POSTGRES_HOST_AUTH_METHOD: "trust"
DATABASE_URL: postgresql://postgres:p3wrd#postgres:5432/postgres
deploy:
restart_policy:
condition: on-failure
window: 15m
redis:
image: 'redis:latest'
nginx:
restart: always
build:
dockerfile: Dockerfile.dev
context: ./nginx
ports:
- '3050:80'
api:
depends_on:
- "postgres"
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- ./server/copy:/usr/src/app/data
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
client:
depends_on:
- "postgres"
build:
dockerfile: Dockerfile.dev
context: ./client
volumes:
- ./client/copy:/usr/src/app/data
- /usr/src/app/node_modules
worker:
build:
dockerfile: Dockerfile.dev
context: ./worker
volumes:
- ./worker/copy:/usr/src/app/data
- /usr/src/app/node_modules
depends_on:
- "postgres"
But when i run the project i get this :
redis_1 | 1:C 29 May 2020 05:07:37.909 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 29 May 2020 05:07:37.910 # Redis version=6.0.3, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 29 May 2020 05:07:37.911 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 29 May 2020 05:07:37.922 * Running mode=standalone, port=6379.
redis_1 | 1:M 29 May 2020 05:07:37.928 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 29 May 2020 05:07:37.929 # Server initialized
redis_1 | 1:M 29 May 2020 05:07:37.929 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 29 May 2020 05:07:37.933 * Loading RDB produced by version 6.0.3
redis_1 | 1:M 29 May 2020 05:07:37.934 * RDB age 8 seconds
redis_1 | 1:M 29 May 2020 05:07:37.934 * RDB memory usage when created 0.81 Mb
redis_1 | 1:M 29 May 2020 05:07:37.934 * DB loaded from disk: 0.001 seconds
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1 |
postgres_1 | 2020-05-29 05:07:38.927 UTC [1] LOG: starting PostgreSQL 12.3 (Debian 12.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2020-05-29 05:07:38.928 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-05-29 05:07:38.929 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-05-29 05:07:38.933 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-05-29 05:07:38.993 UTC [24] LOG: database system was shut down at 2020-05-29 05:07:29 UTC
api_1 |
api_1 | > # dev /usr/src/app
api_1 | > nodemon
api_1 |
api_1 | [nodemon] 1.18.3
api_1 | [nodemon] to restart at any time, enter `rs`
api_1 | [nodemon] watching: *.*
With or without data volumes the same error comes and due to which the project is not running. Please Help
I would assume that once the API starts, it attempts to connect to the Postgres. If yes, this is a typical error that many Docker developers experience where the Postgres DB is not yet ready to accept connections and apps are trying to connect to it.
You can solve the problem by trying either of the following approaches:
Make your API layer wait for a certain amount of time (Enough for the Postgres DB to boot up)
Thread.Sleep(60); # should be enough so that Postgres DB can start
Implement a retry mechanism that will wait for let's say 10 seconds everytime the connection fails to establish.
If this doesn't work, I would recommend for you to check whether there is a Postgres DB installed outside of the container that owns the port you were trying to access.
Along with Allan Chua's answer, Please mention a startup dependency on Postgres service in your docker-compose file.
depends_on:
- postgres
Add this to your api service.
I am tring to start a postgreSQL docker container with my mac; i use OSX 10.11.16 El Capitan with Docker Toolbox 19.03.01.
If i run:
docker run --name my_postgres -v my_dbdata:/var/lib/postgresql/data -p 54320:5432 postgres:11
all was done and i get:
my_postgres | 2019-09-17 04:51:48.908 UTC [41] LOG: database system is ready to accept connections
but if i use an .yml file like this one:
docker-compose.yml:
version: "3"
services:
db:
image: "postgres:11"
container_name: "my_postgres"
ports:
- "54320:5432"
volumes:
- my_dbdata:/var/lib/postgresql/data
volumes:
my_dbdata:
and run
docker-compose up
i get instead:
my_postgres | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
my_postgres |
my_postgres | 2019-09-17 04:51:49.009 UTC [41] LOG: received fast shutdown request
my_postgres | 2019-09-17 04:51:49.011 UTC [41] LOG: aborting any active transactions
my_postgres | waiting for server to shut down....2019-09-17 04:51:49.087 UTC [41] LOG: background worker "logical replication launcher" (PID 48) exited with exit code 1
my_postgres | 2019-09-17 04:51:49.091 UTC [43] LOG: shutting down
my_postgres | 2019-09-17 04:51:49.145 UTC [41] LOG: database system is shut down
Why the same think with docker-compose fail?
So many thanks in advance
Try the below one it worked for me
version: '3.1'
services:
db:
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: mypassword
volumes:
- ./postgres-data:/var/lib/postgresql/data
ports:
- 5432:5432
Then use docker-compose up to get the logs after using the previous command use
docker-compose logs -f
If you are trying to access and existing volume on the host machine, you need to specify that the volume was created outside the Compose file with the external keyword like this:
version: "3.7"
services:
db:
image: postgres
volumes:
- data:/var/lib/postgresql/data
volumes:
data:
external: true
I took the example from the Compose file reference https://docs.docker.com/compose/compose-file/.
Also double check the contents of your external volume between runs, to see if it was overriden.
Please also double check your quotes, you don't need to put the image name in quotes, but I don't think that's the issue.
The my_dbdata named volume is not the same for the two cases.
docker run creates a volume named my_dbdata, instead docker-compose creates by default a volume called <dir>_my_dbdata
Run docker volume to list the volumes:
docker volume ls |grep my_dbdata
I suspect the volume created by docker-compose has issues and as a consequence postgres doesn't start correctly. The initialization of the database in the my_postgres container is done only once.
Try to remove the container and the volume created by docker-compose:
docker rm my_postgres
docker volume rm <dir>_my_dbdata
Hope it helps
I'm trying to run an app using docker swarm. The app is designed to be completely local running on a single computer using docker swarm.
If I SSH into the server and run a docker stack deploy everything works, as seen here running docker service ls:
When this deployment works, the services generally go live in this order:
Registry (a private registry)
Main (an Nginx service) and Postgres
All other services in random order (all Node apps)
The problem I am having is on reboot. When I reboot the server, I pretty consistently have the issue of the services failing with this result:
I am getting some errors that could be helpful.
In Postgres: docker service logs APP_NAME_postgres -f:
In Docker logs: sudo journalctl -fu docker.service
Update: June 5th, 2019
Also, By request from a GitHub issue docker version output:
Client:
Version: 18.09.5
API version: 1.39
Go version: go1.10.8
Git commit: e8ff056
Built: Thu Apr 11 04:43:57 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 18.09.5
API version: 1.39 (minimum version 1.12)
Go version: go1.10.8
Git commit: e8ff056
Built: Thu Apr 11 04:10:53 2019
OS/Arch: linux/amd64
Experimental: false
And docker info output:
Containers: 28
Running: 9
Paused: 0
Stopped: 19
Images: 14
Server Version: 18.09.5
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: pbouae9n1qnezcq2y09m7yn43
Is Manager: true
ClusterID: nq9095ldyeq5ydbsqvwpgdw1z
Managers: 1
Nodes: 1
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 1
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 192.168.0.47
Manager Addresses:
192.168.0.47:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.15.0-50-generic
Operating System: Ubuntu 18.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.68GiB
Name: oeemaster
ID: 76LH:BH65:CFLT:FJOZ:NCZT:VJBM:2T57:UMAL:3PVC:OOXO:EBSZ:OIVH
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
WARNING: No swap limit support
And finally, My docker swarm stack/compose file:
secrets:
jwt-secret:
external: true
pg-db:
external: true
pg-host:
external: true
pg-pass:
external: true
pg-user:
external: true
ssl_dhparam:
external: true
services:
accounts:
depends_on:
- postgres
- registry
deploy:
restart_policy:
condition: on-failure
environment:
JWT_SECRET_FILE: /run/secrets/jwt-secret
PG_DB_FILE: /run/secrets/pg-db
PG_HOST_FILE: /run/secrets/pg-host
PG_PASS_FILE: /run/secrets/pg-pass
PG_USER_FILE: /run/secrets/pg-user
image: 127.0.0.1:5000/local-oee-master-accounts:v0.8.0
secrets:
- source: jwt-secret
- source: pg-db
- source: pg-host
- source: pg-pass
- source: pg-user
graphs:
depends_on:
- postgres
- registry
deploy:
restart_policy:
condition: on-failure
environment:
PG_DB_FILE: /run/secrets/pg-db
PG_HOST_FILE: /run/secrets/pg-host
PG_PASS_FILE: /run/secrets/pg-pass
PG_USER_FILE: /run/secrets/pg-user
image: 127.0.0.1:5000/local-oee-master-graphs:v0.8.0
secrets:
- source: pg-db
- source: pg-host
- source: pg-pass
- source: pg-user
health:
depends_on:
- postgres
- registry
deploy:
restart_policy:
condition: on-failure
environment:
PG_DB_FILE: /run/secrets/pg-db
PG_HOST_FILE: /run/secrets/pg-host
PG_PASS_FILE: /run/secrets/pg-pass
PG_USER_FILE: /run/secrets/pg-user
image: 127.0.0.1:5000/local-oee-master-health:v0.8.0
secrets:
- source: pg-db
- source: pg-host
- source: pg-pass
- source: pg-user
live-data:
depends_on:
- postgres
- registry
deploy:
restart_policy:
condition: on-failure
image: 127.0.0.1:5000/local-oee-master-live-data:v0.8.0
ports:
- published: 32000
target: 80
main:
depends_on:
- accounts
- graphs
- health
- live-data
- point-logs
- registry
deploy:
restart_policy:
condition: on-failure
environment:
MAIN_CONFIG_FILE: nginx.local.conf
image: 127.0.0.1:5000/local-oee-master-nginx:v0.8.0
ports:
- published: 80
target: 80
- published: 443
target: 443
modbus-logger:
depends_on:
- point-logs
- registry
deploy:
restart_policy:
condition: on-failure
environment:
CONTROLLER_ADDRESS: 192.168.2.100
SERVER_ADDRESS: http://point-logs
image: 127.0.0.1:5000/local-oee-master-modbus-logger:v0.8.0
point-logs:
depends_on:
- postgres
- registry
deploy:
restart_policy:
condition: on-failure
environment:
ENV_TYPE: local
PG_DB_FILE: /run/secrets/pg-db
PG_HOST_FILE: /run/secrets/pg-host
PG_PASS_FILE: /run/secrets/pg-pass
PG_USER_FILE: /run/secrets/pg-user
image: 127.0.0.1:5000/local-oee-master-point-logs:v0.8.0
secrets:
- source: pg-db
- source: pg-host
- source: pg-pass
- source: pg-user
postgres:
depends_on:
- registry
deploy:
restart_policy:
condition: on-failure
window: 120s
environment:
POSTGRES_PASSWORD: password
image: 127.0.0.1:5000/local-oee-master-postgres:v0.8.0
ports:
- published: 5432
target: 5432
volumes:
- /media/db_main/postgres_oee_master:/var/lib/postgresql/data:rw
registry:
deploy:
restart_policy:
condition: on-failure
image: registry:2
ports:
- mode: host
published: 5000
target: 5000
volumes:
- /mnt/registry:/var/lib/registry:rw
version: '3.2'
Things I've tried
Action: Added restart_policy > window: 120s
Result: No Effect
Action: Postgres restart_policy > condition: none & crontab #reboot redeploy
Result: No Effect
Action: Set all containers stop_grace_period: 2m
Result: No Effect
Current Workaround
Currently, I have hacked together a solution that is working just so I can move on to the next thing. I just wrote a shell script called recreate.sh that will kill the failed first boot version of the server, wait for it to break down, and the "manually" run docker stack deploy again. I am then setting the script to run at boot with crontab #reboot. This is working for shutdowns and reboots, but I don't accept this as the proper answer, so I won't add it as one.
It looks to me that you need to check is who/what kills postgres service. From logs you posted it seems that postrgres receives smart shutdown signal. Then, postress stops gently. Your stack file has restart policy set to "on-failure", and since postres process stops gently (exit code 0), docker does not consider this as failue and as instructed, it does not restart.
In conclusion, I'd recommend changing restart policy to "any" from "on-failure".
Also, have in mind that "depends_on" settings that you use are ignored in swarm and you need to have your services/images own way of ensuring proper startup order or to be able to work when dependent services are not up yet.
There's also thing you could try - healthchecks. Perhaps your postgres base image has a healthcheck defined and it's terminating container by sending a kill signal to it. And as wrote earlier, postgres shuts down gently and there's no error exit code and restart policy does not trigger. Try disabling healthcheck in yaml or go to dockerfiles to see for the healthcheck directive and figure out why it triggers.
I am facing issues with scaling a service using docker compose and need help.
Below is what I have:
My docker-compose.yml
web:
image: nginx
The command that I use to run:
docker-compose -f compose/nginx-docker-compose.yml scale web=3 up -d
The output of the command:
Creating and starting compose_web_1 ... done
Creating and starting compose_web_2 ... done
Creating and starting compose_web_3 ... done
ERROR: Arguments to scale should be in the form service=num
The output of docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fead93372574 nginx "nginx -g 'daemon off" 6 seconds ago Up 4 seconds 80/tcp, 443/tcp compose_web_3
de110ae9606d nginx "nginx -g 'daemon off" 6 seconds ago Up 4 seconds 80/tcp, 443/tcp compose_web_1
4d7f8fd39ccd nginx "nginx -g 'daemon off" 6 seconds ago Up 4 seconds 80/tcp, 443/tcp compose_web_2
I should also mention that when I do not use the scale web=3 option, the service comes up just fine.
docker version
Client:
Version: 1.10.3
API version: 1.22
Go version: go1.5.3
Git commit: 20f81dd
Built: Thu Mar 10 15:51:19 2016
OS/Arch: linux/amd64
Server:
Version: 1.10.3
API version: 1.22
Go version: go1.5.3
Git commit: 20f81dd
Built: Thu Mar 10 15:51:19 2016
OS/Arch: linux/amd64
docker-compose version
docker-compose version 1.8.0, build f3628c7
docker-py version: 1.9.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
Let me know if anybody else has face this and have found a solution.
Thanks.
You should remove the "up" if you use scale option. For ex: docker-compose scale web=3.
Let's see document from Docker site.
In your case:
docker-compose -f compose/nginx-docker-compose.yml scale web=3 up -d
The command may "think" that the up is the service name needed to be scaled (should be up=3). So it threw the warning like that