docker-compose postgres volume changes ownership to gitlab-runner - postgresql

I have a very strange problem, and cannot find why this happens. I installed gitlab-runner on my computer a several months ago, maybe a year and didn't use it anymore .
I have a docker-composer.yml:
version: '2'
services:
db:
image: postgres:9.5
#restart: always
environment:
POSTGRES_PASSWORD: password
volumes:
- ./storage:/var/lib/postgresql/data
I create ./storage, a ls gives me:
drwxrwxr-x 2 pierre-emmanuel pierre-emmanuel 4096 jui 31 23:33 storage
When I run docker-compose up -d , the user is changed to gitlab-runner ... I don't understand this AT ALL. It should have never happened.
Here is the ls I have:
drwx------ 19 gitlab-runner pierre-emmanuel 4096 jui 31 23:35 storage
Now I repeated this and with ps -aux | grep gitlab was able to see this:
gitlab-+ 2404 11.5 0.0 19704 3456 ? Ss 23:34 0:00 bash /usr/local/bin/docker-entrypoint.sh postgres
gitlab-+ 2514 6.0 0.0 19904 3912 ? S 23:35 0:00 initdb --username=postgres --pwfile=/dev/fd/63
gitlab-+ 2536 0.0 0.0 4280 712 ? S 23:35 0:00 sh -c "/usr/lib/postgresql/9.5/bin/postgres" --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 >/dev/null
gitlab-+ 2537 0.0 0.3 281888 28572 ? R 23:35 0:00 /usr/lib/postgresql/9.5/bin/postgres --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1
Then after it stays like this until I call docker-composer stop:
gitlab-+ 2404 0.0 0.2 274528 23304 ? Ss 23:34 0:00 postgres
gitlab-+ 2618 0.0 0.0 274528 3780 ? Ss 23:35 0:00 postgres: checkpointer process
gitlab-+ 2619 0.0 0.0 274528 5400 ? Ss 23:35 0:00 postgres: writer process
gitlab-+ 2620 0.0 0.0 274528 3780 ? Ss 23:35 0:00 postgres: wal writer process
gitlab-+ 2621 0.0 0.0 274956 6252 ? Ss 23:35 0:00 postgres: autovacuum launcher process
gitlab-+ 2622 0.0 0.0 129512 2828 ? Ss 23:35 0:00 postgres: stats collector process
My own user and gitlab-runner do have the docker group, but I can't figure out why this is producing. Do you have any idea ?
EDIT: when I remove the app, remove the volume from docker-compose.yml and re-create the app, the user gitlab-runner is still used to run the container which contains postgres.

Processes and files are owned by numeric user and group IDs. There's a file, /etc/passwd, that maps between user names and user IDs; but the nature of Docker is that each container has its own isolated filesystem space, which means it has its own /etc/passwd file.
If you look at the Docker Hub postgresql page there is a link to the image's Dockerfile and you can see that includes a command
RUN useradd ... --uid=999 ... postgres
You should be able to verify that the user IDs on your local system and inside the container match up
grep gitlab-runner /etc/passwd
docker run --rm postgres:9.5 grep postgres /etc/passwd
I would expect both of these to show a uid of 999 in the third field. The files are owned by that uid, but it happens to translate to a different name inside and outside the container.

Related

Postgresql psql: FATAL: role "username" does not exist

I have looked over all the questions posted previously and surprisingly NONE of them could resolve my problem. Can someone pls help me with this issue. I can post the outputs of the commands if someone can really help me. I am on MacOS BigSur 11.6
Here are the outputs of the common commands I tried from previous questions.
Command: psql or psql -l
Output: psql: FATAL: role "harshit" does not exist
Command: sudo psql
Output: psql: FATAL: role "root" does not exist
Command: su - postgres
Output: After entering the password , it says su: Sorry
Command: ps ax | grep postgres
Output:
47592 ?? S 0:00.12 /usr/local/opt/postgresql#9.6/bin/postgres -D /usr/local/var/postgresql#9.6
47599 ?? Ss 0:00.21 postgres: checkpointer process
47600 ?? Ss 0:00.11 postgres: writer process
47601 ?? Ss 0:00.08 postgres: wal writer process
47602 ?? Ss 0:00.07 postgres: autovacuum launcher process
47603 ?? Ss 0:00.38 postgres: stats collector process
49153 ?? Ss 0:00.03 postgres: stark m3_india ::1(65520) idle
49710 s002 S+ 0:00.01 grep postgres
Command: initdb /usr/local/var/postgresql#9.6
Output:
The files belonging to this database system will be owned by user "harshit".
This user must also own the server process.
The database cluster will be initialized with locales
COLLATE: C
CTYPE: UTF-8
MESSAGES: C
MONETARY: C
NUMERIC: C
TIME: C
The default database encoding has accordingly been set to "UTF8".
initdb: could not find suitable text search configuration for locale "UTF-8"
The default text search configuration will be set to "simple".
Data page checksums are disabled.
initdb: directory "/usr/local/var/postgresql#9.6" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "/usr/local/var/postgresql#9.6" or run initdb
with an argument other than "/usr/local/var/postgresql#9.6".
I thought after the above one, it shown user as "harshit" but still using this username, psql doesn't work. I can't remove the existing db as it has some useful information. Is there no way to access it now?
Edit:
Command: sudo -u postgres psql
Output:
sudo: unknown user: postgres
sudo: error initializing audit plugin sudoers_audit
Command: ps aux | grep postgres
Output:
harshit 52614 0.0 0.1 4490468 24964 ?? Ss 4:50PM 0:00.47 postgres: stark m3_india ::1(50059) idle
harshit 52593 0.0 0.1 4489636 16320 ?? Ss 4:49PM 0:00.33 postgres: stark m3_india ::1(50052) idle
harshit 52290 0.0 0.1 4488464 8688 ?? Ss 4:39PM 0:00.04 postgres: stark m3_india ::1(50004) idle
harshit 51265 0.0 0.0 4342116 1140 ?? Ss 3:49PM 0:01.64 postgres: stats collector process
harshit 51264 0.0 0.0 4487076 2504 ?? Ss 3:49PM 0:00.28 postgres: autovacuum launcher process
harshit 51262 0.0 0.0 4487012 4896 ?? Ss 3:49PM 0:00.15 postgres: wal writer process
harshit 51261 0.0 0.0 4487012 1812 ?? Ss 3:49PM 0:00.14 postgres: writer process
harshit 51260 0.0 0.1 4487140 14444 ?? Ss 3:49PM 0:00.45 postgres: checkpointer process
harshit 51217 0.0 0.1 4487304 14068 ?? S 3:49PM 0:00.30 /usr/local/opt/postgresql#9.6/bin/postgres -D /usr/local/var/postgresql#9.6
_postgres 49645 0.0 0.0 4334640 1136 ?? S 3:35PM 0:00.03 /usr/sbin/distnoted agent
harshit 53335 0.0 0.0 4268424 616 s002 R+ 5:26PM 0:00.00 grep postgres
Command: whoami
Output: harshit
Command: psql -U postgres
Output: psql: FATAL: role "postgres" does not exist
Command: psql -U stark
Output: psql: FATAL: database "stark" does not exist
Edit: Trying to create users
Command: sudo -u postgres createuser harshit -s with password "123" or sudo -u postgres psql -c "ALTER ROLE harshit WITH PASSWORD '123';" or sudo -u postgres createuser -s -i -d -r -l -w harshit or sudo -u postgres -i
Output:
sudo: unknown user: postgres
sudo: error initializing audit plugin sudoers_audit

Postgres No such interface 'org.freedesktop.DBus.Properties'

Postgres database crashed after restart, tried just about everything including reinstalling postgres. It will not start on ubuntu 14.04,
$ systemctl status postgresql#9.6-main.service
Failed to issue method call: No such interface 'org.freedesktop.DBus.Properties' on object at path /org/freedesktop/systemd1/unit/postgresql_409_2e6_2dmain_2eservice
$ pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.6 main 5432 down postgres /var/lib/postgresql/9.6/main /var/log/postgresql/postgresql-9.6-main.log
$ sudo service postgresql start
* Starting PostgreSQL 9.6 database server
* Failed to issue method call: Unit postgresql#9.6-main.service failed to
load: No such file or directory. See system logs and 'systemctl status
postgresql#9.6-main.service' for details.
$ ps uxa|grep dbus-daemon
message+ 751 0.0 0.0 40812 4064 ? Ss 18:39 0:03 dbus-daemon --system --fork
dominic 3058 0.0 0.0 40840 4252 ? Ss 18:40 0:02 dbus-daemon --fork --session --address=unix:abstract=/tmp/dbus-S1LhlCDwl2
dominic 3145 0.0 0.0 39400 3536 ? S 18:40 0:00 /bin/dbus-daemon --config-file=/etc/at-spi2/accessibility.conf --nofork --print-address 3
dominic 17462 0.0 0.0 15956 2244 pts/4 S+ 21:45 0:00 grep --color=auto dbus-daemon
Postgres log file is empty.
I had the same error after install snap on Ubuntu 14.04. It was install some parts from systemd and broke postgresql init script.
You need to add parameter --skip-systemctl-redirect to pg_ctlcluster in file /usr/share/postgresql-common/init.d-functions
The function you need to change:
do_ctl_all() {
...
# --skip-systemctl-redirect fix postgresql No such interface 'org.freedesktop.DBus.Properties'
if [ "$1" = "stop" ] || [ "$1" = "restart" ]; then
ERRMSG=$(pg_ctlcluster --skip-systemctl-redirect --force "$2" "$name" $1 2>&1)
else
ERRMSG=$(pg_ctlcluster --skip-systemctl-redirect "$2" "$name" $1 2>&1)
fi
...
}
Ubuntu 14.04 did not switch to systemd yet. I highly recommend upgrading to 16.04 or even better, 18.04.

Why I can't do anything with postgres?

I am using Linux Mint 17. I installed postgres with apt-get. I got postgres user account, I didn't have to create it. Then I login as postgres and followed these steps:
$ initdb -D /usr/local/pgsql/data
...
Success.
You can now start the database server of the Postgres-XC coordinator using:
postgres --coordinator -D /usr/local/pgsql/data
or
pg_ctl start -D /usr/local/pgsql/data -Z coordinator -l logfile
You can now start the database server of the Postgres-XC datanode using:
postgres --datanode -D /usr/local/pgsql/data
or
pg_ctl start -D /usr/local/pgsql/data -Z datanode -l logfile
Then I issued these 2 commands:
$ pg_ctl start -D /usr/local/pgsql/data -Z coordinator -l logfile
$ pg_ctl start -D /usr/local/pgsql/data -Z datanode -l logfile
I didn't get any response.
Then I tried createdb and createuser but I can't. Her you can see some informations, I hope it helps.
darko#darko-Lenovo-G570 ~ $ ps aux | grep postgres
postgre+ 1332 0.0 0.3 53304 6800 ? S 07:59 0:00 /usr/bin/postgres --datanode -D /var/lib/postgres-xc/DN1
postgre+ 1363 0.0 0.3 53304 6808 ? S 07:59 0:00 /usr/bin/postgres --datanode -D /var/lib/postgres-xc/DN2
postgre+ 1372 0.0 0.0 53304 1224 ? Ss 07:59 0:00 postgres: checkpointer process
postgre+ 1373 0.0 0.0 53304 1456 ? Ss 07:59 0:00 postgres: writer process
postgre+ 1374 0.0 0.0 53304 1224 ? Ss 07:59 0:00 postgres: wal writer process
postgre+ 1375 0.0 0.1 53724 2220 ? Ss 07:59 0:00 postgres: autovacuum launcher process
postgre+ 1376 0.0 0.0 23256 1256 ? Ss 07:59 0:00 postgres: stats collector process
postgre+ 1419 0.0 0.0 53304 1232 ? Ss 07:59 0:00 postgres: checkpointer process
postgre+ 1420 0.0 0.0 53304 1464 ? Ss 07:59 0:00 postgres: writer process
postgre+ 1421 0.0 0.0 53304 1232 ? Ss 07:59 0:00 postgres: wal writer process
postgre+ 1423 0.0 0.1 53724 2228 ? Ss 07:59 0:00 postgres: autovacuum launcher process
postgre+ 1424 0.0 0.0 23256 1264 ? Ss 07:59 0:00 postgres: stats collector process
postgre+ 1440 0.0 0.3 53304 6856 ? S 07:59 0:00 /usr/bin/postgres --coordinator -D /var/lib/postgres-xc/CN
postgre+ 1451 0.0 0.0 53304 1224 ? Ss 07:59 0:00 postgres: pooler process
postgre+ 1453 0.0 0.0 53304 1712 ? Ss 07:59 0:00 postgres: checkpointer process
postgre+ 1454 0.0 0.0 53304 1464 ? Ss 07:59 0:00 postgres: writer process
postgre+ 1455 0.0 0.0 53304 1456 ? Ss 07:59 0:00 postgres: wal writer process
postgre+ 1456 0.0 0.5 63548 11736 ? Ss 07:59 0:00 postgres: autovacuum launcher process
postgre+ 1457 0.0 0.0 23256 1264 ? Ss 07:59 0:00 postgres: stats collector process
postgre+ 1481 0.0 0.1 16412 2716 ? S 07:59 0:00 /usr/bin/gtm -D /var/lib/postgres-xc/GTM -l /var/log/postgres-xc/datanode.log
darko 8426 0.0 0.0 5908 856 pts/0 S+ 18:34 0:00 grep --colour=auto postgres
darko#darko-Lenovo-G570 ~ $ sudo -u postgres -i
[sudo] password for darko:
postgres#darko-Lenovo-G570 ~ $ createuser darko
createuser: could not connect to database postgres: FATAL: role "postgres" does not exist
I was confused with roles, databases and privilegies so I decided to remove postgresql and install it again. Actually I think that I didn't install it correctly at first because first installation was
$ sudo apt-get install postgresql
and nothing worked for me, as I described in question. So I decided to kill all postgres processes:
$ sudo pkill postgres
remove postgresql
$ sudo apt-get remove postgresql
and build postgresql form source as described in manual postgresql-9.6-A4.pdf (you can download it from internet). This manual is great, you can find everything there.
MAYBE SOME OF MY COMMANDS ARE NOT 100% CORRECT BECAUSE IT HAS PASSED FEW DAYS SINCE I SOLVED MY PROBLEM.

Correct way to start mysqld_safe

I've been searching around a lot but could not figure out how to start mysqld in "safe mode".
This is what I got so far:
[root#localhost bin]# service mysqld_safe start
mysqld_safe: unrecognized service
I'm running CentOS, this is my mysql version:
[root#localhost ~]# mysql --version
mysql Ver 14.12 Distrib 5.0.95, for redhat-linux-gnu (i686) using readline 5.1
Any help would be appreciated!
Starting mysqld should do the trick:
[root#green-penny ~]# service mysqld start
Starting mysqld: [ OK ]
[root#green-penny ~]# ps axu | grep mysql
root 7540 0.8 0.0 5112 1380 pts/0 S 09:29 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid --basedir=/usr --user=mysql
mysql 7642 1.5 0.7 135480 15344 pts/0 Sl 09:29 0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
root 7660 0.0 0.0 4352 724 pts/0 S+ 09:29 0:00 grep mysql
(Note that mysqld_safe is running.)

postgresql start db

I have postgresql running (/opt/local/lib/postgresql90/bin).
The data base is set up # /Users/demet8/postgres/data.
I check to make sure postgresql is running with
ps aux | grep postgres
demet8 9851 0.0 0.1 614276 1184 ?? Ss 12:40PM 0:00.24 postgres: autovacuum launcher process
demet8 9850 0.0 0.0 614020 436 ?? Ss 12:40PM 0:00.91 postgres: wal writer process
demet8 9849 0.0 0.0 614020 496 ?? Ss 12:40PM 0:01.22 postgres: writer process
demet8 9847 0.0 0.1 614020 2248 s002 S 12:40PM 0:00.61 /opt/local/lib/postgresql90/bin/postgres -D /Users/demet8/postgres/data
demet8 11127 0.0 0.0 599820 468 s002 S+ 2:05PM 0:00.00 grep postgres
demet8 9852 0.0 0.0 610092 368 ?? Ss 12:40PM 0:00.26 postgres: stats collector process
I go back into: /opt/local/lib/postgresql90/bin & now try to start the db with ./pg_ctl start. This is the error msg I get:
pg_ctl: no database directory specified and environment variable PGDATA unset
Try "pg_ctl --help" for more information.
Any suggestions on to what I can be doing wrong. I googled around the web for answers & I haven't found a solution.
$ su -
# su - postgres
Proceed from there.
I know this question is already answered but, if you are new to Postgres and seeing similar issues, try the below commands to resolve the error pg_ctl: no database directory specified and environment variable PGDATA unset Try "pg_ctl --help" for more information.
1. su - postgres
2. find / -name pg_ctl --> gives you the location where pg_ctl present, in my case it is there in `/usr/lib/postgresql/11/bin/`
3. /usr/lib/postgresql/11/bin/pg_ctl -D /var/lib/postgresql/11/main