I am trying to daemonize a celery configuration on an aws instance.
Following the celery docs, I have:
/etc/init.d/celeryd
/etc/default/celerybeat
I've created both a celery user and celery group and set permissions like so:
sudo chown -R celery:celery /var/log/celery/
sudo chown -R celery:celery /var/run/celery/
sudo chown celery:celery /home/sfree/meampy/bin/celery
When I check the file's permissions, it looks good:
(meampy)[]$ ls -l /home/sfree/meampy/bin/celery
-rwxrwxrwx 1 celery celery 237 Sep 13 15:15 /home/sfree/meampy/bin/celery
But when I run the script:
sudo sh -x /etc/init.d/celeryd start
...
Starting celeryd...
+ _chuid -f /var/log/celery/beat.log -l INFO --detach --pidfile=/var/run/celery/beat.pid
+ su celery -c '/home/sfree/meampy/bin/celery beat -f /var/log/celery/beat.log -l INFO --detach --pidfile=/var/run/celery/beat.pid'
bash: /home/sfree/meampy/bin/celery: Permission denied
+ exit 0
If I run the offending line solo, I get the same error.
meampy is the name of my virtualenv. Is the virtualenv the reason I am running into permission problems?
EDIT: the permissions on the virtualenv:
lrwxrwxrwx 1 sfree www-data 24 Sep 1 19:49 meampy -> /usr/local/python/meampy
I added the celery user to the www-data group, still same error
Because virtualenv meampy is belong to sfree, you should modify your celery config file (/etc/default/celeryd)
set
CELERYD_USER="sfree"
CELERYD_GROUP="sfree"
and do not forget
sudo chown -R sfree:sfree /var/log/celery/
sudo chown -R sfree:sfree /var/run/celery/
Related
using vscode and wsl2, I have tried to launch a container using the default method and no customization. This generated the same error as below.
so following vscode docs I set a "workspaceMount" in devcontainer.json
"workspaceMount": "source=${localWorkspaceFolder},target=/workspaces/myRepo,type=bind,consistency=delegated",
"workspaceFolder": "/workspaces",
I select Reopen in container, the launch sequence happens but an error is generated
a mount config is invalid, make sure it has the right format and a source folder that exists on the machine where the Docker daemon is running
the log error is
Command failed: docker run -a STDOUT -a STDERR --mount source=d:\git\myRepo,target=/workspaces/myRepo,type=bind,consistency=delegated --mount type=volume,src=vscode,dst=/vscode -l vsch.quality=stable -l vsch.remote.devPort=0 -l vsch.local.folder=d:\git\myRepo --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --entrypoint /bin/sh vsc-myRepo-a878aa9edbcf04f717c76e764dabcde6 -c echo Container started ; trap "exit 0" 15; while sleep 1 & wait $!; do :; done
by launching the container from docker desktop I can confirm
cd /workspaces
ls -l
drwxr-xr-x 2 root root 4096 Dec 3 11:48 myRepo
Is this issue due to owner root:root ?
Should this be changed by chown in the Dokerfile? if so could you provide a sample code to do this, is it by RUN chown ...?
I guess you followed the documentation in https://code.visualstudio.com/docs/remote/containers-advanced
The source should contains the subfolder "myRepo" and the target only "workspaces"
"workspaceMount": "source=${localWorkspaceFolder}/myRepo,target=/workspaces,type=bind,consistency=delegated",
"workspaceFolder": "/workspaces",
I'm trying to redirect stdout and stdin from within a script, designed to be run by either root or a privileged user, to a directory (/var/log/backups/) which is owned by root, group syslog. It works fine when run as root, but not when run as a user belonging to group syslog.
$ grep syslog /etc/group
adm:x:4:syslog,ubuntu,me
syslog:x:108:me
$ ls -dl /var/log/backup
drwxrwxr-x 2 root syslog 4096 Jun 8 14:17 /var/log/backup
part of my script:
LOG_DIR="/var/log/backup/"
sudo_cmd=
user=`whoami`
if [ ${user} != "root" ]; then
sudo_cmd="sudo"
fi
echo "sudo_cmd: " ${sudo_cmd}
...
logfile=${LOG_DIR}${filnam}.log
${sudo_cmd} touch ${logfile}
${sudo_cmd} chgrp syslog ${logfile}
${sudo_cmd} chmod 664 ${logfile}
${sudo_cmd} exec >> ${logfile} 2>&1
Even when run outside of the script, the redirection fails when run as a not-root user with group write access:
$ ls -l $logfile
-rw-rw-r-- 1 root syslog 0 Jun 8 15:08 /var/log/backup/foo.log
$ ${sudo_cmd} exec >> ${logfile} 2>&1
sh: 13: cannot create /var/log/backup/foo.log: Permission denied
clues?
I am creating a docker image for postgresql installation from sources. Here is my Dockerfile which I created according to the postgresql documentation:
FROM ubuntu
RUN apt-get update && apt-get install gcc zlib1g-dev libreadline6-dev apt-utils make -y
RUN mkdir -p /tmp/downloads
ADD https://ftp.postgresql.org/pub/source/v9.6.6/postgresql-9.6.6.tar.gz /tmp/downloads
RUN cd /tmp/downloads && tar -zxf postgresql-9.6.6.tar.gz
RUN cd /tmp/downloads/postgresql-9.6.6 && make configure
RUN cd /tmp/downloads/postgresql-9.6.6 && ./configure
RUN cd /tmp/downloads/postgresql-9.6.6 && make
RUN cd /tmp/downloads/postgresql-9.6.6 && su
RUN cd /tmp/downloads/postgresql-9.6.6 && make install
RUN cd /tmp/downloads/postgresql-9.6.6 && adduser postgres
RUN cd /tmp/downloads/postgresql-9.6.6 && mkdir /usr/local/pgsql/data
RUN cd /tmp/downloads/postgresql-9.6.6 && chown postgres /usr/local/pgsql/data
RUN cd /tmp/downloads/postgresql-9.6.6 && su postgres
RUN cd /tmp/downloads/postgresql-9.6.6 && /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
RUN cd /tmp/downloads/postgresql-9.6.6 && /usr/local/pgsql/bin/createdb test
RUN cd /tmp/downloads/postgresql-9.6.6 && /usr/local/pgsql/bin/psql test
So when I run docker build -t roksolanad/psql:latest . I get errors:
initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.
The command '/bin/sh -c cd /tmp/downloads/postgresql-9.6.6 && /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data' returned a non-zero code: 1
So how can I fix my Dockerfile? I would be very grateful for some help!
Problem is that each RUN creates its own shell. Which means that username changes only persistent for that one instance. This is different from any filesystem changes which will persist.
Try chaining your commands together like this instead:
RUN cd /tmp/downloads/postgresql-9.6.6 &&\
make configure &&\
./configure &&\
make &&\
su &&\
make install &&\
adduser postgres &&\
mkdir /usr/local/pgsql/data &&\
chown postgres /usr/local/pgsql/data
RUN cd /tmp/downloads/postgresql-9.6.6 &&\
su postgres &&\
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data &&\
/usr/local/pgsql/bin/createdb test &&\
/usr/local/pgsql/bin/psql test
As said by #Harald Nordgren, you need sum up all the RUN commands in one command, if possible.
Along with that there are couple of things which are causing the failures
1) adding "postgres" user:
"adduser" expects the additional parameters to be configured when you add a user, as you have mentioned in the comments like asking for password and so. So you need to modify the command to disable the arguments as well as the password like below:
adduser postgres --gecos '' --disabled-login
2) executing postgress using root user
When you execute the command "su postgres", it executes with root user. But whereas we have changed the permissions in the above command "chown postgres /usr/local/pgsql/data"
For this you need a execute the command as "postgres" user which can be enabled by adding USER in dockerfile.
Finally your Dockerfile looks something like this:
FROM ubuntu
RUN apt-get update && apt-get install gcc zlib1g-dev libreadline6-dev apt-utils make -y
RUN mkdir -p /tmp/downloads
ADD https://ftp.postgresql.org/pub/source/v9.6.6/postgresql-9.6.6.tar.gz /tmp/downloads
RUN cd /tmp/downloads && tar -zxf postgresql-9.6.6.tar.gz
RUN cd /tmp/downloads/postgresql-9.6.6 &&\
make configure &&\
./configure &&\
make &&\
su &&\
make install &&\
adduser postgres --gecos '' --disabled-login &&\
mkdir /usr/local/pgsql/data &&\
chown postgres /usr/local/pgsql/data
USER postgres
#use below command only if it is necessary, it is similar to "cd" linux command
WORKDIR /tmp/downloads/postgresql-9.6.6
RUN /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
With this I am able to create a Docker image and tested as well. Adding more information which may be useful.
# docker run -it postgres:2.0 /bin/bash
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ ps -eaf
UID PID PPID C STIME TTY TIME CMD
postgres 1 0 0 10:58 ? 00:00:00 /bin/bash
postgres 9 1 0 10:59 ? 00:00:00 ps -eaf
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start
server starting
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ ps -eaf
UID PID PPID C STIME TTY TIME CMD
postgres 1 0 0 10:58 ? 00:00:00 /bin/bash
postgres 13 1 0 10:59 ? 00:00:00 /usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data
postgres 15 13 0 10:59 ? 00:00:00 postgres: checkpointer process
postgres 16 13 0 10:59 ? 00:00:00 postgres: writer process
postgres 17 13 0 10:59 ? 00:00:00 postgres: wal writer process
postgres 18 13 0 10:59 ? 00:00:00 postgres: autovacuum launcher process
postgres 19 13 0 10:59 ? 00:00:00 postgres: stats collector process
postgres 20 1 0 10:59 ? 00:00:00 ps -eaf
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ /usr/local/pgsql/bin/createdb test
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ /usr/local/pgsql/bin/psql test
psql (9.6.6)
Type "help" for help.
test=#
test=#
There are other ways to build this docker image but this way I am able to.
I believe I ended up mixing up permissions at /etc/ssl directories tree as the last modification was made on 18th November and a day after I could not get my PostgreSQL to work.
When I type in
sudo service postgresql start
I get
FATAL: could not access private key file “/etc/ssl/private/ssl-cert-snakeoil.key”: Permission denied
Checking permissions
~$ sudo -i
~$ ls -la /etc/ssl/private
drw-r----- 2 root ssl-cert 4096 Nov 18 21:10 .
-rwxrwxrwx 1 postgres postgres 1704 Set 4 11:26 ssl-cert-snakeoil.key
Checking group composition
~$ id postgres
uid=114(postgres) gid=127(postgres) groups=127(postgres),114(ssl-cert)
Also I noticed that my ssl-cert-snakeoil.pem file at /etc/ssl/certs/ doesn't have a symlink. I don't know if this makes any difference...
Please, help me sort this out.
Thanks.
Edit: Should it be posted on serverfault instead?
Try adding postgres user to the group ssl-cert
Run the below code to fix your issue:
# > It happened to me and it turned out that I removed erroneously the postgres user from "ssl-cert" group, set it back with
sudo gpasswd -a postgres ssl-cert
# Fixed ownership and mode
sudo chown root:ssl-cert /etc/ssl/private/ssl-cert-snakeoil.key
sudo chmod 740 /etc/ssl/private/ssl-cert-snakeoil.key
# now postgresql starts! (and install command doesn't fail anymore)
sudo /etc/init.d/postgresql start
courtsey to GabLeRoux
Check the output of
$ sudo -u postgres
$ cd /etc/ssl/private
$ ls
If the response is "Permission denied" do
$ chown postgres:ssl-cert /etc/ssl/private/
$ chown postgres:postgres /etc/ssl/private/ssl-cert-snakeoil.key
Only thing that will work if you have changed permissions for /etc/ssl/private
mkdir /etc/ssl/private-copy; mv /etc/ssl/private/* /etc/ssl/private-copy/; rm -r /etc/ssl/private; mv /etc/ssl/private-copy /etc/ssl/private; chmod -R 0700 /etc/ssl/private; chown -R postgres /etc/ssl/private
Copy this whole command (It's a one line code).
If this doesn't work for you, ckeck your postgres user groups by groups postgres and make sure your postgres user have ssl-cert root postgres (Order doesn't matter).
Now lets check your file permissions on ssl/private :
$ ls -la /etc/ssl/
> drwx------ 2 postgres root private
If this is not the output change your permissions with sudo chmod -R 700 /etc/ssl/private and for owners chown -R postgres:root /etc/ssl/private
//Now check permissions on ssl-cert-snakeoil.key,
//which will be inside your **private** directory.
$ ls -la /etc/ssl/private/ssl-cert-snakeoil.key
> -rwx------ 1 postgres root /etc/ssl/private/ssl-cert-snakeoil.key
I was suffering from this issue when attempting to start Postgresql on a remote docker instance. I eventually tracked down the crazy solution here. Basically you have to recreate the directories, chown on it's own doesn't work:
mkdir /etc/ssl/private-copy; mv /etc/ssl/private/* /etc/ssl/private-copy/; rm -r /etc/ssl/private; mv /etc/ssl/private-copy /etc/ssl/private; chmod -R 0700 /etc/ssl/private; chown -R postgres /etc/ssl/private
This error was preventing my PostgreSQL server from running locally.
The following worked for me:
sudo chown postgres:postgres /etc/ssl/private/ssl-cert-snakeoil.key
sudo chmod 600 /etc/ssl/private/ssl-cert-snakeoil.key
Also make sure that /etc/ssl/private has enough permissions.
Some programs can be incredibly pedantic and cost you valuable hours. By running journalctl after sudo systemctl start postgresql I'd see various errors like:
FATAL: could not load private key file "/etc/ssl/private/ssl-cert-snakeoil.key": Permission denied
FATAL: private key file "/etc/ssl/private/ssl-cert-snakeoil.key" must be owned by the database user or root
FATAL: private key file "/etc/ssl/private/ssl-cert-snakeoil.key" has group or world access
DETAIL: File must have permissions u=rw (0600) or less if owned by the database user, or permissions u=rw,g=r (0640) or less if owned by root.
I couldn't make it with work sudo chmod root:root, so I had to settle for sudo chmod postgres:postgres.
EDIT
I haven't tried it, but running deleting and regenerating the snakeoil certificate might work as well:
make-ssl-cert generate-default-snakeoil --force-overwrite
(You may have to run it with sudo, don't know.)
Try setting permissions on the .key file to 600. Postgres doesn't like key files with group or world permissions set. You may also need to change the owner to postgres, though I'm not sure about that.
I am running the postgres server in WSL, and I was facing the error with the ssl-cert file. I managed to make it work by changing the owner of the file to the postgres user I had created, adding the expected user and group IDs to the user as required of the application (111 and 116, respectively, as gleaned from helpful error messages), and voila, I have an active server from within WSL.
sudo useradd postgres
sudo usermod -u 111 -g 116 -a -G ssl-cert postgres
sudo chown postgres /etc/ssl/private/ssl-cert-snakeoil.key
After running the above, there were two more files the user running the server (postgres for me) needed permission to access, both residing in /var/postgresql. I used sudo chown -- twice more to give ownership to postgres. Running sudo service postgresql start will tell you which files you'll need to transfer ownership of through any error messages.
I had other certificates under /etc/ssl/private and hence, changing permissions recursively was out of question.
I tried adding postgres user to ssl-cert group that didn't help either.
I modified the permission of /etc/ssl/private to 716, basically saying that anyone else other than root (user) and ssl-cert (group) can read and execute the directory.
sudo chmod 716 /etc/ssl/private
Then, I modified the ownership of ssl-cert-snakeoil.key
sudo chown postgres:postgres /etc/ssl/private/ssl-cert-snakeoil.key
This worked for me, basically a combination of the answers by #devops and #Noushad
I'm trying to run postgresql on my local machine like I usually do, however it's putting me in a situation where I can't fix. I installed postgresql91 with macports.
These are the three commands I usually have to run to get it running:
sudo sysctl -w kern.sysv.shmall=4096
sudo sysctl -w kern.sysv.shmmax=16777216
sudo su postgres -c "/opt/local/lib/postgresql91/bin/postgres -D /opt/local/var/db/postgresql91/defaultdb -p 55432"
However, it's giving me this error today:
Nets-Mac-Pro:~ emai$ sudo sysctl -w kern.sysv.shmall=4096
Password:
kern.sysv.shmall: 4096 -> 4096
Nets-Mac-Pro:~ emai$ sudo sysctl -w kern.sysv.shmmax=16777216
kern.sysv.shmmax: 16777216 -> 16777216
Nets-Mac-Pro:~ emai$ sudo su postgres -c "/opt/local/lib/postgresql91/bin/postgres -D /opt/local/var/db/postgresql91/defaultdb -p 55432"
postgres cannot access the server configuration file "/opt/local/var/db/postgresql91/defaultdb/postgresql.conf": Permission denied
When I go to /opt/local/var/db/postgresql91/ and do an ls -l this is what comes up:
drwx------ 18 root wheel 612 Jun 28 12:44 defaultdb
So I decided to add the postgres user to the wheel group, and then chmod defaultdb to 770.
drwxrwx--- 18 root wheel 612 Jun 28 12:44 defaultdb
I still get the error:
FATAL: could not open configuration file "/opt/local/var/db/postgresql91/defaultdb/postgresql.conf": Permission denied
And so I change the file rights from:
-rw------- 1 root wheel 19170 Jan 7 11:52 postgresql.conf
to:
-rw-rw---- 1 root wheel 19170 Jan 7 11:52 postgresql.conf
And now it complains that when I run the command again:
Nets-Mac-Pro:~ emai$ sudo su postgres -c "/opt/local/lib/postgresql91/bin/postgres -D /opt/local/var/db/postgresql91/defaultdb -p 55432"
FATAL: data directory "/opt/local/var/db/postgresql91/defaultdb" has wrong ownership
HINT: The server must be started by the user that owns the data directory.
I have no clue how I used to run the postgres server considering the file permissions of the files. Where do I find the data folder that it is hinting me about? Is there a better way to fix this?
Postgres should be owner, and the only user capable of writing to, data directory.
So, do:
sudo chown -Rf postgres:postgres /opt/local/var/db/postgresql91/defaultdb
sudo chmod 700 /opt/local/var/db/postgresql91/defaultdb
and it should be fine.