I am creating a local Label Studio server to host images to annotate in our office. I would like the database back end to be postgresql and not sqlite and be located in a particular directory, not the default and not the same as the 'data-dir'. I have got a test server working across the network with various machines annotating images on the server, but the backend was sqlite for this test.
Everything I've tried to get a postgresql backend db has failed for various reasons. Some commands result in a sqlite db (occasionally with the name 'postgresql') located in my required directory; others create postgres/pyscopg2 errors but I think they're up a garden path.
The host machine is running Ubuntu 20.04 LTS. And serves another postgresql db over the network using other APIs. Postgresql version running is 12.9.
I have created a conda environment and pip installed Label Studio as the documentation suggested.
Here's what I've tried:
Start the conda environment. Follow instructions to assign environment variables from https://labelstud.io/guide/storedata.html#PostgreSQL-database which at time of writing is:
DJANGO_DB=default
POSTGRE_NAME=postgres
POSTGRE_USER=postgres
POSTGRE_PASSWORD=
POSTGRE_PORT=5432
POSTGRE_HOST=db
Then a few variations on the start command (I didn't include the backslashes, just put here for readability/comparability):
label-studio start --init \
-db postgresql \
--database /path/to/label-studio/databases/newdb \
--data-dir /path/to/label-studio/media_dirs/test_proj
result: db is where expected, but:
file newdb
gives "newdb: SQLite 3.x database, last written using SQLite version 3038002"
label-studio start --init \
--database /path/to/label-studio/databases/newdb \
-db postgresql \
--data-dir /path/to/label-studio/media_dirs/test_proj
result: a db at specified path named 'postgresql' and still an sqlite db. This seems to mirror the mistake mentioned at: https://github.com/heartexlabs/label-studio/issues/1660
I have also tried the above two commands with the '--init' argument omitted with same results.
Then I tried adding something on the front of the command suggested at the same link above:
DJANGO_DB=default label-studio start \
--database /path/to/label-studio/databases/newdb \
--data-dir /path/to/label-studio/media_dirs/test_proj
result: psycopg2.OperationalError: FATAL: password authentication failed for user "postgres"
FATAL: password authentication failed for user "postgres"
DJANGO_DB=default POSTGRE_PASSWORD= label-studio start \
--database /path/to/label-studio/databases/newdb \
--data-dir /path/to/label-studio/media_dirs/test_proj
result: psycopg2.OperationalError: fe_sendauth: no password supplied
Any help and resolution would be highly appreciated.
Also, I can't tag this with 'label-studio' because I'm not quite at the required reputation to create a new tag, so if anyone who can feels like doing so, pleaseandthankyou!
Your last option was closer than all the others. Have you tried to run LS using this:
DJANGO_DB=default POSTGRE_NAME=<postgres_name> POSTGRE_USER=<postgres_user> POSTGRE_PASSWORD=<password> POSTGRE_PORT=<db_port> POSTGRE_HOST=<db_host> label-studio
Sure, you have to run postgres service by yourself, configure it properly, create the DB <postgres_name>, the user <postgres_user> and set the password <password>, grant access rights to this user. Also don't forget to specify <db_host> (localhost?), <db_port> (5432?)
Related
I have installed Postgres 12 on Ubuntu by building it from source and I am facing two issues:
Although I followed the installation manual from Postgrez, every time I restart my computer, my Postgres server stopz and is no longer seen as a running process.
To start it the first time after install, I do this from the terminal:
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
/usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start
After a restart, to start DB again when I run: /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data, it throws this error:
initdb: error: directory "/usr/local/pgsql/data" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "/usr/local/pgsql/data" or run initdb
with an argument other than "/usr/local/pgsql/data".
Does that mean that every time I start Postgres after a restart, I have to create a new /data directory?
Upon installing Postgres sing pip or pip3, one can just switch user to postgres and run psql to enter postgres, however now I have to run "/usr/local/bin/psql". Please note I have exported all the paths per https://www.postgresql.org/docs/12/installation.html. How can I fix this? Can an alias be set for this?
After a restart, to start DB again when I run:
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data, it throws this
error:
Does that mean that every
time I start Postgres after a restart, I have to create a new /data
directory?
No, quite the opposite. You don't need to initdb after the first time, you just need to start. It is your attempt to initdb when you don't need to which is causing the error message. Note that attempting to initdb isn't doing any harm, because it refused to run. It just generates log/console noise.
Upon installing Postgres sing pip or pip3, one can just switch user to
postgres and run psql to enter postgres, however now I have to run
"/usr/local/bin/psql". Please note I have exported all the paths per
https://www.postgresql.org/docs/12/installation.html. How can I fix
this?
I don't know what your first sentence means, as you don't use pip or pip3 to install PostgreSQL (or at least, the docs don't describe doing so) although you might use them to install psycopg2 to enable python to talk to PostgreSQL.
You could use an alias, but it would probably make more sense to edit ~/.bash_profile to set the PATH, as described from the page you linked to under Environment Variables.
You have to register postgreSQL as a service.
run this:
pg_ctl register [-N servicename] [-U username] [-P password] [-D datadir] [-S a[uto] | d[emand] ] [-w] [-t seconds] [-s] [-o options]
Example:
pg_ctl register -N postgresql -U OS_username -P OS_password -D '/etc/postgresql/12/data' -w
More info in the manual: pg_ctl
Notes:
Username and Password is related to the OS, not postgresql
If you have doubts read the manual.
/usr/local/pgsql/bin/pg_ctl start -D '/usr/local/pgsql/data'
Export following in postgres user account's ~/.bashrc:
LD_LIBRARY_PATH=/usr/local/pgsql/lib
export LD_LIBRARY_PATH
PATH=/usr/local/pgsql/bin:$PATH
export PATH
So currently I'm working on a project on Heroku with Drupal and my issue is that I want to reset the database each time I deploy to master, yes I know it not ideal but its a development env because I'm working Drupal plugin and it would be nice if changes happened it could just reset to a state.
But when I try to connect using psql and some variables I just get password authentication failed for user even tho I know its the right password because I got it from Heroku itself.
Currently, I have tried using the console to try to make in connection soi could run a DROP TABLE command for me to afterword import an SQL file with the basic setup using pg_dump, and put it into a .sh script and run it with and release: in a procfile
Until now I have this as a release.sh file where I only tried in the console on heroku
PGHOST=HOST PGPORT=5432 \
PGDATABASE=DB \
PGUSER=USER PGPASSWORD=SOMEPASS \
psql
Try below command to reset DB
heroku pg:reset DATABASE_URL
Cannot find any tutorials that show what you do when you deleted default postgres user through root.
Created local postgres server through sudo apt-get install postgresql postgresql-contrib. Inside shell, typed psql and was able to initiate db.
Logged into root and deleted postgres account in linux while trying something out.
2a: sudo pkill -KILL -u postgres
2b: sudo userdel postgres
Deleted postgres server. Re-installed. Got user not found when trying to log into postgres user account through shell.
Tried running postgres server through root and got unable to connect to server through shell.
Re-created postgres user account through root: sudo adduser postgres --disabled-password.
Postgres user through root: sudo usermod -a -G sudo postgres
Still getting unable to connect to server: connection refused. Logged into postgres user. No change.
Re-installed postgres. Same error.
Suggestions?
Thanks a bunch.
You don't exactly need a user who is named postgres, you need a user who owns the data directory, the binaries and configuration files. You should check which uid and gid are now associated with your data directory, then create a user with that specific uid and gid. But check who those id's belong to first. If no user currently has those uid and gid then create a new user with those values:
useradd -u 123 -g 456 postgres
Alternatively, re-create user postgres and assign data directory and binaries to that new user (assuming group postgres still exists):
useradd -G postgres postgres
chown -r postgres:postgres $PGDATA
chown -r postgres:postgres /wherever/bin/is
chown -r postgres:postgres /wherever/conf/is
Found the answer, Read below.
Been on another rabbit hole hunt. Heard about the data file,
how someone deleted it and it has all kinds of info(config
stuff) in there. Also heard how some config files get left
around even after uninstalling the program. Not just config
files but a lot of other files get left around also with
other programs that I have installed and uninstalled.
So here I go to start just deleting a bunch of stuff and then
I go to this link to postgresql.org where it shows about
where the 'data file' is. Or I think maybe it is the 'data file'.
This page says the data file is usually in
'/var/lib/pgsql/data'. I uninstalled postgresql so I didn't
have a pgsql folder but I saw that there was a 'postgresql' folder
there. So I deleted it and now all is well. Now I am
able to reinstall postgresql with no problems, no cluster
problems or nothing. The 'posgresql' folder had 2 hidden files,
I've learned from the past to always check if there are
hidden files. These hidden files names were 'bash_history' and
'psql_history' & you know they were somehow
stopping a fresh complete installation. After deleting them I
was able to reinstall postgresql, with a newly created
'postgres' user waiting for me.
Its 3:43am in the morning & another rabbit hole in the books.
Happy hunting.
Here is the link that I mentioned:
[https://www.postgresql.org/docs/current/storage-file-layout.html][1]
Is there a way to get console access to Dokku's PostgreSQL plugin? On Heroku I'd do heroku pg:psql. Is this possible in a Dokku environment and if so how?
There is in fact a way to do this directly with the dokku-pg-plugin.
The command postgresql:restore <db> < dump_file.sql connects to the specified database and restores it with the provided dump file. If you simply omit the dump file part (< dump_file.sql), a psql console session opens.
Since postgresql:restore <db> is semantically not the best way to open a console session, I have opened a pull request to add the command postgresql:console <db>.
So, until my PR is merged, the options for opening a psql console for a database are either:
doing it manually with psql -h 172.17.42.1 -p <port> -U root db with the <port> and password taken from the output of dokku postgresql:info <db>,
using the semantically incorrect command dokku postgresql:restore <db>, or
use my forked and patched version of the plugin which adds the command postgresql:console <db>.
Edit:
The owner of the dokku-pg-plugin has merged my pull request. If you're using this plugin and are looking for a way to access your PostgreSQL console with it, you might want to update it to the latest version. Once you have done that, you can use the command postgresql:console <db> to open a psql session to the specified database.
This worked for me for my Rails app that I'm running on Dokku:
dokku run <app-name> rails db
That brought up the console for the PostgreSQL container I created (via dokku postgresql:create <db>). I couldn't figure out another way to get at the PostgreSQL instance in that container, short of attempting to directly connect to the DB, with the connection info/credentials listed when you do this:
dokku postgresql:info <db>
I haven't tried that, though I suspect it would work.
With a freshly installed version of Postgres 9.2 via yum repository on Centos 6, how do you run postgres as a different user when it is configured to run as 'postgres:postgres' (u:g) out of the box?
In addition to AndrewPK's explanation, I'd like to note that you can also start new PostgreSQL instances as any user by stopping and disabling the system Pg service, then using:
initdb -D /path/to/data/directory
pg_ctl start -D /path/to/data/directory
This won't auto-start the server on boot, though. For that you must integrate into your init system. On CentOS 6 a simple System V-style init script in /etc/init.d/ and a suitable symlink into /etc/rc3.d/ or /etc/rc3.d/ (depending on default runlevel) is sufficient.
If running more than one instance at a time they must be on different ports. Change the port directive in postgresql.conf in the datadir or set it on startup with pg_ctl -o "-p 5433" .... You may also need to override the unix_socket_directories if your user doesn't have write permission to the default socket directory.
pg_ctl
initdb
This is only for a fresh installation (as it pertained to my situation) as it involves blowing away the data dir.
The steps I took to resolve this issue while utilizing the packaged startup scripts for a fresh installation:
Remove the postgres data dir /var/lib/pgsql/9.2/data if you've already gone through the initdb process with the postgres user:group configured as default.
Modify the startup script (/etc/init.d/postgresql-9.2) to replace all instances of postgres:postgres with NEWUSER:NEWGROUP.
Modify the startup script to replace all instances of postgres in any $SU -l postgres lines with the NEWUSER.
run /etc/init.d/postgres initdb to regenerate the cluster using the new username
Make sure any logs created are owned by the new user or remove old logs if error on initdb (the configuration file in my case was found in /var/lib/pgsql/9.2/data/postgresql.conf).
Startup postgres and it should now be running under the new user/group.
I understand this might not be what other people are looking for if they have existing postgres db's and want to restart the server to run as a different user/group combo - this was not my case, and I didn't see an answer posted anywhere for a 'fresh' install utilizing the pre-packaged startup scripts.