Where are PostgreSQL database files in the cluster directory? - postgresql

On OSX, I recently installed PostgreSQL via Homebrew:
brew install postgresql
I then created a new database cluster:
initdb /usr/local/var/postgres
I confirm that postgresql server is running with the expected database cluster:
$ ps auxwww | grep postgres
0:00.03 /usr/local/opt/postgresql/bin/postgres -D /usr/local/var/postgres
I create a new database:
createdb mynewdb
I see that it exists.
$ psql
<user>=# \l
mynewdb | <user> | UTF8 | en_US.UTF-8 | en_US.UTF-8
But, I see no obvious changes to the cluster directory (e.g., just by checking contents ls -lt).
Where is the database written / stored in the cluster directory (or sub-directories)?

as #ahorsewithnoname says they are in the base directory, (on some other platfotms data/base)
the numeric name is the OID of the database and in the directory will be files named for the OID of the table or index they relate to.
databse oids can be determined with this query. oid is like a hidden unique id column.
select oid,* from pg_catalog.pg_database;
table and index oids can be found like this,
select * from pg_catalog.pg_class;
column relifilenode for the base of the filename (thanks #jjames)

Related

postgresql permissions denied [duplicate]

I'm using the PostgreSql app for mac (http://postgresapp.com/). I've used it in the past on other machines but it's giving me some trouble when installing on my macbook. I've installed the application and I ran:
psql -h localhost
It returns:
psql: FATAL: database "<user>" does not exist
It seems I can't even run the console to create the database that it's attempting to find. The same thing happens when I just run:
psql
or if I launch psql from the application drop down menu:
Machine stats:
OSX 10.8.4
psql (PostgreSQL) 9.2.4
Any help is appreciated.
I've also attempted to install PostgreSql via homebrew and I'm getting the same issue. I've also read the applications documentation page that states:
When Postgres.app first starts up, it creates the $USER database,
which is the default database for psql when none is specified. The
default user is $USER, with no password.
So it would seem the application is not creating $USER however I've installed->uninstalled-reinstalled several times now so it must be something with my machine.
I found the answer but I'm not sure exactly how it works as the user who answered on this thread -> Getting Postgresql Running In Mac: Database "postgres" does not exist didn't follow up. I used the following command to get psql to open:
psql -d template1
I'll leave this one unanswered until someone can provide an explanation for why this works.
It appears that your package manager failed to create the database named $user for you. The reason that
psql -d template1
works for you is that template1 is a database created by postgres itself, and is present on all installations.
You are apparently able to log in to template1, so you must have some rights assigned to you by the database. Try this at a shell prompt:
createdb
and then see if you can log in again with
psql -h localhost
This will simply create a database for your login user, which I think is what you are looking for. If createdb fails, then you don't have enough rights to make your own database, and you will have to figure out how to fix the homebrew package.
From the terminal, just Run the command on your command prompt window. (Not inside psql).
createdb <user>
And then try to run postgres again.
By default, postgres tries to connect to a database with the same name as your user. To prevent this default behaviour, just specify user and database:
psql -U Username DatabaseName
Login as default user: sudo -i -u postgres
Create new User: createuser --interactive
When prompted for role name, enter linux username, and select Yes to superuser question.
Still logged in as postgres user, create a database: createdb <username_from_step_3>
Confirm error(s) are gone by entering: psql at the command prompt.
Output should show psql (x.x.x) Type "help" for help.
Login using default template1 database:
#psql -d template1
#template1=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+---------+----------+-------------+-------------+---------------------
postgres | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gogasca +
| | | | | gogasca=CTc/gogasca
template1 | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gogasca +
| | | | | gogasca=CTc/gogasca
(3 rows)
Create a database with your userId:
template1=# CREATE DATABASE gogasca WITH OWNER gogasca ENCODING 'UTF8';
CREATE DATABASE
Quit and then login again
template1=# \q
gonzo:~ gogasca$ psql -h localhost
psql (9.4.0)
Type "help" for help.
gogasca=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+---------+----------+-------------+-------------+---------------------
gogasca | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
postgres | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gogasca +
| | | | | gogasca=CTc/gogasca
template1 | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gogasca +
| | | | | gogasca=CTc/gogasca
(4 rows)
I faced the same error when I trying to open postgresql on mac
psql: FATAL: database "user" does not exist
I found this simple command to solve it:
method1
$ createdb --owner=postgres --encoding=utf8 user
and type
psql
Method 2:
psql -d postgres
Post installation of postgres, in my case version is 12.2, I did run the below command createdb.
$ createdb `whoami`
$ psql
psql (12.2)
Type "help" for help.
macuser=#
Step 1:
psql -d template1
now you should be on psql terminal
Step 2:
CREATE DATABASE username;
make sure you use semicolon (;) after the database name;
optional: on psql terminal type \ls or \l to list all the databases;
Step 3:
psql -h localhost
now you should be connected;
Try using-
psql -d postgres
I was also facing the same issue when I ran psql
Had the same problem, a simple psql -d postgres did it (Type the command in the terminal)
This error can also occur if the environment variable PGDATABASE is set to the name of a database that does not exist.
On OSX, I saw the following error while trying to launch psql from the Postgress.app menu:
psql: FATAL: database "otherdb" does not exist
The solution to the error was to remove export PGDATABASE=otherdb from ~/.bash_profile:
Further, if PGUSER is set to something other than your username, the following error will occur:
psql: FATAL: role "note" does not exist
The solution is to remove export PGUSER=notme from ~/.bash_profile.
Not sure if it is already added in the answers, Anatolii Stepaniuk answer was very helpful which is the following.
psql -U Username postgres # when you have no databases yet
As the createdb documentation states:
The first database is always created by the initdb command when the data storage area is initialized... This database is called postgres.
So if certain OS/postgresql distributions do that differently, it is certainly not the default/standard (just verified that initdb on openSUSE 13.1 creates the DB "postgres", but not "<user>"). Long story short, psql -d postgres is expected to be used when using a user other than "postgres".
Obviously the accepted answer, running createdb to create a DB named like the user, works as well, but creates a superfluous DB.
Since this question is the first in search results, I'll put a different solution for a different problem here anyway, in order not to have a duplicate title.
The same error message can come up when running a query file in psql without specifying a database. Since there is no use statement in postgresql, we have to specify the database on the command line, for example:
psql -d db_name -f query_file.sql
First off, it's helpful to create a database named the same as your current use, to prevent the error when you just want to use the default database and create new tables without declaring the name of a db explicitly.
Replace "skynotify" with your username:
psql -d postgres -c "CREATE DATABASE skynotify ENCODING 'UTF-8';"
-d explicitly declares which database to use as the default for SQL statements that don't explicitly include a db name during this interactive session.
BASICS FOR GETTING A CLEAR PICTURE OF WHAT YOUR PostgresQL SERVER has in it.
You must connect to an existing database to use psql interactively. Fortunately, you can ask psql for a list of databases:
psql -l
.
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
----------------------------------+-----------+----------+-------------+-------------+-------------------
skynotify | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
myapp_dev | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
postgres | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
ruby-getting-started_development | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/skynotify +
| | | | | skynotify=CTc/skynotify
template1 | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/skynotify +
| | | | | skynotify=CTc/skynotify
(6 rows)
This does NOT start the interactive console, it just outputs a text based table to the terminal.
As another answers says, postgres is always created, so you should use it as your failsafe database when you just want to get the console started to work on other databases. If it isn't there, then list the databases and then use any one of them.
In a similar fashion, select tables from a database:
psql -d postgres -c "\dt;"
My "postgres" database has no tables, but any database that does will output a text based table to the terminal (standard out).
And for completeness, we can select all rows from a table too:
psql -d ruby-getting-started_development -c "SELECT * FROM widgets;"
.
id | name | description | stock | created_at | updated_at
----+------+-------------+-------+------------+------------
(0 rows)
Even if there are zero rows returned, you'll get the field names.
If your tables have more than a dozen rows, or you're not sure, it'll be more useful to start with a count of rows to understand how much data is in your database:
psql -d ruby-getting-started_development -c "SELECT count(*) FROM widgets;"
.
count
-------
0
(1 row)
And don't that that "1 row" confuse you, it just represents how many rows are returned by the query, but the 1 row contains the count you want, which is 0 in this example.
NOTE: a db created without an owner defined will be owned by the current user.
had the problem with using the JDBC driver, so one just has to add the database (maybe redundantly depending on the tool you may use) after the host name in the URL, e.g.
jdbc:postgres://<host(:port)>/<db-name>
further details are documented here: http://www.postgresql.org/docs/7.4/static/jdbc-use.html#JDBC-CONNECT
Connect to postgres via existing superuser.
Create a Database by the name of user you are connecting through to postgres.
create database username;
Now try to connect via username
This worked for me when solving this problem
i ran sudo -i -u postgress --> to gain access to my postgres database.
Then enter your password.
it would allow you to now enter psql
which would prompt you for other command
Cheers!
Had this problem when installing postgresql via homebrew.
Had to create the default "postgres" super user with:
createuser --interactive postgres answer y to for super user
createuser --interactive user answer y to for super user
you can set the database name you want to connect to in env variable PGDATABASE=database_name. If you dont set this psql default database name is as username. after setting this you don't have to createdb
Was running postgres in docker. In cli I was getting the error "root" role doesn't exist.
su - postgres
psql
solved the problem.
PostgreSQL has its own user on the system which is created when PostgreSQL is installed. The postgres user is able to log into PostgreSQL without using a password. No other user is able to log into PostgreSQL.
This means that before using PostgreSQL, you will need to switch to that user account with the command:
su - postgres
You will then be able to log into the PosgreSQL client with the command:
psql
You will not be able to access the database from the command line as any other user.
I still had the issue above after installing postgresql using homebrew - I resolved it by putting /usr/local/bin in my path before /usr/bin
This is a basic misunderstanding. Simply typing:
pgres
will result in this response:
pgres <db_name>
It will succeed without error if the user has the permissions to access the db.
One can go into the details of the exported environment variables but that's unnecessary .. this is too basic to fail for any other reason.

Problem with Docker Compose - FATAL: database "root" does not exist (PostgresSQL) [duplicate]

I'm using the PostgreSql app for mac (http://postgresapp.com/). I've used it in the past on other machines but it's giving me some trouble when installing on my macbook. I've installed the application and I ran:
psql -h localhost
It returns:
psql: FATAL: database "<user>" does not exist
It seems I can't even run the console to create the database that it's attempting to find. The same thing happens when I just run:
psql
or if I launch psql from the application drop down menu:
Machine stats:
OSX 10.8.4
psql (PostgreSQL) 9.2.4
Any help is appreciated.
I've also attempted to install PostgreSql via homebrew and I'm getting the same issue. I've also read the applications documentation page that states:
When Postgres.app first starts up, it creates the $USER database,
which is the default database for psql when none is specified. The
default user is $USER, with no password.
So it would seem the application is not creating $USER however I've installed->uninstalled-reinstalled several times now so it must be something with my machine.
I found the answer but I'm not sure exactly how it works as the user who answered on this thread -> Getting Postgresql Running In Mac: Database "postgres" does not exist didn't follow up. I used the following command to get psql to open:
psql -d template1
I'll leave this one unanswered until someone can provide an explanation for why this works.
It appears that your package manager failed to create the database named $user for you. The reason that
psql -d template1
works for you is that template1 is a database created by postgres itself, and is present on all installations.
You are apparently able to log in to template1, so you must have some rights assigned to you by the database. Try this at a shell prompt:
createdb
and then see if you can log in again with
psql -h localhost
This will simply create a database for your login user, which I think is what you are looking for. If createdb fails, then you don't have enough rights to make your own database, and you will have to figure out how to fix the homebrew package.
From the terminal, just Run the command on your command prompt window. (Not inside psql).
createdb <user>
And then try to run postgres again.
By default, postgres tries to connect to a database with the same name as your user. To prevent this default behaviour, just specify user and database:
psql -U Username DatabaseName
Login as default user: sudo -i -u postgres
Create new User: createuser --interactive
When prompted for role name, enter linux username, and select Yes to superuser question.
Still logged in as postgres user, create a database: createdb <username_from_step_3>
Confirm error(s) are gone by entering: psql at the command prompt.
Output should show psql (x.x.x) Type "help" for help.
Login using default template1 database:
#psql -d template1
#template1=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+---------+----------+-------------+-------------+---------------------
postgres | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gogasca +
| | | | | gogasca=CTc/gogasca
template1 | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gogasca +
| | | | | gogasca=CTc/gogasca
(3 rows)
Create a database with your userId:
template1=# CREATE DATABASE gogasca WITH OWNER gogasca ENCODING 'UTF8';
CREATE DATABASE
Quit and then login again
template1=# \q
gonzo:~ gogasca$ psql -h localhost
psql (9.4.0)
Type "help" for help.
gogasca=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+---------+----------+-------------+-------------+---------------------
gogasca | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
postgres | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gogasca +
| | | | | gogasca=CTc/gogasca
template1 | gogasca | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gogasca +
| | | | | gogasca=CTc/gogasca
(4 rows)
I faced the same error when I trying to open postgresql on mac
psql: FATAL: database "user" does not exist
I found this simple command to solve it:
method1
$ createdb --owner=postgres --encoding=utf8 user
and type
psql
Method 2:
psql -d postgres
Post installation of postgres, in my case version is 12.2, I did run the below command createdb.
$ createdb `whoami`
$ psql
psql (12.2)
Type "help" for help.
macuser=#
Step 1:
psql -d template1
now you should be on psql terminal
Step 2:
CREATE DATABASE username;
make sure you use semicolon (;) after the database name;
optional: on psql terminal type \ls or \l to list all the databases;
Step 3:
psql -h localhost
now you should be connected;
Try using-
psql -d postgres
I was also facing the same issue when I ran psql
Had the same problem, a simple psql -d postgres did it (Type the command in the terminal)
This error can also occur if the environment variable PGDATABASE is set to the name of a database that does not exist.
On OSX, I saw the following error while trying to launch psql from the Postgress.app menu:
psql: FATAL: database "otherdb" does not exist
The solution to the error was to remove export PGDATABASE=otherdb from ~/.bash_profile:
Further, if PGUSER is set to something other than your username, the following error will occur:
psql: FATAL: role "note" does not exist
The solution is to remove export PGUSER=notme from ~/.bash_profile.
Not sure if it is already added in the answers, Anatolii Stepaniuk answer was very helpful which is the following.
psql -U Username postgres # when you have no databases yet
As the createdb documentation states:
The first database is always created by the initdb command when the data storage area is initialized... This database is called postgres.
So if certain OS/postgresql distributions do that differently, it is certainly not the default/standard (just verified that initdb on openSUSE 13.1 creates the DB "postgres", but not "<user>"). Long story short, psql -d postgres is expected to be used when using a user other than "postgres".
Obviously the accepted answer, running createdb to create a DB named like the user, works as well, but creates a superfluous DB.
Since this question is the first in search results, I'll put a different solution for a different problem here anyway, in order not to have a duplicate title.
The same error message can come up when running a query file in psql without specifying a database. Since there is no use statement in postgresql, we have to specify the database on the command line, for example:
psql -d db_name -f query_file.sql
First off, it's helpful to create a database named the same as your current use, to prevent the error when you just want to use the default database and create new tables without declaring the name of a db explicitly.
Replace "skynotify" with your username:
psql -d postgres -c "CREATE DATABASE skynotify ENCODING 'UTF-8';"
-d explicitly declares which database to use as the default for SQL statements that don't explicitly include a db name during this interactive session.
BASICS FOR GETTING A CLEAR PICTURE OF WHAT YOUR PostgresQL SERVER has in it.
You must connect to an existing database to use psql interactively. Fortunately, you can ask psql for a list of databases:
psql -l
.
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
----------------------------------+-----------+----------+-------------+-------------+-------------------
skynotify | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
myapp_dev | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
postgres | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
ruby-getting-started_development | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/skynotify +
| | | | | skynotify=CTc/skynotify
template1 | skynotify | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/skynotify +
| | | | | skynotify=CTc/skynotify
(6 rows)
This does NOT start the interactive console, it just outputs a text based table to the terminal.
As another answers says, postgres is always created, so you should use it as your failsafe database when you just want to get the console started to work on other databases. If it isn't there, then list the databases and then use any one of them.
In a similar fashion, select tables from a database:
psql -d postgres -c "\dt;"
My "postgres" database has no tables, but any database that does will output a text based table to the terminal (standard out).
And for completeness, we can select all rows from a table too:
psql -d ruby-getting-started_development -c "SELECT * FROM widgets;"
.
id | name | description | stock | created_at | updated_at
----+------+-------------+-------+------------+------------
(0 rows)
Even if there are zero rows returned, you'll get the field names.
If your tables have more than a dozen rows, or you're not sure, it'll be more useful to start with a count of rows to understand how much data is in your database:
psql -d ruby-getting-started_development -c "SELECT count(*) FROM widgets;"
.
count
-------
0
(1 row)
And don't that that "1 row" confuse you, it just represents how many rows are returned by the query, but the 1 row contains the count you want, which is 0 in this example.
NOTE: a db created without an owner defined will be owned by the current user.
had the problem with using the JDBC driver, so one just has to add the database (maybe redundantly depending on the tool you may use) after the host name in the URL, e.g.
jdbc:postgres://<host(:port)>/<db-name>
further details are documented here: http://www.postgresql.org/docs/7.4/static/jdbc-use.html#JDBC-CONNECT
Connect to postgres via existing superuser.
Create a Database by the name of user you are connecting through to postgres.
create database username;
Now try to connect via username
This worked for me when solving this problem
i ran sudo -i -u postgress --> to gain access to my postgres database.
Then enter your password.
it would allow you to now enter psql
which would prompt you for other command
Cheers!
Had this problem when installing postgresql via homebrew.
Had to create the default "postgres" super user with:
createuser --interactive postgres answer y to for super user
createuser --interactive user answer y to for super user
you can set the database name you want to connect to in env variable PGDATABASE=database_name. If you dont set this psql default database name is as username. after setting this you don't have to createdb
Was running postgres in docker. In cli I was getting the error "root" role doesn't exist.
su - postgres
psql
solved the problem.
PostgreSQL has its own user on the system which is created when PostgreSQL is installed. The postgres user is able to log into PostgreSQL without using a password. No other user is able to log into PostgreSQL.
This means that before using PostgreSQL, you will need to switch to that user account with the command:
su - postgres
You will then be able to log into the PosgreSQL client with the command:
psql
You will not be able to access the database from the command line as any other user.
I still had the issue above after installing postgresql using homebrew - I resolved it by putting /usr/local/bin in my path before /usr/bin
This is a basic misunderstanding. Simply typing:
pgres
will result in this response:
pgres <db_name>
It will succeed without error if the user has the permissions to access the db.
One can go into the details of the exported environment variables but that's unnecessary .. this is too basic to fail for any other reason.

Importing postgres database in a docker postgres container

I am trying to import an existing database into a postgres docker container.
This is how I proceed:
docker run --name pg-docker -e POSTGRES_PASSWORD=***** -d -p 5432:5432 -v BCS/postgres_data postgres
Then
docker exec -it pg-docker bash
psql -U postgres
postgres=# CREATE DATABASE myDB;
psql -U postgres myDB < BCS/mydb.sql
but when i execute the command \dt I have this error Did not find any relations.
knowing that my database has already tables.
So waht I am doing wrong?
First thing better to go with the approach that is mentioned by #LinPy.
Or better to copy at build time.
Dockerfile
FROM postgres
COPY mydb.sql /docker-entrypoint-initdb.d/
Another option, you do need to script for only DB creation.
FROM postgres
ENV POSTGRES_DB=mydb
The will create DB for you.
POSTGRES_DB
This optional environment variable can be used to define a different
name for the default database that is created when the image is first
started. If it is not specified, then the value of POSTGRES_USER will
be used.
In the above, the Postgres entrypoint will take care of the SQL script.
Second thing, the current issue with database name, Postgress will not treat them in uppercase simply, unless you did some trick.
Postgresql treats the db name as lowercase, normalising. However, the
field in the postgresapi does not replicate this behaviour, thus
allowing you to create a database with Capital letters. The fix could
be to warn the user that no uppercase letters are allowed in the db
aname and to add in a validation rule to the API to stop a user
creating such a database.
postgres-api
Change your command to
create DB
docker exec -it pg-docker bash
psql -U postgres
postgres=# CREATE DATABASE myDB;
verfiy DB
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
mydb | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
so the import command will be
psql -U postgres mydb < BCS/mydb.sql
or
psql -d mydb -U postgres -f ab.sql
You can also have your sql scripts being redirected to the container like that:
Postgres user:
docker exec -i my-postgres-container psql -U postgres < created-db.sql
Regular user:
docker exec -i my-postgres-container psql -d my-db -U my-user < create-schema.sql
If you are sure that the database installed everything correctly, are you are still not seeing tables, there are 2 things you should double-check:
When you connect, are you connecting to the right database? If you are using psql in a terminal, the database is specified with the -d switch.
psql -h <host> -U <user> -d <dbname>
You can also change your database after you connect using the \connect <dbname> command.
Are you specifying the right schema? \dt will show you tables, but you need to specify a schema first using set schema:
postgres=# \dt
...
<no tables>
...
postgres=# set schema 'my_schema';
postgres=# \dt
...
<my tables>
...

Docker compose postgresql service - can't create user and database during build?

I have wasted an entire day on this, and to say I'm not impressed by the unnecessary complexity of what should be a simple task - would be a gross understatement.
Ok, having got that off my chest, I am building a django application using docker-machine, docker-compose, postgresql and redis - by following this tutorial.
I have managed to get the basic tutorial to work - but it does not suit my needs, as I need to create a user and a database for my application - as opposed to using 'postgres' for both.
I have used the answer from #dnephin to a similar question, and modified my code as follows:
I created a new Dockerfile in a new directory ./database/:
FROM postgres:9.6
COPY . /fixtures
WORKDIR /fixtures
RUN /fixtures/setup.sh
./database/setup.sh contents:
#!/bin/bash
set -e
pg_createcluster 9.6 main --start
/etc/init.d/postgresql start
su - postgres # makes no effing difference ...
psql -f create_fixtures.sql
/etc/init.d/postgresql stop
./database/create_fixtures.sql contents:
CREATE DATABASE mydatabase WITH ENCODING 'UTF8';
CREATE USER webuser ENCRYPTED PASSWORD 'deadbeefsnaf0' NOSUPERUSER NOCREATEDB NOCREATEROLE;
GRANT ALL PRIVILEGES ON mydatabase TO webuser;
and finally my postgres service in the docker_compose.yml is modified to use build:
postgres:
build: ./database/
...
When I run docker-compose build, the build goes through the motions and then barfs at where I'm importing the SQL fixtures file via psql:
frothing#themouth:~/path/to/directory$ docker-compose build
redis uses an image, skipping
Building postgres
Step 1/4 : FROM postgres:9.6
---> ff0943ecbb3c
Step 2/4 : COPY . /fixtures
---> fae19dc88da8
Removing intermediate container 84b860aee55c
Step 3/4 : WORKDIR /fixtures
---> aa88438dc69f
Removing intermediate container b801ddc3b374
Step 4/4 : RUN /fixtures/setup.sh
---> Running in ca3e89ec2460
Creating new cluster 9.6/main ...
config /etc/postgresql/9.6/main
data /var/lib/postgresql/9.6/main
locale en_US.utf8
socket /var/run/postgresql
port 5432
Starting PostgreSQL 9.6 database server: main.
psql: FATAL: role "root" does not exist
ERROR: Service 'postgres' failed to build: The command '/bin/sh -c /fixtures/setup.sh' returned a non-zero code: 2
I tried to solve this using the useless documentation on docker for postgresql service - but got no where.
How can I solve this?
Volumes are not available at build time. You can create /var/lib/postgresql/data in your script but it will be overwritten by VOLUME /var/lib/postgresql/data from postgres:9.6 image.
In your case: just use the following docker file:
FROM postgres:9.6
COPY ./create_fixtures.sql /docker-entrypoint-initdb.d/create_fixtures.sql
They get automatically executed once the container starts. Here is an example:
$ docker run -d --name mydb -p 33306:3306 yourtag
$ docker exec -ti mydb psql -U postgres
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
------------+----------+----------+------------+------------+-----------------------
mydatabase | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | webuser=CTc/postgres
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
Outdated answer:
Your script should work on a container except in the fixture you have to execute psql like this:
su postgres -c "psql -f create_fixtures.sql"
su --login postgres does not work because postgres can't open a bash or shell. You can try around with docker run --rm -ti postgres:9.6 bash.
Sorry I have to tell you there is one more error in your sql script: GRANT ALL PRIVILEGES ON DATABASE mydatabase TO webuser - the keyword DATABASE is necessary here.
Here is a complete log how I tested and can confirm this works:
docker run --rm -ti postgres:9.6 bash
root#be03ab1eb704:/# cat > test.sql <<EOF
> CREATE DATABASE mydatabase WITH ENCODING 'UTF8';
> CREATE USER webuser ENCRYPTED PASSWORD 'asdf123' NOSUPERUSER NOCREATEDB NOCREATEROLE;
> GRANT ALL PRIVILEGES ON DATABASE mydatabase TO webuser;
> EOF
root#be03ab1eb704:/# pg_createcluster 9.6 main --start
Creating new PostgreSQL cluster 9.6/main ...
/usr/lib/postgresql/9.6/bin/initdb -D /var/lib/postgresql/9.6/main --auth-local peer --auth-host md5
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/9.6/main ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
/usr/lib/postgresql/9.6/bin/pg_ctl -D /var/lib/postgresql/9.6/main -l logfile start
Ver Cluster Port Status Owner Data directory Log file
9.6 main 5432 online postgres /var/lib/postgresql/9.6/main /var/log/postgresql/postgresql-9.6-main.log
root#be03ab1eb704:/# /etc/init.d/postgresql start
[ ok ] Starting PostgreSQL 9.6 database server: main.
root#be03ab1eb704:/# su postgres -c "psql -f test.sql"
CREATE DATABASE
CREATE ROLE
GRANT
root#be03ab1eb704:/# /etc/init.d/postgresql stop
[ ok ] Stopping PostgreSQL 9.6 database server: main.
root#be03ab1eb704:/# exit
exit
The official postgresql docker image automatically imports scripts at the first start of a container. So if you mount your directory with your init sql script to container path '/docker-entrypoint-initdb.d/' it should be run.
For example if you have your import script myImport.sql and it's on your host in a directory /opt/import, you can mount the import directory on container start to your postgres image to /docker-entrypoint-initdb.d and the sql file will be executed after initial database setup.
docker run -p $toHostParam:5432 -e POSTGRES_PASSWORD="$dbPwd" \
-e POSTGRES_USER="$dbUser" \
-e POSTGRES_DB="$dbName" \
-e PGDATA=/opt/pgdata \
-v ${dataDir}:/opt/pgdata \
# look here
# mount of local import dir
-v /opt/import:/docker-entrypoint-initdb.d \
postgres:9.6
Take a look at the postgesql image start script here (from line 126):
https://github.com/docker-library/postgres/blob/master/9.6/docker-entrypoint.sh
If you want a specific db user or database you can also customize your postgresql container with environment variables.
Take a look at the 'Environment Variables' section here: https://hub.docker.com/_/postgres/
Try with this setup.sh
#!/bin/bash
set -e
pg_createcluster 9.6 main --start
su postgres sh -c "/etc/init.d/postgresql start && psql -f create_fixtures.sql && /etc/init.d/postgresql stop"
Try a explicit user when running psql:
psql -U postgres -f create_fixtures.sql

PostGIS only works with root user

I have a database and am trying to use PostGis with it.
When I run:
$ psql -h localhost -d $APP_DB_NAME -U $APP_DB_USER
# SELECT PostGIS_version();
I get the following error:
ERROR: function postgis_version() does not exist
LINE 1: SELECT PostGIS_version();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
But when I enter the database as root:
$ sudo su postgres -c psql $APP_DB_NAME
# SELECT PostGIS_version();
It runs fine:
postgis_version
---------------------------------------
2.1 USE_GEOS=1 USE_PROJ=1 USE_STATS=1
(1 row)
I am installing the PostGis extension as the root user, since my user doesn't have super_user access.
$ su - postgres -c psql $APP_DB_NAME
# CREATE EXTENSION postgis;
My guess is that the owner of the schema in which postgis is installed is not the user you're logging in as, but rather the "root" user (which is postgres).
$ psql postgres
postgres=# \dn
List of schemas
Name | Owner
--------------------+----------
public | postgres
pg_catalog | postgres
my_data | someuser
When connecting as postgres, postgis will work in this case, but if you connect as any other user, it will fail, and say it's not there, even though if you try to create extension postgis it will say it's installed already.
To correct this, make sure all the schemas are owned by the user you're actually connecting as:
ALTER SCHEMA public OWNER TO my_user;