I know this is a very basic question, but i could not find a answer around google.
But what is the main difference between those two tablespaces?
From Documentation:
Two tablespaces are automatically created by initdb. The pg_global tablespace is used for shared system catalogs. The pg_default tablespace is the default tablespace of the template1 and template0 databases (and, therefore, will be the default tablespace for other databases as well, unless overridden by a TABLESPACE clause in CREATE DATABASE).
Related
I have a situation in which summing the size of the tables in a tablespace (using pg_class among others) reveals that there is 550G of datafiles in a particular tablespace in a particular database.
However, there is 670G of files in that directory on the server.
FWIW, I don't know how that can be. No files have been written to that directory via any mechanism other than Postgres. My best guess is perhaps the database crashed while an autovacuum was going on, leaving orphan files laying around...does that sound plausible?)
SO, I've worked out a way, by reading the contents of a ls command into the database, strip off the numeric extensions for tables > 1G in size, and compare them with the contents of pg_class, and have, in fact, found about 120G of files not reflected in pg_class.
My question is, is it safe for me to delete these files, or could they be in active use by the database but not reflected in pg_class?
Do not manually delete files in the PostgreSQL data directory.
This is not safe and will corrupt your database.
The safe way to purge any files that don't belong to the database is to perform a pg_dumpall, stop the server, remove the data directory and the contents of all tablespace directories, breate a new cluster with inindb and restore the dump.
If you want to investigate the issue, you could try to create a new tablespace and move everything from the old to the new tablespace. I will describe that in the rest of my answer.
Move all the tables and indexes in all databases to the new tablespace:
ALTER TABLE ALL IN TABLESPACE oldtblsp SET TABLESPACE newtblsp;
ALTER INDEX ALL IN TABLESPACE oldtblsp SET TABLESPACE newtblsp;
If oldtblsp is the default tablespace of a database:
ALTER DATABASE mydb SET TABLESPACE newtblsp;
Then run a checkpoint:
CHECKPOINT;
Make sure you forgot no database:
SELECT datname
FROM pg_database d
JOIN pg_tablespace s
ON d.dattablespace = s.oid
WHERE s.spcname = 'oldtblsp';
Make sure that there are no objects in the old tablespace by running this query in all databases:
SELECT t.relname, t.relnamespace::regnamespace, t.relkind
FROM pg_class t
JOIN pg_tablespace s
ON t.reltablespace = s.oid
WHERE s.spcname = 'oldtblsp';
This should return no results.
Now the old tablespace should be empty and you can
DROP TABLESPACE oldtblsp;
If you really get an error
ERROR: tablespace "tblsp" is not empty
there might be some files left behind.
Delete them at your own risk...
I am new on Amazon RDS.
I'm trying to create the tablespaces of my application, but I can't find where I should store it.
Is it possible to create tablespaces on PostgreSQL RDS?
I didn't find RDS documentation about this, but apparently Postgres tablespaces work fine on RDS. You don't have direct access to the underlying EBS volume that RDS is using, but if you try a CREATE TABLESPACE command with some arbitrary directory:
CREATE TABLESPACE testspace LOCATION '/foo/bar/baz';
You can see that the tablespace gets created like so:
mydb=> \db
List of tablespaces
Name | Owner | Location
------------+------------+-------------------------------------------
pg_default | rdsadmin |
pg_global | rdsadmin |
testspace | db_user | /rdsdbdata/db/base/tablespace/foo/bar/baz
(3 rows)
And you should be able to create tables in that tablespace just fine:
mydb=> CREATE TABLE test (a int) TABLESPACE testspace ;
CREATE TABLE
At the time of writing in 2019 RDS does support PostgreSQL tablespaces, although not for IO splitting or isolation (as all tablespaces will be created on the same volume). AWS documentation for this feature can be found here.
Things like log-shipping/standby servers heavily depend on tablespace location. I wouldn't risk it and blindly use a location I don't have access to and not fully aware of(as Josh suggested)
I guess we just all have to admit it that much as we like AWS services, there are things where they lag behind, especially with regards to SaaS solutions to external\independent products(RDS, Docker orchestration on ECS, etc.)
UPDATE:
If this post is correct, RDS replication for Postgres does depend on streaming replication, which makes any manipulations with tablespace paths very dangerous unless you can find it in their official docs or via enterprise support that such strategy will always work as you expect.
Is there any way of storing database other than the fixed data directory in postgres? I have a situation where I need to store database at any location irrespective of data directory.
You can add a tablespace.
A tablespace is basically a location to store databases and/or tables. You create a tablespace using CREATE TABLESPACE:
CREATE TABLESPACE mytablespace LOCATION '/path/to/some/location';
You can then create tables directly in that tablespace:
CREATE TABLE whatever (thing integer) TABLESPACE mytablespace;
Or set the default tablespace:
SET default_tablespace = mytablespace;
You can also set the default tablespace at database creation time:
CREATE DATABASE mydatabase TABLESPACE mytablespace;
You are looking for the 'Data Directory' in linux for example its in
/usr/local/pgsql/data
if you install from source code. Each distribution is different though, try reading up on
File Locations in postgres and Creating a Database Cluster
also try using the query
show data_directory;
in windows try looking around
C:\Program Files (x86)\PostgreSQL\[VERSION]\data\global
I have created a tablespace named hdd_tablespace and I wan't all new databases to be automatically created there. Basically when I execute:
CREATE DATABASE FOO;
I want this database to be created in hdd_tablespace.
I have updated postgresql.conf with:
default_tablespace = 'hdd_tablespace'
However, new databases are still created in pg_default.
Actually it turned out I had also to update template1 database that is a template database.
I had to;
ALTER DATABASE template1 SET TABLESPACE hdd_tablespace;
Coming from MySQL and not knowing about ROLEs I absentmindedly tried this
GRANT CREATE ON TABLESPACE pg_default TO username;
It didn't have the desired effect. The command that I was looking for was:
ALTER ROLE username WITH CREATEDB;
But what's the difference precisely? Does giving someone the CREATEDB role implicitly give them CREATE ON TABLESPACE ...? Is there a table where I can see all this?
From the docs, GRANT CREATE ON TABLESPACE means (my emphasis):
For tablespaces, allows tables,
indexes, and temporary files to be
created within the tablespace, and
allows databases to be
created that have the
tablespace as their default
tablespace. (Note that revoking this
privilege will not alter the placement
of existing objects.)
They are completely different privileges. CREATEDB means the role can create database. If your role doesn't have that, it can't create databases, period.
Granting CREATE for a tablespace to a role means that the role will be able to use that tablespace as default tablespace for the database. So a role that has CREATEDB will be able to create a database; just not on that tablespace. Note that there's always a pg_default tablespace that will be used as default tablespace for databases that don't otherwise have a default tablespace.
Tablespaces are not the same as databases.
It looks like the first statement gives you the ability to create tablespaces. This are physical files (or spaces on your storage device) that hold the data/indexes/... of the database.
So effectively you granted two different rights with the commands.