Just wondering if its possible to set a default tablespace in postgres to keep indexes. Would like the databases to live on the default tablespace for postgres, however, would like to get the indexes on a different set of disks just to keep the i/o traffic separated.
It does not appear to me that it can be done without going in and doing an ALTER index TABLESPACE command, and then the index is moved and will stay there, but the databases and indexes are part of a django app, so non-django intervention can cause some problems.
It doesn't look like there's a way to specify a default index tablespace (that is other than the database's own default tablespace). I've always specified them manually. It's fairly easy to write a query to check for indices that are in the wrong tablespace, so you could at least install that as part of your database monitoring.
IMHO you shouldn't have to be worried about making changes like that to the database outside of the app since it's a database implementation detail the app doesn't grok. If the app has a problem with it, the app is broken.
Currently django supports default tablespace for database and for indexes:
DEFAULT_INDEX_TABLESPACE
Default: '' (Empty string)
Default tablespace to use for indexes on fields that don't specify one, if the backend supports it.
DEFAULT_TABLESPACE
Default: '' (Empty string)
Default tablespace to use for models that don't specify one, if the backend supports it.
Related
I'm managing a PostgreSQL database server for some users who need to create temporary tables. One user accidentally sent a query with ridiculously many outer joins, and that completely filled the disk up.
PostgreSQL has a temp_file_limit parameter but it seems to me that it is not relevant:
It should be noted that disk space used for explicit temporary tables, as opposed to temporary files used behind-the-scenes in query execution, does not count against this limit.
Is there a way then to put a limit on the size on disk of "explicit" temporary tables? Or limit the row count? What's the best approach to prevent this?
The only way to limit a table's size in PostgreSQL is to put it in a tablespace on a file system of an appropriate size.
Since temporary tables are created in the default tablespace of the database you are connected to, you have to place your database in that size restricted tablespace. To keep your regular tables from being limited in the same way, you'd have to explicitly create them in a different, less limited tablespace. Make sure that your user has no permissions on that less limited tablespace.
This is a rather unappealing solution, so maybe you should rethink your requirement. After all, the user could just as well fill up the disk by inserting the data into a permanent table.
Currently I am setting up Master/Master Replication with bucardo between 5 Nodes on different locations (should provide location transparency). The database holds ~500 Tables which should be replicated. I grouped them into smaller replication herds of 50 Tables at maximum based on their dependency on each other. All tables have primary keys defined and the sequencers on each node are set up to provide system wide unique identities (based on residue class)
To get an initial database on each node, I made a --data-only custom format pg_dump into a File and restored this on each node via pg_restore. Bucardo sync is setup with the bucardo_latest strategy to resolve conflicts. Now when I start syncing bucardo is deleting all datasets in the origin database first and inserting it again from one of the restored nodes, because all restored datasets have a "later timestamp" (the point in time when I called pg_restore). This ultimately prohibits the inital startup as bucardo needs very much time and also fails, as there are lots of datasets to solve and timeouts often too short.
I also have 'last_modified' timestamps on each table which are managed by UPDATE triggers, but as I understand it, pg_dump inserts data via COPY, and therefore these triggers don't get fired.
Which timestamp does bucardo use to find out who is bucardo_latest?
Do I have to call pg_dump with something like set SESSION_REPLICATION_ROLE = 'replica';?
I just want bucardo to keep track of every new change, not executing pseudo changes because of the restore.
EDIT: pg_restore has definitely fired several triggers at restore time...as said I keep track on user and last modification date in each table, and those values are set to the user and timestamp when the restore was done. I am aware, that I can set SESSION_REPLICATION_ROLE for a plain text format restore via psql. Is this also possible for pg_restore somehow?
The common approach is make the dump/restore process before configure the replication.
So an option will be:
drop the bucardo schema in each database
do a bucardo remove for each object (most of them allow use all, like bucardo remove table all
dump/restore your data
Configure again the replication. Just make sure that when adding the sync, set the option onetimecopy=0. It's the default but I feel safer making it explicit.
Which timestamp does bucardo use to find out who is bucardo_latest?
bucardo handles its own timestamp value. Each table should have a trigger named like bucardo.delta_myschema_mytable that makes and insert in a table named like bucardo.delta_myschema_mytable. This table has a column txntime timestamp with time zone not null default now() and this is the timestamp used.
Do I have to call pg_dump with something like set SESSION_REPLICATION_ROLE = 'replica';?
AFAIK, if bucardo triggers are already set in the tables, the option --disable-triggers of pg_restore should do the trick.
You can also check these articles about working with large databases and the use of session_replication_role
So I created a tablespace and added two datafiles to it but when i dropped the table space using drop tablespace tbs1 including contents it didn't drop the datafiles and i was wondering why or if i did anything wrong. If you do know a better way to do it feel free to provide it.
Also why does a user default tablespace stay after deleting that specific tablespace.
It didn't remove the datafiles because you didn't ask it to, try
DROP TABLESPACE tbs1 INCLUDING CONTENTS AND DATAFILES;
The only real answer to your secondary question is because it does. Oracle decided to make it work that way and I think this might be because it's difficult to decide which tablespace is to be used instead. i.e. it is probably safer to leave the user with a default of a non-existent tablespace until you define a new default than it is to randomly pick one; after all dropping a tablespace and its contents is not normally a trivial event but something with ramifications that should be carefully considered in advance.
I have seen this answer, How to apply PostgreSQL UNLOGGED feature to an existing table?, which basically suggests that the way to convert a table to unlogged is to run:
CREATE UNLOGGED TABLE your_table_unlogged AS SELECT * FROM your_table;
Is this still the case, because while this is an of obvious working solution, for a large table there are potential time and disk space factors that could come into play. And, if yes, could someone please explain briefly how the architecture of Postgres means you need to rewrite an entire table in order to make it unlogged?
Update: In PostgreSQL 9.5+ there is ALTER TABLE ... SET LOGGED and ... SET UNLOGGED
Converting from UNLOGGED to LOGGED requires that the whole table's data be written to xlogs if wal_level is > minimal so replicas get a copy. So it's not free, but it can still be worth creating a table unlogged, populating it then setting it logged if you have a bunch of cleanup and deletion and merging work to do on the table after initial load.
Yes, that's still the case in 9.4.
Converting from logged to UNLOGGED isn't theoretically hard AFAIK, but nobody's done the work to do it. The main thing is that all constraints and types etc referring to it must be re-checked to make sure there's no reference from another logged table to this table. Most attention has been paid to the other case, so if this feature is important to you, consider funding its development or getting involved in development yourself.
Converting UNLOGGED to logged may become possible for nodes that aren't involved in streaming replication or using an archive_command. It's not simple otherwise because of the need to cope with the fact that the data for the table wasn't sent, but suddenly changes to it are - the replication protocol would need further enhancement to allow the table to be base-copied before continuing.
Apparently this (alter table ... set logged | unlogged) has been implemented in (the upcoming) postgresql 9.5.
I have a collection of CRM 2011 databases, all of which are installed on the same SQL server running SQL 2008 R2. One of those databases has a collation of Modern_Spanish_Cl_Al, while all of the others are set to Latin1_General_Cl_Al. I want all of them to be set to Latin1, so I would like to go back and fix the Spanish database to be like the others.
How do I change the database collation from Modern_Spanish_Cl_Al to Latin1_General_Cl_Al?
I tried doing it at the database level using the UI, but I received a million errors like:
The statistics 'TeamBase.fndx_Sync_VersionNumber' is dependent on database collation. The database collation cannot be changed if a schema-bound object depends on it. Remove the dependencies on the database collation and then retry the operation.
I tried going to the column level while also using the UI, but received a different error message:
Saving changes is not permitted. The changes you have made require the following tables to be dropped and re-created...
I am dealing with this issue in a test environment, so I am willing to experiment, and I realize that what I am trying to do might not be possible without breaking CRM.
Did you try creating a new DB with the correct collation
CREATE DATABASE DatabaseName COLLATE Latin1_General_Cl_Al;
And then Backup-Restore your DB with incorrect collation into the new one?
Should this work, you'll have to import a new organization based on your new database, using the Import Organization Wizard in the Deployment Manager.
edit
After looking into this, it seems quite hard to change collation, even when doing backup/restore.
Your only option if you want to change the collation on the database level seems to be an
ALTER DATABASE ... COLLATE
but that will only affect new objects. You would then have to use ALTER TABLE to change your existing tables.
In addition, because the collation specifies how data is sorted and stored, you need to export all your data and bring it in again (using BCP and BULK INSERT for instance).
To add to this, certain properties of a column prevents its collation from being changed (such as certain constraints and schema bound function references). If you do want to change the collation of the entire database, the easiest way is probably to script the database and create a new with the right collation and then shuffle the data into the new database.