I recently dual booted my system by installing Ubuntu over Windows. Now I have to import a file in postgresql , which is stored in host file system. The host filesystem has 190 GB of space. But when I log into postgres as sudo su postgres, it would take me into root filesystem(the default postgres folder) and query would be executed in that. Now my data set is of 3 GB and after sometime query would return 'OUT of disk space' as the root filesystem is of 3.5-4 GB. So it would be great if anyone can suggest solution to this? . Do I need to change default folder of postgres?
Thanks
Ravinder
I'd create a new file on host, which I'd configure as a second hard drive image for your Ubuntu. And I'd use this drive to create partition there and mount it where the PGDATA directory would be.
Related
I have an important issue with my database and I don't know how to fix it.
I have postgresql 9.6 in CentOS running. After a system reboot, postgrsql-service doesn't start, so following the instructions in the shell, I launched "sudo /usr/pgsql-9.6/bin/postgresql96-setup initdb"
What my surpirse when I started pgsql... and it is a new empty instance, I had a database with a size of 4GB and has dissapear...
However data must be still in the files, because data folder have a size of 4GB, how I can recover this last situation?
Thank you very much,
Regards,
I'm trying to create the first copy of my database. I'm using PostgreSQL and Ubuntu 16+ with Django technology.
I found this documentation to create a copy:
I'm trying to export the entire database to a file so that I can add it to another server. I tried this:
pg_dump app_prod > test_copy
pg_dump --host=localhost --username=app --dbname=app_prod --file=testdb.sql
after selecting ls my directory can see the database. But by running eg WinSCP it is not visible.
How can I take these files, copy them to my Windows system and upload to another Ubuntu server?
I think that it is enough to make them visible in WinSCP. How can I do this?
EDIT:
drwxr-xr-x 3 postgres postgres 4096 Oct 4 08:06 9.5
-rw-rw-r-- 1 postgres postgres 3578964 Jan 18 10:46 test_copy
-rw-rw-r-- 1 postgres postgres 0 Jan 18 10:54 testdb.sql
It seems like this was resolved in the comments: you were looking at the wrong folder in the WinSCP folder explorer.
There are a few items worth noting to bolster the good advice already given:
Your ls -l output indicates that the SQL file is zero bytes in size, so something has gone wrong there. If you manage to transfer it to your local machine, you will find it is empty.
Also, try not to store database dumps in /var/lib/postgresql - this is where your PostgreSQL database keeps live database files on the server, and you don't want to risk changing or deleting anything here. Use /home/maddie instead (change the username as appropriate).
We have a PostgreSQL instance running in a VM in the Google Cloud. The nature of the queries that we run involves lots of PostgreSQL temporary table space. (5 or 6 or more TB of disk I/O every day)
This I/O continues to be a major bottleneck in our database. Currently I have it all happening on an SSD persistent disk - not because we need to save any of the data in the event of a reboot, but because PostgreSQL lays out a file structure on the disk that it then uses for the temporary tables and if the file structure is missing when the database starts up, it isn't very good.
What I'd like to do is configure the temporary tablespace on the local SSD's because of their much higher I/O throughput. Unfortunately, they get wiped out on every reboot. I'd like a simple way to be able to re-layout the disk after reboot and before PostgreSQL starts back up.
I could tar up the empty file structure and then write a script that untars it after every boot. Does that make sense? Is there a better way/best practice for doing this?
What would be awesome is if there was a PostgreSQL extension out there that did this magically.
Ideas?
I dug a bit into my previous tests and here is some summary:
PostgreSQL tablespace is just a directory - no big deal. Plus - if you will use it only as temporary table space there will be no persistent file left when you shutdown database.
You can create tablespace for temp tables on any location you want and then go to this location and check directory structure to see what PG created. But you must do under OS because PG will show you only tablespace main directory - both \db+ in psql or select oid, spcname, pg_tablespace_location(oid) from pg_tablespace; work the same way.
My example:
(I used /tempspace/pgtemp as presumed mounting point) CREATE TABLESPACE p_temp OWNER xxxxxx LOCATION '/tempspace/pgtemp'; created in my case structure /tempspace/pgtemp/PG_10_201707211
I set temp_tablespaces = 'pg_temp' in postgresql.conf and reloaded configuration.
When I used create temp table .... PG added another subdirectory - /tempspace/pgtemp/PG_10_201707211/16393 = oid of schema - but this does not matter for temp tablespace because if this subdirectory will be missing PG will create it.
PG created in this subdir files for temp table.
When I closed this session files for temp table were gone.
Now I stopped PG and tested what would happened if directories will be missing:
I deleted PG_10_201707211 with its subdir
started PG and log showed message LOG: could not open tablespace directory "pg_tblspc/166827/PG_10_201707211": No such file or directory but PG started
I tried to create temp table - I got error message ERROR: could not create directory "pg_tblspc/166827/PG_10_201707211/16393": No such file or directory SQL state: 58P01
Now (with running PG) I issued these commands in OS:
sudo mkdir -p /tempspace/pgtemp/PG_10_201707211
sudo chown postgres:postgres -R /tempspace/pgtemp
sudo chmod 700 -R /tempspace/pgtemp
I tried to create temp table again and insert and select values and everything worked OK
So conclusion is - since PG tablespace is no "big magic" just directories you can simply create bash script running on linux startup which will check (and mount if necessary) local SSD and create necessary directories for PG temp tablespace.
I have USB mounted SATA drive with some Postgresql databases on it (hard drive contains installed ArchLinux) and would like to move them to current system. Unfortunately I cannot boot from that drive and dump the SQL files. Can someone tell is it hard to move physical files and directories or some other method how I could migrate those databases.
Maybe there is a way to create a symbolic link for current postgresql to use that hard drive as well?
I receive the same error when I try to create a database with
CREATE DATABASE dwh;
and
createdb dwh;
namely:
createdb: database creation failed: ERROR: could not create directory "base/16385": No space left on device
and
ERROR: could not create directory "base/16386": No space left on device
I am using a postgres AMI on aws (PostgreSQL/Ubuntu provided by OpenLogic)https://aws.amazon.com/marketplace/ordering/ref=dtl_psb_continue?ie=UTF8&productId=13692aed-193f-4384-91ce-c9260eeca63d®ion=eu-west-1
provisioned with m2.xlarge machine, which should have 17GB RAM and 350GB SSD
Based on the description provided, you have not mapped your Postgres /data directory to your actual 350GB partition.
If you are running production server, 1st of all - try to clean up the logs (/pg_log folder) to save disk space and bring up the box to normal operation AND create backup of your database.
Run df -h to see disk devices utilization and lsblk what is mounted to your disk. It highly likely, that AWS by default gave you not extended 350GB volume. You have 2 options:
Add new disk take a look at Ubuntu add new drive procedure and map it to your Postgres /data folder
Try to do perform resize of the existing file system with resize2fs, relevant answer can be found at AskUbuntu