I'm not sure that I actually have a problem, but I am confused. I wanted to move a PostgreSQL database from PythonAnywhere to an AWS RDS server. I connected to both servers with pgAdmin from my Windows PC (ssh tunnel with PuTTY). I then did a backup of the database from PythonAnywhere and then did a restore to a clean database on the RDS server.
The backup had no issues, but, while the restore seemed to run fine, pgAdmin showed the process "Failed". The database on the RDS server looks fine. I checked row counts on a few tables, and they matched what I had on PythonAnywhere. I don't see any messages in pgAdmin other than that the process failed. I don't see anything in the pgAdmin logs to indicate what might be wrong. Do I have a problem? Should I use a command line restore instead?
Thanks for any insights.
--Al
Using cli is a better way, the ide may not give full errors. pg_dump will work for you. Maybe you should build a EC2 for this job.
Related
This question already has answers here:
Export and import table dump (.sql) using pgAdmin
(6 answers)
Closed 1 year ago.
Let I first state that I am not a DBA-guy but I do have a question regarding restoring remote databases using PG Admin.
I have this PG Admin tool (v4.27) running in a Docker container and I use this portal to maintain two separate Postgress databases, both running in a Docker container as well. I installed PG Agent in both database containers and run scheduled daily backup's, defined via PG Admin and stored in the container of each corresponding databases. So far so good.
Now I want to restore one of these databases by using the latest daily backup file (*.sql), but the Restore Dialog of PG Admin only looks for files stored locally (the PG Admin container)?
Whatever I tried or searched for on the internet, to me it seems not possible to show a list of remote backup files in PG Admin or run manually a remote SQL file. Is this even possible in PG Admin? Running psql in the query editor is not possible (duh ...) and due to not finding the remote SQL-restore file I have no clue how to run this code within PG Admin on the remote corresponding database container.
The one and only solution so far I can think of, is scheduling a restore which has no calendar and should be triggered manually when needed, but it's not the prettiest solution.
Do I miss something or did I overlook the right documentation or have I created a silly, unmaintainable solution?
Thanks in advance for thinking along and kind regards,
Aad Dijksman
You cannot restore a plain format dump (an SQL script) with pgAdmin. You will have to use psql, the command line client.
COPY statements and data are mixed in such a dump, and that would make pgAdmin choke.
The solution by #Laurenz Albe points out that it is best to use the command line psql here, and that would be my first go-to.
However, if for whatever reason you don't have access to the command line and are only able to connect to this database via pgadmin, there is another solution which you can find here:
Export and import table dump (.sql) using pgAdmin
I recommend looking at the solution by Tomas Greif.
I'm giving a try on PoWA, but I've got a little problem.
My Postgres database is running on AWS RDS.
PoWA needs HypoPG in order to suggest indexes.
But RDS doesn't support HypoPG extension. So I had to install PoWA at my backup database (outside RDS).
The problem is: PoWA isn't analyzing the restored database. It can't recognizes any data. If I execute some SQL queries manually it works though.
Is there something that I can be missing?
And, when I tried Ankane Dexter, I could show it the log files path (dumped at the backup database in parallel). Is there a way to do so in PoWA?
Thanks.
I'm currently in the process of switching my cloud server from Heroku to Digital Ocean. However is there a way to migrate the database from the heroku server to the digital ocean one? I use postgresql for my database
I hope you already got a solution, but in case you didn’t, I’ll provide a simple guide on how I did it. I am going to assume that you have already created a postgres database on digitalocean. Also you need navigate to your project directory and log in to heroku using the heroku cli. And, you need to have postgresql installed or a psql client. Installing postgresql would do it as it comes with psql.
Step 1: Create a backup and download the backup from heroku postgres
heroku pg:backups:capture --app <app_name>
heroku pg:backups:download --app <app_name>
The first command will create a backup of your database and the second command will download it to your current directory, its a .dump file. If you would like to read more, here is an article.
Step 2: Connect to your remote (digital ocean’s) database using psql
Before you can do this, you need to go and add your machine you are connecting from to the list of database’s list of trusted sources, If you don’t, you’ll get a Connection Timed Out error. That’s because the database’s firewall doesn’t allow you to connect to the database from your local machine or resource (for security reasons).
Step 3: Import the Database
pg_restore -d "postgresql://<database_username>:<database_password>#<host>:<port>/<database>?sslmode=require" --jobs 4 -c "/path/to/dump_file.dump"
This will import your database from your dump file. Just substitute the variables will your connection parameters that you get from your dashboard. If you would like to read more, here is another article for this step.
Another thing to make clear is, sometimes, you will see some harmless error messages when running this command, but it will push through anyway. To learn more about pg_restore read this article.
And that’s it, your database has been migrated. Now, can you confirm it worked?, well, as for me, I used pgAdmin to connect to the remote database and I saw the tables and data as expected.
Hope this helps anyone with the same problem :)
Let's say you have added a postgres cluster on pgadmin4.
Does using delete/drop actually drop the cluster? or it just drops from pgadmin4? and nothing happens to the actual cluster.
I am a superuser, so definitely don't want to try it.
I always have to be very careful while disconnecting the server.
Yes, the Delete/Drop option will remove the server from the PgAdmin4 Server tree, but does not delete/drop any data or databases
I have Pgagent installed on my Debian OS. Along with Postgresql 9.4.
I have checked .pgpass file as this seems to be the most common cause for a job to not run.
host port 5432 database = * username = postgres password = xxxx.
for both local and the remote host. The database I'm trying to set a job for is on a remote host.
I made sure it was enabled. It's just a simple INSERT script that should repeat every 5 minutes.
No errors are being triggered that I can find. Any ideas of what would cause the job not to run at all - even when selecting 'run now'?
Check postgre db, pgAgent Catalog, pga_jobsteplog
IDK about Linux but I had similar problem in windows where the thing won't run and it doesn't raise any notice on the error even after doing RUN NOW. The only error i could find out was that if i click on the job and click on statistics, i could see like shit ton of times it ran and everytime it ran, its status was F.
The reason for this failure is becuase the pgagent couldn't connect to the main database of postgresql.
The services of pgagent isn't running at all (as we can see this information under services in task manager in windows).
Forcing the service to run would create a failure which can be viewed in the event manager in windows.
To solve this issue, first try putting that pgpass.txt file in the environment variable (if not automatically put), if this didn't work, then what I did was to uninstall and delete all possible folders of Postgres, pgagent, and pgadmin, clearing out all temp files, clearing out registry details which have been put by Postgres, pgagent, and pgadmin and also from environment variable. Then reinstall it and it would normally work :)