I need to copy a schema in Postgres to another database on a remote server, but I keep up ending to get a fail like:
pg_dump: too many command-line arguments (first is "--n")
My code:
pg_dump postgres -n my_local_shema | psql -h 11.22.33.44 -U my_user_on_remote_server-d postgres
I have tried for hours and with different commands but I keep getting the "too many command lines".
Try with a reversed order like this:
pg_dump -n my_local_shema postgres
Ok. This command structure works as a charm:
pg_dump -n my_local_shema_name -d my_local_database -U my_local_username | psql -h 111.222.333.444 -U my_user_name_on_remote_Server my_Database_name_on_remote_server
Step-by-step-guide
I copied a shema with all tables and indexes to another database on another server.
The 111.222.333.444 is the IP of the remote server.
In preparation(I dont know if it is actually needed), I first created a shema on the remote server with an identical shemaname as the one, I wanted to copy. I also checked, that the firewall was open for datatransfer from the old server to the new one.
Then, i opened a commandpromt (I use windows) and opened the folder where the pgdump.exefile was. Here typed the command.
Last it asked me to type in a password. First it promted it. THen it was silent - nothing happened, and I did not know, what to expect. Last I typed in the password 2 times (i use the same password both on the old server and the new, upgraded one). Then things started to work and it wrote a lot with alter table, ect, ect.
Hope others can use it. :-)
Related
I am using an official postgres v12 docker image that I want to initialise with two SQL dump files that are gathered from a remote legacy v9.2 postgres server during the docker build phase:
RUN ssh $REMOTE_USER#$REMOTE_HOST "pg_dumpall -w -U $REMOTE_DB_USER -h localhost -p $REMOTE_DB_PORT --clean --globals-only -l $REMOTE_DB_NAME" >> dump/a_globals.sql
RUN ssh $REMOTE_USER#$REMOTE_HOST "pg_dump -w -U $REMOTE_DB_USER -h localhost -p $REMOTE_DB_PORT --clean --create $REMOTE_DB_NAME" >> dump/b_db.sql
By placing both a_globals.sql and b_db.sql files into the docker image folder docker-entrypoint-initdb.d, then the database is initialised with the legacy SQL files when the v12 container starts (as described here). Docker is working correctly, the dump files are retrieved successfully. However I am running into problems initialising the container's database and require guidance:
When the container starts to initialise its DB, it stops with ERROR: role $someDBRole does not exist. This is because the psql v9.2 dump SQL files DROP roles before reinstating them; the container DB does not like this. Unfortunately it is not until psql v9.4 that pg_dumpall and pg_dump have the option to --if-exists (see pg_dumpall v9.2 documentation). What would you suggest that I do in order to remedy this? I could manually edit the SQL dump files, but this would be impractical as the snapshots of the legacy DB need to be automated. Is there a way to suppress this error during container startup?
If I want to convert from ASCII to UTF-8, is it adequate to simply set the encoding option for pg_dumpall and pg_dump? Or do I need to take into consideration other issues when upgrading?
Is there a way to supress the removal and adding of the postgres super user which is in the dump SQL?
In general are there any other gotchas when containerising and/or updating a postgres DB.
I'm not familiar with Docker so I don't know how straightforward it'll be do to these things, but in general, pg_dump/dumpall output, when it's in SQL format, will work just fine after having gone through some ugly string manipulation.
Pipe it through sed -e 's/DROP ROLE/DROP ROLE IF EXISTS/', ideally when writing the .sqls, but it's fine to just run sed -i -e <...> to munge the files in-place after they're created if you don't have a full shell available. Make it sed -r -e '/^DROP ROLE/DROP ROLE IF EXISTS/ if you're worried about strings containing DROP ROLE in your data, at the cost of portability (AFAIK -r is a GNU addition to sed).
Yes. It's worth checking the data in pg12 to make sure it got imported correctly, but in the general case, pg_dump has been aware of encoding considerations since time immemorial, and a dump->load is absolutely the best way to change your DB encoding.
Sure. Find the lines that do it in your .sql, copy enough of it to be unique, and pipe it through grep -v <what you copied> :D
I can't speak to the containerizing aspect of things, but - and this is more of a general practice, not even really PG-specific - if you're dealing with a large DB that's getting migrated, prepare a small one, as similar as possible to the real one but omitting any bulky data, to test with to get everything working so that doing the real migration is just a matter of changing some vars (I guess $REMOtE_HOST and $REMOTE_PORT in your case). If it's not large, then just be comfortable blowing away any pg12 containers that failed partway through the import, figure out & do whatever to fix the failure, and start from the top again until it works end-to-end.
I have successfully logged in a PostgreSQL database with psql -d <name> -h <IP address> -U <user name>. However, commands like \d and queries like SELECT * FROM Table LIMIT 5; return nothing without errors, but only \copyright returns the paragraph. Even \? and \h give me nothing. I am pretty sure that this database is not empty because I can retrieve information with the same logins and queries with pandas.read_sql in Python and RPostgreSQL::dbGetQuery in R. Hence it should be problems in psql.
My wild guess is this might be a similar problem of pager when executing git diff as I just fixed that a few days ago. Is it possible? If so, how can I change the pager settings? If not, what can be reasons causing the situation and how to fix it?
The information of OS and PostgreSQL are listed below:
Ubuntu 14.04.5 LTS
psql (10.1, server 9.4.15)
I have two apps, with same tables. One of app collecting data from web. I want to send the datas to my second(web app)'s app database.
With the code below, I have created the file with datas:
pg_dump -U username -t public."table_name" -d database name --inserts > table_name.sql
The problem is that I just want to insert data's which does not exist in second database.
If I try the code below, I get a lot of already exists errors:
psql -U username second_database_name < table_name.sql
One of error:
multiple primary keys for table "table_name" are not allowed
Another one:
relation "table_name_attribute_442....c74_uniq" already exists
--clean , --if-exists ... What should I do?
The way I did it, was to do a pg_dump that creates a compressed archive suitable for use with pg_restore, which has the needed flags to allow the data to be imported without throwing errors.
For example:
pg_dump -Fc -h 127.0.0.1 {db_name_here} > {dump_file_name_here}
The "-Fc" gets you the file-type that pg_restore wants; it will reject a dump made without those magic letters.
Now you can restore the file with:
pg_restore -O -h 127.0.0.1 --clean --disable-triggers -d {target_db_name} {dump_file_name_here}
Voila - the data is now in the target_db.
If the 'psql' command has the equivalent flags, I don't know what they are, and did not find them in SO searching. But, hopefully, this provides a DB dump/restore that 'just works' for those who need to get back to using the DB, instead of fiddling with trying to get a simple dump/restore to happen with the expected behavior.
Also note, if you do not specify ...
-h 127.0.0.1
... it will go looking for a unix-socket, which may or may not be configured correctly. Chances are, your use-case for the DB addresses the db that way, so you never manually-configured the unix-socket to match whatever the command defaults to trying to find, which is different than how setup sets the default socket (of course, because things "just working," so you can "just use it" is just not possible).
I have a dev server with a copy of a database which can be edited and a live real server with that same database in a different state. To move the database from the dev to the live I run from the dev server:
mysqldump -u root -p --opt db_name tbl_name | mysql -u user_name -p --host=live_IP -C db_name
With the appropriate values in db_name, tbl_name, user_name, and live_IP. However, this currently drops the table on the live server and copies the dev version over -- effectively overwriting everything and trashing any new data in the live table. What I really want is to have new lines in the dev server and conflicting lines come from the dev server's copy but that any new lines in the live server's copy to remain unchanged. This is some sort of a merge and I have not been able to find any mention to something like this in the documentation but it seems like it should be possible since it is a common need.
You can do this by adding mysqldump options before the pipe. In your case I think you want --insert-ignore, --no-create-db, and --no-create-info.
It might be a dead simple question yet I still wanted to ask. I've created a Node.js application and deployed it on Heroku. I've also set up the database connection without having any trouble as well.
However, I cannot get the load the local data in my MongoDB to MongoLab I use on heroku. I've searched google and could not find a useful solution so I ended up trying these commands;
mongodump
And:
mongorestore -h mydburl:mydbport -d mydbname -u myusername -p mypassword --db Collect.1
Now when I run the command mongorestore, I received the error;
ERROR: multiple occurrences
Import BSON files into MongoDB.
When I take a look at the DB file for MongoDB I've specified and used during the local development, I see that there are files Collect.0, Collect.1 and Collect.ns. Now I know that my db name is 'Collect' since when I use the shell I always type `use Collect`. So I specified the db as Collect.1 in command line but I still receive the same errors. Should I remove all the other collect files or there is another way around?
You can't use 'mongorestore' against the raw database files. 'mongorestore' is meant to work off of a dump file generated by 'mongodump'. First us 'mongodump' to dump your local database and then use 'mongorestore' to restore that dump file.
If you go to the Tools tab in the MongoLab UI for your database, and click 'Import / Export' you can see an example of each command with the correct params for your database.
Email us at support#mongolab.com if you continue to have trouble.
-will
This can done by two steps.
1.Dump the database
mongodump -d mylocal_db_name -o dump/
2.Restore the database
mongorestore -h xyz.mongolab.com:12345 -d remote_db_name -u username -p password dump/mylocal_db_name/