I use Docker to develop.
docker exec -it <My-Name> mongo
I want to import the data to MongoDB from a JSON file, but it fails.
The command is
mongoimport -d <db-name> -c <c-name> --file xxx.json
What can I do?
With your description it seems that
you have a datafile in json format in your host machine and you want to import this data into mongodb which is running in a container.
You can follow following steps to do so.
#>docker exec -it <container-name> mongo
#>docker cp xxx.json <container-name-or-id>:/tmp/xxx.json
#>docker exec <container-name-or-id> mongoimport -d <db-name> -c <c-name> --file /tmp/xxx.json
In the last step you have to use file path that is available in the container.
To debug more if required you can login into the container and execute the way you do on the Linux machines.
#>docker exec -it <container-name-or-id> sh
sh $>cat /tmp/xxx.json
sh $>mongoimport -d <db-name> -c <c-name> --file /tmp/xxx.json
Run without copy file:
docker exec -i <container-name-or-id> sh -c 'mongoimport -c <c-name> -d <db-name> --drop' < xxx.json
Step 1: Navigate to the directory where the JSON file located from your host terminal.
Step 2: Use command "docker cp xxx.json mongo:/tmp/xxx.json” to copy the JSON file from current host directory to container "tmp" directory.
Step 3: Navigate to container command shell by using command “docker container exec -it mongo bash”.
Step 4: Import the collection from "tmp" folder to container database by using command : "mongoimport --uri="<mongodb connection uri>" —collection=<c-name> --file /tmp/xxx.json"
What we have:
mongodb running in docker container.
json file to import from local machine
mongoimport.exe on local machine
What to do to import this json file as a collection:
mongoimport --uri=<connection-string> --collection=<collection-name> --file=<path-to-file>
Example:
mongoimport --uri="mongodb://localhost:27017/test" --collection=books --file:"C:\Data\books.json"
More details regarding mongoimport here: https://www.mongodb.com/docs/database-tools/mongoimport/
Related
I have a dump of my production db, which I can restore easily in my docker container with: docker exec -it my_db_container pg_restore --user=my_user --dbname=dbname sql/current.dump. Everything works, data are here.
But when I re-dump my local database from the docker with docker exec -it my_db_container -U my-user -F c -b dbname > docker/db/current_stripped.dump back to project folder, my dump file is created (with appropriate size and content) but I can not use it for restoring (docker exec -it whasq-db pg_restore --user=my-user --dbname=dbname sql/current_stripped.dump) it again to a fresh db due to an error: pg_restore: [custom archiver] could not read from input file: end of file however the restore command is the same (except the my_user which is postgres in production) as used in production env.
I had the same problem and solved it as follows. I switched to the -f or --file option instead of piping (taken from https://stackoverflow.com/a/51073680/6593069). And I created the dump file inside the container and then copied it to my host.
That would mean in your case:
docker exec -it my_db_container -U my-user -F c -b dbname -f current_stripped.dump
docker cp my_db_container:current_stripped.dump .
I create backups like this: docker exec DOCKER pg_dump -U USER -F t DB | gzip > ./FILE.tar.gz
What's the best way to restore the database given that the database runs within a container?
For your case:
docker exec -it <CONTAINER> gunzip < backup.tar.gz | pg_restore -U <USER> -F t -d <DB>
Remote restore is also available if your container is public facing and remote connections are allowed in pg_hba.conf for postresql:
gunzip < backup.tar.gz | pg_restore -U <USER> -F t -d <DB> -h <HOST_IP> -p 5432
As a rule of thumb, it is good idea to document your backup and restore commands specific to the project.
How take backup of the data which is existing in the running PostgreSQL container
Create some folder in your root
mkdir -p '/myfolder/bdbackup'
download the postgres image which you are using and execute the following command
docker run --name demo1 -e POSTGRES_PASSWORD=passowrd -v /myfolder/bdbackup:/var/lib/postgresql/data -d postgres
docker exec -it demo1 psql -U postgres
Back up will be stored in the following folder /myfolder/bdbackup
you can kill the container and stop the container any time but data will be stored in the host.
and once again re-run the postgres the container with same command
docker run --name demo2 -e POSTGRES_PASSWORD=passowrd -v /myfolder/bdbackup:/var/lib/postgresql/data -d postgres
docker exec -it demo1 psql -U postgres
and execute following query select * from emp;
you can see the data has restored...
I've create a docker image in order to seed my dockerized mongo instance:
FROM mongo:2.6
MAINTAINER Living Digital Way
COPY ./clients.init.json .
COPY ./users.init.json .
CMD mongoimport --host mongo --db lvdb --collection clients --type json --file ./clients.init.json --jsonArray --upsert --upsertFields client_id
CMD mongoimport --host mongo --db lvdb --collection users --type json --file ./users.init.json --jsonArray --upsert --upsertFields username
I first kick off my mongo instance:
docker run --name mongo -p 27017:27017 --hostname mongo mongo:2.6
After that, I perform my image:
docker run --link mongo:mongo registry.private.com/mongo_seed:demo
This is the output:
# docker run --name mongo-seed --link mongo:mongo registry.private.com/mongo-seed:demo
Unable to find image 'registry.private.com/mongo-seed:demo' locally
v1: Pulling from mongo-seed
046d0f015c61: Already exists
ba95eb02831f: Already exists
53dc8636c4de: Already exists
a1ba40c46d70: Already exists
58b7d37cc7a7: Already exists
6fc4041cef29: Already exists
4cb494f83a39: Already exists
29839a673e80: Pull complete
cc731752cc1a: Pull complete
Digest: sha256:9a88d141b426fb7e96d2418f63c1587f6c055602d77c13ddd4ef744d66d6acc2
Status: Downloaded newer image for registry.private.com/mongo-seed:demo
connected to: mongo
2016-09-09T12:11:42.194+0000 imported 1 objects <<<<<<<<<<<<<<<<
As you can see only the last CMD is performed.
Am I doing something wrong?
There can only be one CMD instruction in a Dockerfile. If you list more than one CMD then only the last CMD will take effect. I suggest you put the two commands in to a separate import.sh, copy it to your container and run it using CMD.
COPY ./clients.init.json .
COPY ./users.init.json .
COPY ./import.sh .
RUN ["chmod", "+x", "import.sh"] # -> only required, if import.sh is not executable
CMD ["import.sh"]
i had to update a collection on my local machine and want to upload this to a docker instance, here is my procedure
$ #backup local database
$ mongodump --collection styles --db ccc-mongo
$ tar -zcvf dump.tar.gz dump
$ scp dump.tar.gz me#remote:/home/me
$ ssh me#remote
me#remote: ~ $ docker exec -i mongo-dev sh -c "cd /tmp && tar xzf - && mongorestore --collection styles --db ccc-mongo" < dump.tar.gz
this gives me an error
connected to: 127.0.0.1
Fri Dec 18 16:05:26.768 ERROR: ERROR: root directory must be a dump of a single database
Fri Dec 18 16:05:26.768 ERROR: when specifying a db name with --db
Import BSON files into MongoDB.
i just need to restore one collection without removing the other data, any advice much appreciated
this worked for me
docker exec -i mongo-dev sh -c "cd /tmp && tar xzf - && mongorestore --db ccc-mongo --drop dump/ccc-mongo && rm -Rf dump" < dump.tar.gz
need to make sure in the dump to just include the collection i want to restore only
I figured out how to do it
NOTE: please explain your question briefly. Question is not clear, please specify where you want to mondodump and restore your data - into server database or your ubuntu running computer. If into server from your local database then follow following steps:-
Here are the instructions
Copy dump folder to server
scp -r /local_path/to/dump_folder root#111.222.33.4:/remote/path
SSH into server
ssh root#111.222.33.4
Copy from root of server to inside docker container
docker cp dump_folder mongodb:/dump_folder
Go into mongodb docker container
docker exec -it mongodb bash
check if copied folder exists
ls (you should see dump_folder, if you named it the same folder as in this example)
use mongorestore
mongorestore --drop -d AppName dump_folder
Setup for the problem:
Create a data volume container
$ docker create --name dbdata -v /dbdata mongo /bin/true
Start mongo in a container linked to the data volume container
$ docker run -d --name mongo --volumes-from dbdata mongo
Verify you can connect to mongo using the mongo client
$ docker run -it --link mongo:mongo --rm mongo sh -c 'exec mongo "$MONGO_PORT_27017_TCP_ADDR:$MONGO_PORT_27017_TCP_PORT/test"'
The problem:
The docker-machine ssh takes a host and a command argument to execute on the host. I'd like to execute the following mongodump command, which works once I ssh into the docker host:
$ docker-machine ssh demo
root#demo:~# docker run --rm --link mongo:mongo -v $HOME:/backup mongo bash -c 'mongodump --out /backup --host $MONGO_PORT_27017_TCP_ADDR'
2015-09-15T16:34:02.676+0000 writing test.samples to /backup/test/samples.bson
2015-09-15T16:34:02.678+0000 writing test.samples metadata to /backup/test/samples.metadata.json
2015-09-15T16:34:02.678+0000 done dumping test.samples (1 document)
2015-09-15T16:34:02.679+0000 writing test.system.indexes to /backup/test/system.indexes.bson
However, using the docker-machine ssh command to execute the above command in a single step doesn't work for me:
$ docker-machine ssh demo -- docker run --rm --link mongo:mongo -v $HOME:/backup mongo bash -c 'mongodump --out /backup --host $MONGO_PORT_27017_TCP_ADDR'
SSH cmd error!
command: docker run --rm --link mongo:mongo -v /Users/tony:/backup mongo bash -c mongodump --out /backup --host $MONGO_PORT_27017_TCP_ADDR
err : exit status 1
output : 2015-09-15T16:53:07.717+0000 Failed: error connecting to db server: no reachable servers
So if the container to run the mongodump command can't connect to the mongo container, I figure there's probably an issue with --host $MONGO_PORT_27017_TCP_ADDR (it should be passed as is into the container, so premature expansion causing an empty string?), but a bit stumped trying to get it right. Any ideas are appreciated.
Update: I'm one step closer. The following appears to execute the command correctly, although the data isn't written to the system and the session hangs:
$ docker-machine ssh demo -- $(docker run --rm --link mongo:mongo -v $HOME:/backup mongo bash -c 'mongodump --out /backup --host $MONGO_PORT_27017_TCP_ADDR')
2015-09-15T18:02:03.347+0000 writing test.samples to /backup/test/samples.bson
2015-09-15T18:02:03.349+0000 writing test.samples metadata to /backup/test/samples.metadata.json
2015-09-15T18:02:03.349+0000 done dumping test.samples (1 document)
2015-09-15T18:02:03.350+0000 writing test.system.indexes to /backup/test/system.indexes.bson
The question asked for a solution based on docker ssh, but since no one responded, I'll answer the question myself with what is a better solution anyway.
As suggested by Nathan LeClaire (#upthecyberpunks) to me over twitter, the better solution is to avoid the hassle altogether and simply run a container to execute the mongodump command.
$ docker run \
--rm \
--link mongo:mongo \
-v /root:/backup mongo bash \
-c 'mongodump --out /backup --host $MONGO_PORT_27017_TCP_ADDR'
Not technically required for the answer, but the resulting test db backup file can then be transferred from the docker host machine to your current directory via docker scp:
$ docker-machine scp -r dev:/root/test .
Since I cannot add a comment to the orginal nice answer, just add a little explanation here, $MONGO_PORT_27017_TCP_ADDR should be the ip of our machine, for example, the virtual machine's ip in my virtualbox is 100.100.100.10, so the last line shoulb be:
-c 'mongodump --out /backup --host 100.100.100.10' or
-c 'mongodump --out /backup --host 100.100.100.10:27017'.
If not add the host field, great chances are that we will encounter some error like:
*** Failed: error connecting to db server: no reachable servers.
And thanks again to the orginal answer ^_^.