I use the mongoimport command in shell to load a json file in MongoDB.
mongoimport \
--host ${MONGO_HOST}:${MONGO_PORT} \
--db ${MONGO_DB} \
--type json \
--collection ${COLLECTION} \
--file ${DATA_IN_PATH}/${FILENAME}.json \
--upsert \
--upsertFields ${UPSERT_FIELDS}
I would like to count the number of documents in my collection, before and after the load and get it in a shell variable
I tried Using the --eval command and put the result into a variable:
CollCount=$(mongo \
${MONGO_HOST}:${MONGO_PORT}/${MONGO_DB} \
--eval "db.getCollection('${COLLECTION}').count({})")
but my var CollCount contains:
MongoDB shell version: 2.6.9 connecting to: localhost:27017/mydb 12236
(The value 12236 is correct.)
Is it a better way to do this?
Related
Mongodb version 3.2.12. I have two local databases, "base1" and "base2"
I want to copy all data (all collections) from base1 over to base2, replacing everything there (like when dumping production to a dev environment).
Any pipe command (or other simple way) to do this?
I tried
mongodump --archive --db base1 | mongorestore --db base2 --archive
lists a lot of "writing base1.collectionname to archive on stdout", but nothing gets written to base2.
I also tried
mongodump --db base1 --gzip --archive=/path/to/file.gz
mongorestore --db base2 --gzip --archive=/path/to/file.gz
Dump works, restore just says "creating intents for archive", "done"
I come across the same issue and after some googling and search I found this post
https://stackoverflow.com/a/43810346/3785901
I tried this command mentioned:
mongodump --host HOST:PORT --db SOURCE_DB --username USERNAME --password PASSWORD --archive | mongorestore --host HOST:PORT --nsFrom 'SOURCE_DB.*' --nsTo 'TARGET_DB.*' --username USERNAME --password PASSWORD --archive --drop
and it works like a charm.
It should work in your case, good luck.
I use following commands:
mongodump \
--host ${mongo.host} \
--port ${mongo.port} \
--username ${mongo.backup_restore_user} \
--password ${mongo.backup_restore_password} \
--db ${mongo.db} \
--gzip \
--dumpDbUsersAndRoles \
--archive=${archive}
and
mongorestore \
--keepIndexVersion \
--drop \
--gzip \
--restoreDbUsersAndRoles \
--db ${mongo.db} \
--host ${mongo.host} --port ${pims.mongo.port} \
--username ${mongo.backup_restore_user} \
--password ${mongo.backup_restore_password} \
--archive=${archive}
I'm tring another aproach. Full dump and the daily dump of new data using:
oid=$(mongo --quiet --eval 'ObjectId.fromDate(ISODate("2017-03-29 00:10:20"))')
mongodump -q "{_id:{$gt:$oid}}" -d dbname --collection name_data
But I'm getting:
Failed: error parsing query as json: invalid character ':' looking for beginning of object key string
The first $ needs to be escaped:
mongodump -q "{_id:{\$gt:$oid}}" -d dbname --collection name_data
I am trying to mongoexport (Version 2.6) MongoDB data into csv format using the command as follows
mongoexport --port 27017 -d test -q "{userId:{$exists:true} , name:'John'}"-c user_datas -f userId --csv -o /myOutFile.csv
and i got this error message:
assertion: 16619 code FailedToParse: FailedToParse: First character in field must be [A-Za-z$_]: offset:9 of:{userId:{true},name:John}
according to error message that something happened on '$exists' that caused the error .
whats wrong with my command?
You need to invert the quotes:
'{userId: {$exists: true} , name: "John"}'
Working command:
mongoexport \
--port 27017 \
-d test \
-q '{userId: {$exists: true} , name: "John"}' \
-c user_datas \
-f userId \
--csv \
-o /myOutFile.csv
UPDATE: this post applied to meteor.com free hosting, which has been shutdown and replaced with Galaxy, a paid Meteor hosting service
I'm using this command
C:\kanjifinder>meteor mongo --url kanjifinder.meteor.com
to get access credentials to my deployed mongo app, but I can't get mongoimport to work with the credentials. I think I just don't exactly understand which part is the username, password and client. Could you break it down for me?
result from server (I modified it to obfuscate the real values):
mongodb://client:e63aaade-xxxx-yyyy-93e4-de0c1b80416f#meteor.m0.mongolayer.com:27017/kanjifinder_meteor_com
my mongoimport attempt (fails authentication):
C:\mongodb\bin>mongoimport -h meteor.m0.mongolayer.com:27017 -u client -p e63aaade-xxxx-yyyy-93e4-de0c1b80416f --db meteor --collection kanji --type csv --file c:\kanjifinder\kanjifinder.csv --headerline
OK got it. This helped:
http://docs.mongodb.org/manual/reference/connection-string/
mongoimport --host meteor.m0.mongolayer.com --port 27017 --username client --password e63aaade-xxxx-yyyy-93e4-de0c1b80416f --db kanjifinder_meteor_com --collection kanji --type csv --file c:\kanjifinder\kanjifinder.csv --headerline
Using mongodump and mongorestore also works:
Dump data from existing mongodb (mongodb url: mongodb://USER:PASSWORD#DBHOST/DBNAME)
mongodump -h DBHOST -d DBNAME -u USER -p PASSWORD
This will create a "dump" directory, with all the data going to dump/DBNAME.
Get the mongodb url for the deployed meteor app (i.e. www.mymeteorapp.com)
meteor mongo --url METEOR_APP_URL
Note: the PASSWORD expires every min.
Upload the db dump data to the meteor app (using an example meteor db url)
mongorestore -u client -p dcc56e04-a563-4147-eff4-5ae7c1253c9b -h production-db-b2.meteor.io:27017 -db www_mymeteorapp_com dump/DBNAME/
All the data should get transferred!
If you get auth_failed error message your mongoimport version is too different from what's being used in meteor.com. So you need to upgrade. For ubuntu see https://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/#install-the-latest-stable-version-of-mongodb
#!/bin/sh
# Script to import csvfile to meteor application deployed to free meteor.com hosting.
# Make sure your versions of mongo match with the metor.com mongo versions.
# As Jan 2016 it seems to be 3.x something. Tested with mongoimport 3.12.
if [ $# -eq 0 ]
then
echo "usage: $0 xxx.meteor.com collection filename.csv"
exit 1
fi
URL=$1
COLLECTION=$2
FILE=$3
echo Connecting to $URL, please stand by.... collection=$COLLECTION file=$FILE
PUPMS=`meteor mongo --url $URL | sed 's/mongodb:\/\// -u /' | sed 's/:/ -p /' | sed 's/#/ -h /' | sed 's/\// -d /'`
mongoimport -v $PUPMS --type csv --headerline --collection $COLLECTION --file $FILE
How do I export the results of a MongoDB command to a flat file
For example, If I am to get db.collectionname.find() into a flat file.
I tried db.collectionname.find() >> "test.txt" doesnt seem to work.
you can try the following from the command line
mongo 127.0.0.1/db --eval "var c = db.collection.find(); while(c.hasNext()) {printjson(c.next())}" >> test.txt
assuming you have a database called 'db' running on localhost and a collection called 'collection' this will export all records into a file called test.txt
If you have a longer script that you want to execute you can also create a script.js file
and just use
mongo 127.0.0.1/db script.js >> test.txt
I hope this helps
I know of no way to do that from the mongo shell directly, but you can get mongoexport to execute queries and send the results to a file with the -q and -o options:
mongoexport -h mongo.dev.priv -d models -c profiles -q '{ $query : { _id : "MRD461000" } }' -o MRD_Series1.json
The above hits queries the profiles collection in the models database grabbing the JSON document for _id = "MRD641000". Works for me.
Use this
mongo db_name --username user --password password < query1.js >> result.txt
Try this - returns a json file with the data of the query, you can change .json for .txt and other.
mongoexport --db products --collection clicks --query '{"createdInt":{$gte:20190101}, "clientId":"123", "country":"ES"}' --out clicks-2019.json
Having missed the db needing to be the actual db in Peshkira's answer, here is a general syntax for a one liner in shell (assuming no password):
mongo <host>:<db name> --eval "var x = <db name>.<collection name>.<query>; while(x.hasNext()) { printjson( x.next() ) }" >> out.txt
I tested it both on my mac and Google cloud Ubuntu 15 with Mongo 3+.
Install MongoDB Compass, then it will have a tool to export query result to Json/CSV files.
mongoexport --host 127.0.0.1 --port 27017 --username youruser -p yourpass \
-d yourDatabaseName -c collectionName --type csv \
--fields field1,field2 -q '{"field1" : 1495730914381}' \
--out report.csv
mongoexport --db db_name --collection collection_name --csv --out file_name.csv -f field1,field2, field3