I'm trying to export just one object with mongoexport, filtering by its ID.
I tried:
mongoexport -d "kb_development" -c "articles" -q "{'_id': '4e3ca3bc38c4f10adf000002'}"
and many variations, but it keeps saying
connected to: 127.0.0.1
exported 0 records
(and I'm sure there is such an object in the collection)
In mongo shell I would use ObjectId('4e3ca3bc38c4f10adf000002'), but it does not seem to work in the mongoexport query.
I think you should be able to use ObjectId(...) in the query argument to mongoexport:
mongoexport -d kb_development -c articles -q '{_id: ObjectId("4e3ca3bc38c4f10adf000002")}'
If that does not work, you can use the "strict mode" javascript notation of ObjectIds, as documented here:
mongoexport -d kb_development -c articles -q '{_id: {"$oid": "4e3ca3bc38c4f10adf000002"}}'
(Also note that strict mode JSON is the format produced by mongoexport)
You have to specify the _id field by using the ObjectId type. In your question it was specified as a string.
CODE ::
mongoexport -h localhost -d my_database -c sample_collection -q '{key:ObjectId("50584580ff0f089602000155")}' -o my_output_file.json
NOTE :: dont forgot quotes in query
My MongoDB verion: 3.2.4. when I use mongoexport tool in mongo shell：
NOT WORK:
-q '{"_id":ObjectId("5719cd12b1168b9d45136295")}'
-q '{_id: {"$oid": "5719cd12b1168b9d45136295"}}'
WORKs:
-q "{_id:ObjectId('5719cd12b1168b9d45136295')}"
- Though in mongo doc , it says that
You must enclose the query in single quotes (e.g. ') to ensure that it
does not interact with your shell environment.
- But, single quote(') does not work! please use double quote(")!
for mongoexport version: r4.2.3
mongoexport -q '{"_id": {"$oid": "4e3ca3bc38c4f10adf000002"}}'
and for a nested field
mongoexport -q '{"_id": {"$oid": "4e3ca3bc38c4f10adf000002"}}' --fields parentField.childField
You do not have to add ObjectId or $oid as suggested by answers above. As has been mentioned by #Blacksad, just get your single and double quotes right.
mongoexport -d kb_development -c articles -q '{_id:"4e3ca3bc38c4f10adf000002"}'
many of the answers provided here didn't work for me, the error was with my double quotes. Here is what worked for me:
mongoexport -h localhost -d database_name -c collection_name -q {_id:ObjectId('50584580ff0f089602066633')} -o output_file.json
remember to use single quote only for the ObjectId string.
Related
I'm trying to create a mongo dump for 1 single document inside a mongo collection.
When I do this on windows command line:
mongodump /host:x.x.x.x /port:27017 /username:my_user /password:my_pass -d my_db -o C:\fsdump -c "fs.files" -q '{_id: ObjectId("28ad7bkjia3e927d690385ec")}'
I get this error:
positional arguments not allowed: [ObjectId(28ad7bkjia3e927d690385ec)}']
when I change the id in mongo from ObjectId("28ad7bkjia3e927d690385ec") to "28ad7bkjia3e927d690385ec", and when I dump like this:
mongodump /host:x.x.x.x /port:27017 /username:my_user /password:my_pass -d my_db -o C:\fsdump -c "fs.files" -q '{_id: "28ad7bkjia3e927d690385ec"}'
then it works as expected
so my question is how can I use mongodump and do filtering on specific ObjectId's ?
or is there another way to create an export for a subset of documents in a collection (instead of the entire collection)?
ObjectId("28ad7bkjia3e927d690385ec") is a javascript function call to the ObjectId constructor. This is not valid JSON. "28ad7bkjia3e927d690385ec" is also not a valid ObjectId.
Mongodump uses an extended form of JSON, which had a tag to indicate the field type, so you would specify an ObjectId like this:
-q '{"_id":{"$oid":"5f76b7cc0311bd14f80a3dec"}}'
I am trying to export mongodb collection with:
mongoexport -d InventoryPLC -c Spectra -o spectra.json;
but it is giving me error:
Syntax missing ; before statement
I search on mongodb website and other reference websites:
mongoexport -d InventoryPLC -c Spectra;
I want to export collection.
I checked stackoverflow examples and mongodb documentation:
mongoexport -d InventoryPLC -c Spectra -o spectra.json;
I want exported collection, but have had no luck.
Appreciate the help.
You are probably trying to run mongoexport from inside the Mongo shell. That command must be executed outside it, from your OS command shell.
I have two collections in my mongodb namely
1.companies
2.contacts
Both the companies and contact collection are interlinked. I want to export a particular companies contact into a csv. I have tried a mongo export command as follows
mongoexport --csv -d dbname -c contacts
-q {"employment_details.company_id":ObjectId("50926cff9fe3125819006dc7")};
-f {"first_name","last_name","title"} -o export.csv
I get a error as follows
SyntaxError: missing ; before statement (shell):1.
Please help me. Thanks in Advance
There could be a couple of things going on here. First, are you running mongoexport from the command line or from the mongo shell? The mongoexport command is run from the command line.
Secondly, you need to properly format the query and field parameters. You could enclose the query with single quotes, and the filed name is not a JSON document, but just a list of fields.
This would look like the following from the command line:
mongoexport --csv -d dbname -c contacts -q '{"employment_details.company_id":ObjectId("50926cff9fe3125819006dc7")}' -f "first_name","last_name","title" -o export.csv
The following query will work if it is running from commandLine
mongoexport -h host -d dbname -c contacts --csv -q '{"employment_details.company_id":ObjectId("50926cff9fe3125819006dc7")}' -f first_name,last_name,title -o export.csv
Can you use mongodump to dump the latest "x" documents from a collection? For example, in the mongo shell you can execute:
db.stats.find().sort({$natural:-1}).limit(10);
Is this same capability available to mongodump?
I guess the workaround would be to dump the above documents into a new temporary collection and mongodump the entire temp collection, but would be great to just be able to do this via mongodump.
Thanks in advance,
Michael
mongodump does not fully expose the cursor interfaces.
But you can work around it, using the --query parameter.
First get the total number of documents of the collection
db.collection.count()
Let's say there are 10000 documents and you want the last 1000.
To do so get the id of first document you want to dump.
db.collection.find().sort({_id:1}).skip(10000 - 1000).limit(1)
In this example the id was "50ad7bce1a3e927d690385ec".
Now you can feed mongodump with this information, to dump all documents a with higher or equal id.
$ mongodump -d 'your_database' -c 'your_collection' -q '{_id: {$gte: ObjectId("50ad7bce1a3e927d690385ec")}}'
UPDATE
The new parameters --limit and --skip were added to mongoexport will be probably available in the next version of the tool: https://github.com/mongodb/mongo/pull/307
Building off of Mic92's answer, to get the most recent 1000 items from a collection:
Find the _id of the 1000th most recent item:
db.collection.find('', {'_id':1}).sort({_id:-1}).skip(1000).limit(1)
It will be something like 50ad7bce1a3e927d690385ec.
Then pass this _id in a query to mongodump:
$ mongodump -d 'your_database' -c 'your_collection' -q '{"_id": {"$gt": {"$oid": "50ad7bce1a3e927d690385ec"}}}'
mongodump supports the --query operator. If you can specify your query as a json query, you should be able to do just that.
If not, then your trick of running a query to dump the records into a temporary collection and then dumping that will work just fine. In this case, you could automate the dump using a shell script that calls a mongo with a javascript command to do what you want and then calling mongodump.
I was playing with a similar requirement (using mongodump) where I wanted to do sequential backup and restore. I would take dump from last stored timestamp.
I couldn't get through --query '{ TIMESTAMP : { $gte : $stime, $lt : $etime } }'
Some points to note:
1) use single quote instead of double
2) do not escape $ or anything
3) replacing $stime/$etime with real numbers will make the query work
4) problem I had was with getting $stime/$etime resolved before mongodump executes itself
under -x it showed as
+ eval mongodump --query '{TIMESTAMP:{\$gte:$utc_stime,\$lt:$utc_etime}}'
++ mongodump --query '{TIMESTAMP:$gte:1366700243}' '{TIMESTAMP:$lt:1366700253}'
Hell, the problem was evident. query gets converted into two conditionals.
The solution is tricky and I got it after repeated trials....
escape { and } ie use { ..} . This fixes the problem.
try this:
NUM=10000
doc=selected_doc
taskid=$(mongo 127.0.0.1/selected_db -u username -p password --eval "db.${doc}.find({}, {_id: 1}).sort({_id: -1}).skip($NUM).limit(1)" | grep -E -o '"[0-9a-f]+"')
mongodump --collection $doc --db selected_db --host 127.0.0.1 -u username -p password -q "{_id: {\$gte: $taskid}}" --out ${doc}.dump
_id-based approaches may not work if you use a custom _id for your collection (such as returned by a 3rd party API). In that case, you should depend on a createdAt or equivalent field:
COL="collectionName"
HOW_MANY=10000
DATE_CUTOFF=$(mongo <host, user, pass...> dbname --quiet \
--eval "db.$COL.find({}, { createdAt: 1 }).sort({ createdAt: -1 }).skip($HOW_MANY).limit(1)"\
| grep -E -o '(ISODate\(.*?\))')
echo "Copying $HOW_MANY items after $DATE_CUTOFF..."
mongodump <host, user, pass...> -d dbname -c ${COL}\
-q "{ createdAt: { \$gte: $DATE_CUTOFF} }" --gzip
Strategy is simple but there are some challenges to doing that. I am assuming we are using _id field to do that. And we know _id field is incremental thus it is a good indicator to find recent documents.
Find X'th record accordingly in the collection
Extract the _id field of document
Use the _id field in mongodump --query
Find X'th record accordingly in the collection
We can achive that by using --eval with mongo tool.
Sort the documents newest to oldest
limit X record reverse sort it
take the first document (limit:1)
stringify Id
mongo --host=$mongodb_uri --quiet --eval db.myCollection.aggregate([{\$sort:{_id:-1}},{\$limit:$MAX_DOCUMENT},{\$sort:{_id:1}},{\$limit:1},{\$project:{_id:{\$toString:\"\$_id\"}}}])
result={ "_id" : "62440d84c18a957093f6c8a3" }
Extract the _id field of document
We need the exact value of _id so we do some regex
$(echo $result | sed -e 's/{ "_id" : "\(.*\)" }/\1/')
lastId=62440d84c18a957093f6c8a3
Use the _id field in mongodump --query
mongodump does not accept ObjectId so we should use $oid to indicate ObjectId fields.
query="{\"_id\":{\"\$gte\":{\"\$oid\":\"$lastId\"}}}"
Here is the complete bash script
dump()
{
local lastIdQuery="db.$collection.aggregate([{\$sort:{_id:-1}},{\$limit:$MAX_DOCUMENT},{\$sort:{_id:1}},{\$limit:1},{\$project:{_id:{\$toString:\"\$_id\"}}}])"
echo "lastIdQuery $lastIdQuery"
local lastIdResult=$(mongo --host=$mongodb_uri --quiet --eval "$lastIdQuery")
echo "lastIdResult $lastIdResult"
local lastId=$(echo $lastIdResult | sed -e 's/{ "_id" : "\(.*\)" }/\1/')
echo $lastId
query="{\"_id\":{\"\$gte\":{\"\$oid\":\"$lastId\"}}}"
echo "query $query"
mongodump --uri=$mongodb_uri --collection $collection --query="$query" --out=$outFolder
}
mongodb_uri='mongodb://localhost:27017/myDb'
outFolder=./backup
MAX_DOCUMENT=100
collection="users"
dump
mongoexport -h db.mysite.com -u myUser -p myPass -c myCollection
But the response I get is:
ERROR: too many positional options
What's that about?
I had this same problem. In my case, I was using mongoexport with the --query option, which expects a JSON document, such as:
mongoexport ... --query {field: 'value'} ...
I needed to surround the document with quotes:
mongoexport ... --query "{field: 'value'}" ...
I had the same problem. Found a group post somewhere which said to remove the space between the '-p' and the password, which worked for me.
Your sample command should be:
mongoexport -h db.mysite.com -u myUser -pmyPass -c myCollection
The same error I have encountered while importing a csv file.
But its just, the fact that the field list which you pass for that csv file import may have blank spaces.
Just clear the blank spaces in field list.
Its the parsing error.
I had the same issue with mongodump. After searching a bit, I found out that using the --out parameter to specify the output directory would solve this issue. The syntax for using the out parameter is
mongoexport --collection collection --out collection.json
Also in case your Mongodb instance isn't running, then you could use the --dbpath to specify the exact path to the files of your instance.
Source: http://docs.mongodb.org/manual/core/import-export/
I had the same issue with the mongoexport utility (using 2.0.2). My fix was to use the FULL parameter name (i.e. not -d, instead use --db).
Sometimes editor will screw it up (such as evernote). I fixed the issue by retyping the command in terminal.
I was also stuck in same situation and found what was causing it.
Make sure you are exporting in CSV format by adding parameter --type csv
Make sure there are no spaces in fields name,
Example: --fields _id, desc is wrong but --fields id,desc,price is good
This also works if you place the -c option first. For me, this order does work:
mongoexport -c collection -h ds111111.mlab.com:11111 -u user -p pass -d mydb
You can also leave the pass out and the server will ask you to enter the pass. This only works if the server supports SASL authentication (mlab does not for example).
for the (Error: Too many arguments)
Dont Use Space Between the Fields
try:
mongoexport --host localhost --db local --collection epfo_input --type=csv --out epfo_input.csv --fields cin,name,search_string,EstablishmentID,EstablishmentName,Address,officeName
Dont_Try:
mongoexport --host localhost --db local --collection epfo_input --type=csv --out epfo_input.csv --fields cin,name,search_string,Establishment ID,Establishment Name,Address,office Name
Had a similar issue
$too many positional arguments
$try 'mongorestore --help' for more information
Simply fix for me was to wrap the path location in quotes " "
This Failed:
mongorestore -h MY.mlab.com:MYPORT -d MYDBNAME -u ADMIN -p PASSWORD C:\Here\There\And\Back\Again
This Worked:
mongorestore -h MY.mlab.com:MYPORT -d MYDBNAME -u ADMIN -p PASSWORD "C:\Here\There\And\Back\Again"
I had the same issue with starting mongod. I used the following command:
./mongod --port 27001 --replSet abc -- dbpath /Users/seanfoley/Downloads/mongodb-osx-x86_64-3.4.3/bin/1 --logpath /Users/seanfoley/Downloads/mongodb-osx-x86_64-3.4.3/log.1 --logappend --oplogSize 5 --smallfiles --fork
The following error message appeared:
Error parsing command line: too many positional options have been specified on the command line
What fixed this issue was removing the single space between the '--' and 'dbpath'
I had the same issue while using the "mongod --dbpath" command. What I was doing looked somewhat like this:
mongod --dbpath c:/Users/HP/Desktop/Mongo_Data
where as the command syntax was supposed to be:
mongod --dbpath=c:/Users/HP/Desktop/Mongo_Data
This worked for me. Apart from this one may take a note of the command function and syntaxes using the mongod --help command.
In my case, I had to write the port separately from the server connection. This worked for me:
mongoexport --host=HOST --port=PORT --db=DB --collection=COLLECTION
--out=OUTPUT.json -u USER -p PASS
Create a json file in the same folder where you have your mongod.exe.
eg: coll.json
and open a command prompt in this folder.
type this below in CMD.
mongoexport --db databasename --collection collectionname --out
coll.json
and you will see like a progress bar very cool exporting all data.