everyone I'm trying to export query in csv format from cmd with mongoexport. I wrote the following line:
mongoexport --db=wine --collection=review --type=json --fields=_id,points,title,description,taster_name,taster_twiter_handle,price,designation,variety,region_1,region_2,province,country,winery --query="{'taster_name':{exists:true}}" --out=review_cleaned.csv
I think that is correct but I'm obtaining this error:
2021-12-08T17:02:36.695+0100 connected to: mongodb://localhost/
2021-12-08T17:02:36.760+0100 Failed: error parsing query as Extended JSON: invalid JSON input
What am I wrong?
Thanks in advance.
You must escape the double quotes on the query with a backslash:
mongoexport --db=wine --collection=review --type=json --fields=_id,points,title,description,taster_name,taster_twiter_handle,price,designation,variety,region_1,region_2,province,country,winery --query=\"{'taster_name':{exists:true}}\" --out=review_cleaned.csv
I am new to mongodb and i am trying to import .json files.I created a database on sql developer and exported three of my tables in 3 separate .json files that look like this,
{"results":[{"columns":[{"name":"CLUBID","type":"NUMBER"},{"name":"MANAGERID","type":"NUMBER"},{"name":"NAME","type":"VARCHAR2"},{"name":"CITY","type":"VARCHAR2"},{"name":"CREATION_DATE","type":"DATE"}],"items":
[
{"clubid":2001,"managerid":5376,"name":"FC KOPRITIS","city":"LAKONIA","creation_date":"03\/07\/99"}
,{"clubid":2002,"managerid":5377,"name":"FC NOE","city":"KITHERA","creation_date":"10\/11\/14"}
,{"clubid":2003,"managerid":5378,"name":"FC KRK","city":"MELOS","creation_date":"31\/01\/39"}
,{"clubid":2004,"managerid":5379,"name":"FC FOCUSRITE","city":"THERA","creation_date":"02\/02\/02"}
,{"clubid":2005,"managerid":5380,"name":"FC GHOST","city":"SERIFOS","creation_date":"05\/08\/64"}
,{"clubid":2006,"managerid":5431,"name":"FC ALITIS","city":"LIMNOS","creation_date":"22\/10\/45"}
,{"clubid":2007,"managerid":5432,"name":"FC VLOSPA","city":"MIKONOS","creation_date":"30\/08\/85"}
,{"clubid":2008,"managerid":5433,"name":"FC MADCLIP","city":"CAPITAL","creation_date":"01\/04\/01"}
,{"clubid":2009,"managerid":5436,"name":"FC SNIK","city":"ATHENS","creation_date":"18\/07\/98"}
,{"clubid":2010,"managerid":5435,"name":"FC YTM","city":"XANTHI","creation_date":"20\/04\/18"}
]}
I tried using mongoimport --jsonArray --file club.json but it didn't work.
I get errors like "unexpected EOF" or "no collection specified"
The following steps resulted in a successful import:
Get the desired records to import (cleanse the data):
{"clubid":2001,"managerid":5376,"name":"FC KOPRITIS","city":"LAKONIA","creation_date":"03\/07\/99"}
{"clubid":2002,"managerid":5377,"name":"FC NOE","city":"KITHERA","creation_date":"10\/11\/14"}
{"clubid":2003,"managerid":5378,"name":"FC KRK","city":"MELOS","creation_date":"31\/01\/39"}
{"clubid":2004,"managerid":5379,"name":"FC FOCUSRITE","city":"THERA","creation_date":"02\/02\/02"}
{"clubid":2005,"managerid":5380,"name":"FC GHOST","city":"SERIFOS","creation_date":"05\/08\/64"}
{"clubid":2006,"managerid":5431,"name":"FC ALITIS","city":"LIMNOS","creation_date":"22\/10\/45"}
{"clubid":2007,"managerid":5432,"name":"FC VLOSPA","city":"MIKONOS","creation_date":"30\/08\/85"}
{"clubid":2008,"managerid":5433,"name":"FC MADCLIP","city":"CAPITAL","creation_date":"01\/04\/01"}
{"clubid":2009,"managerid":5436,"name":"FC SNIK","city":"ATHENS","creation_date":"18\/07\/98"}
{"clubid":2010,"managerid":5435,"name":"FC YTM","city":"XANTHI","creation_date":"20\/04\/18"}
Verify your file location, name and path. My file is in a sub directory of my current working directory called testData and the name is json1.JSON.
Execute the import (the db and collection will be created if they don't exist):
mongoimport --db tst2 --collection so2 --file testData/json1.JSON
Results:
2020-04-06T21:39:10.177-0400 connected to: mongodb://localhost/
2020-04-06T21:39:10.179-0400 10 document(s) imported successfully. 0 document(s) failed to import.
I'm trying to import from CSV into mongodb 3.4 using mongoimport, and I would like for empty columns to be imported as null values for the fields.
I was under the impression from the mongoimport documentation, that if --ignoreBlanks was not specified that I would get the behavior I wanted.
--ignoreBlanks
Ignores empty fields in csv and tsv exports. If not
specified, mongoimport creates fields without values in imported
documents.
However, when I try to load this sample data without --ignoreblanks:
field_1.string(),field_2.int32(),field_3.string()
A,5,B
C,,D
E,7,F
then I get an error on any field that is not a string.
mongoimport --collection null_test --type csv --headerline --columnsHaveTypes --file null_test.csv --verbose
2017-08-08T16:55:42.989+0000 filesize: 67 bytes
2017-08-08T16:55:42.989+0000 using fields: field_1,field_2,field_3
2017-08-08T16:55:43.001+0000 connected to: localhost:27017
2017-08-08T16:55:43.001+0000 ns: DEV.null_test
2017-08-08T16:55:43.001+0000 connected to node type: standalone
2017-08-08T16:55:43.001+0000 using write concern: w='1', j=false, fsync=false, wtimeout=0
2017-08-08T16:55:43.001+0000 using write concern: w='1', j=false, fsync=false, wtimeout=0
2017-08-08T16:55:43.001+0000 Failed: type coercion failure in document #1 for column 'field_2', could not parse token '' to type int32
2017-08-08T16:55:43.001+0000 imported 0 documents
For fields that are strings it loads an empty string rather than a null.
What am I doing wrong here? Is it possible to load fields as NULL using mongoimport with CSV or TSV files?
For what it's worth, if I use mongoimport to import a json file with NULLs, it imports them just fine as actual NULLs.
[
{
"field1": "A",
"field2": null,
"field3": "C"
},
{
"field1": 1,
"field2": 5,
"field3": null
}
]
MongoDB will never import null values from CSV data.
I'm guessing that's because it doesn't make too much sense given that querying for "field": null will return all documents where "field" is missing or null.
The -ignoreBlanks option will simply prevent the import from creating empty string ("") values for missing fields which would otherwise be the default.
You can get what you want, though, by post-processing your imported documents using the following update:
collection.update({'field_2': {$exists: false}}, {$set: {'field_2': null}})
mongoimport --db dbName --collection collectionName < /Users/pratikjoshi/Desktop/FileName.json --jsonArray
I run this command in shell scripts!
Here is my json file content
[
{
"trackingRecordId":5742294,
"longitude":77.126205,
"latitude":28.54711,
"batteryPerc":100,
"speed":0.13,
"createdOnDt":"2016-01-14 00:00:01"
},
{
"trackingRecordId":5742293,
"longitude":72.86727,
"latitude":19.112692,
"batteryPerc":51.82,
"speed":10,
"createdOnDt":"2016-01-13 23:59:59"
}
]
Well it clearly shows that your JSON is corrupted,
just goto JSON Validator, validate your JSON file and try again.
Hope it helps
EDIT
Well your json is completely valid and your command for export is also valid and working pretty much OK with me.
I have a json file of around 4M json lines, I tried to use:
mongoimport --db mydb --collection mycoll --file myfile.json
and what happened was weird. I got this error:
2018-12-29T17:00:50.424+0200 connected to: localhost
2018-12-29T17:00:50.483+0200 Failed: error processing document #1428: invalid character 'S' after object key:value pair
2018-12-29T17:00:50.483+0200 imported 1426 documents
so first I went to count this collection in mongo and saw that there are 1000 documents and not 1426 as the above mentioned.
second, I located a json with the 'S' in it, which is just a string that looks like "name" : "Double 'S' Transport" and left only this json in the file, import and it worked well.
does anyone understands why is it happening? my suspicion is that mongoimport dosent work on files that big...
any help would be great :)