Failed: error processing document in mongoimport - mongodb
I am getting the below error while reading a csv file:
Failed: error processing document #1: invalid character 'a' in literal new or null (expecting 'e' or 'u')
There are some blank fields, which I am suspecting needs to be presented as 'null' to be read properly. Am I correct here?
SAMPLE CSV:
name,year,battle_number,attacker_king,defender_king,attacker_1,attacker_2,attacker_3,attacker_4,defender_1,defender_2,defender_3,defender_4,attacker_outcome,battle_type,major_death,major_capture,attacker_size,defender_size,attacker_commander,defender_commander,summer,location,region,note
Battle of the Golden Tooth,298,1,Joffrey/Tommen Baratheon,Robb Stark,Lannister,,,,Tully,,,,win,pitched battle,1,0,15000,4000,Jaime Lannister,"Clement Piper, Vance",1,Golden Tooth,The Westerlands,
Battle at the Mummer's Ford,298,2,Joffrey/Tommen Baratheon,Robb Stark,Lannister,,,,Baratheon,,,,win,ambush,1,0,,120,Gregor Clegane,Beric Dondarrion,1,Mummer's Ford,The Riverlands,
I guess you didn't specify the file type with --type csv so mongoimportassumes you import a JSON file by default
--> Try to import with --type csv --headerline
In AWS documentation, it missing following 3 lines in CSV file import.
--type=csv \
--headerline \
--ignoreBlanks \
After I added following 3 lines to the code as following, CSV file imported successfuly to AWS documentdb.
mongoimport --ssl \
--host="sample-cluster.node.us-east-1.docdb.amazonaws.com:27017" \
--collection=sample-collection \
--db=sample-database \
--type=csv \
--headerline \
--ignoreBlanks \
--file=<yourFile> \
--numInsertionWorkers 4 \
--username=sample-user \
--password=abc0123 \
--sslCAFile rds-combined-ca-bundle.pem
Related
mongoexport works on local but not on remote server
When I run this code to export my local database, it works well and a file is successfully generated: mongoexport \ --host="localhost:27017" \ -d springDb \ -c posts \ --out social.json But when I try to access my remote server with the similar command here: mongoexport \ --host="mongodb+srv://clustxxx.xxx.mongodb.net/xxxx" \ -d socialDeveloper \ -c posts \ --out social.json I get this error: error parsing command line options: error parsing URI from mongodb:///clustxxx.xxx.mongodb.net/xxxx:27017/?replicaSet=mongodb+srv:: error parsing uri: must have at least 1 host What am I doing wrong please?
MongoDB: add a date using mongoimport with --columnsHaveTypes --headerline
I have a test.tsv TSV file that has a header looking like this sample.string() organism.string() capture.string() sex.string() and I am using the following command to import my file into mongodb mongoimport --quiet -d somedb --collection=somecollection --file=test.tsv --type tsv --mode upsert --upsertFields sample --columnsHaveTypes --headerline I wish to add the date following mongodb format at the moment of my file import. I found a way to do it while in mongodb but I can't seem to find anything about mongoimport. Is there a command (or a workaround) that looks like the following and allows us to add the date of the file content ? mongoimport --quiet -d somedb --collection=somecollection --file=test.tsv --type tsv --mode upsert --upsertFields sample --columnsHaveTypes --headerline --addDate Thanks in advance.
I found a workaround for this:
First, get the date from bash with some formatting:
date=$(date -u +"%d-%m-%Y %T")
Now, add this variable to all the lines in file.tsv like so:
awk -v date="$date" '{print $0 "\t" date}' file.tsv
Then, we can change the header by adding the date format type
$ sed -i '1s/.*/Sample.string()\tOrganism.string()\tCapture.string()\tSex.string()\tDate.date(02-01-2006 15:04:05)/' file.tsv
$ head file.tsv
Sample.string() Organism.string() Capture.string() Sex.string() Date.date(02-01-2006 15:04:05)
Then, we can import with mongoimport:
$ mongoimport -d somedb --collection=somecollection --file=file.tsv --type tsv --mode upsert --upsertFields Sample --columnsHaveTypes --headerline
2019-09-30T11:23:37.934+0200 connected to: xxxx
2019-09-30T11:23:37.940+0200 imported 1 document
Return MongoDB query result to shell
I use the mongoimport command in shell to load a json file in MongoDB.
mongoimport \
--host ${MONGO_HOST}:${MONGO_PORT} \
--db ${MONGO_DB} \
--type json \
--collection ${COLLECTION} \
--file ${DATA_IN_PATH}/${FILENAME}.json \
--upsert \
--upsertFields ${UPSERT_FIELDS}
I would like to count the number of documents in my collection, before and after the load and get it in a shell variable
I tried Using the --eval command and put the result into a variable:
CollCount=$(mongo \
${MONGO_HOST}:${MONGO_PORT}/${MONGO_DB} \
--eval "db.getCollection('${COLLECTION}').count({})")
but my var CollCount contains:
MongoDB shell version: 2.6.9 connecting to: localhost:27017/mydb 12236
(The value 12236 is correct.)
Is it a better way to do this?
Mongo Import - Failed: fields '' and '.' are incompatible
I'm importing external data from Oracle to HDFS using Sqoop, after that I'm trying to import it to MongDB, follow the command that I'm usign and the error message: -- Import external file to HDFS sqoop import \ --connect "jdbc:oracle:thin:#(description=(address=(protocol=tcp)(host=hostname)(port=1521))(connect_data=(service_name=SID)))" \ --username user --table schema. my_table \ --num-mappers 1 --verbose -P --Import HDFS file to MongoDB hadoop fs -text /user/cloudera/mytable/part* | mongoimport -d database -c my_table --type csv --headerline Error: mongoimport: /usr/lib64/libcrypto.so.10: no version information available (required by mongoimport) mongoimport: /usr/lib64/libssl.so.10: no version information available (required by mongoimport) 2015-12-08T07:08:24.001-0800 Failed: fields '' and '.' are incompatible 2015-12-08T07:08:24.001-0800 imported 0 documents text: Unable to write to output stream.
mongoDB mongoimport error .. getting file doesn't exist error
I am a newbie in Mongodb. when i am trying to mongoimport i am getting the below error. I have the file placed in c:\mongo\data\db\mongo.csv ... can anyone pls help me. C:\mongodb\bin>mongoimport.exe -d test -c foo --file c:\mongo\data\db\mongo.csv --type csv connected to: 127.0.0.1 file doesn't exist: c:\mongo\data\db\mongo.csv
either you should give path like as you are in C directive only
C:\mongodb\bin>mongoimport.exe -d test -c foo --file /mongo/data/db/mongo.csv --type csv
or you can give path in a qoute (" ") as mentioned by Gianpj
C:\mongodb\bin>mongoimport.exe -d test -c foo --file "c:\mongo\data\db\mongo.csv" --type csv
Are you sure the file does exist? There are no spaces in the file path? Try with double quotes: --file "c:\mongo\data\db\mongo.csv" Lastly where did got that .csv file? from mongoexport --csv ?