i have a .csv file that holds person datas.A line for a person like this.
"20781","IN","Ms.","Roy","","Serrano","2002-04-16 00:00:00","20781#adventure-works.com"
How can i give field names to mongoimport?
Field names -> id,type,firstname,middlename,lastname,modifieddate,e-mail.
you can do like Bellow
mongoimport -d test -c person -type csv --file person.csv --fields "id,type,firstname,middlename,lastname,modifieddate,e-mail"
this will import your file as in one document.
Related
I am exporting collection to CSV format in MongoDB. while exporting into CSV I have put all the fieldname into a text file. but, it is not exporting the data in the collection, only header are exported.
WHERE IS THE PROBLEM?
mongoexport --db users --collection contacts --type=csv --fieldFile c:/data.txt --out /opt/backups/contacts.csv
I have saved fieldname in data.txt as
email XOXA
name XOXA
You don't need to export field names to a separate file if you are not using really old mongo version.
Try this
mongoexport --db users --collection contacts --type=csv -f email,name --out /opt/backups/contacts.csv
Make sure that you don't have spaces between field names
-f email, name // this is wrong
If this command works, then you have problem with your fieldFile.
I'm trying to import a csv into mongodb using the following command:
mongoimport --db users --collection contacts --file data.csv
--headerline
The database exists but not the collection, I want to create it and use the first row of the csv as the field names. Why am I getting error:
error validating settings: must specify --fields, --fieldFile or
--headerline to import this file type
I also would like to know:
how to copy/import data from one collection into another (basically
the syntax)
how datatypes from csv are handled in mongodb when
imported; do I need to specify datatypes for headers or will mongodb
read it from csv types?
To solve this:
Either make sure the first line of your data.csv file has field names of the data to be parsed and then execute:
mongoimport --db users --collection contacts --type csv --headerline --file data.csv
Or
Define the list of field names that the values of csv would be parsed in using --fields
mongoimport --db users --collection contacts --type csv --file data.csv --fields["name","surname","etc"]
You should write command like this:
mongoimport --db users --collection contacts --type csv --file data.csv --fields "name","surname","etc"
I have an Excel file that I converted to a csv and imported into my running MongoDB storage, but there was trouble with one column of the data from the csv file. One column, called Room, occasionally, but not always, contains values separated by a comma (ex. "101, 103").
Running:
$ mongoimport -d mydb -c things --type csv --file locations.csv --headerline
gave no errors, but for documents that are supposed to have 2 values for Room, there was just one. For example "101, 102" became "101," in the db.
Is there an option for mongoimport that allows me to specify an array for a certain column?
First you need to Import the data from CSV as
$ mongoimport -d mydb -c things --type csv --file locations.csv --headerline
After that , you just have to use
db.things.find().snapshot().forEach(function (el) { el.Room = el.Room.split(','); db.things.save(el); });
So, It will solve your problem.
I seem to keep having this error when i try and import anything?
In terminal I input:
name:~ computer$ mongoimport --db users --collection contacts --type csv --file /Users/computer/Desktop/ftse100.csv
connected to: 127.0.0.1
assertion: 9998 you need to specify fields
I wouldn't know what to ask. I tried adding --field after this command line but just get help information.
ER
As per mongodb docs
--fields <field1[,field2]>, -f
Specify a comma separated list of field names when importing csv or tsv files that do not have field names in the first (i.e. header) line of the file.
mongoimport --db users --collection contacts --type csv --file /Users/computer/Desktop/ftse100.csv --fields field1, field2,field3
As per your question, there is a typo it's not --field instead --fields
In 2.4.6, mongoimport does not find the header in csv files that I make, with or without double quote boundaries.
If I chop off the header line and supply that same text to the -f or --fields option, it my files import fine.
If you want to add all columns, use --headerline option instead of -fields.
In your case it would be:
mongoimport --db users --collection contacts --type csv --headerline --file /Users/computer/Desktop/ftse100.csv
I'm trying to import and merge multiple CSVs into mongo, however documents are getting replaced rather than merged.
For example, if I have one.csv:
key1, first column, second column
and two.csv:
key1, third column
I would like to end up with:
key1, first column, second column, third column
But instead I'm getting:
key1,third column
Currently I'm using:
mongoimport.exe --ftype csv --file first.csv --fields key,firstColumn,secondColumn
mongoimport.exe --ftype csv --file second.csv --fields key,thirdColumn --upsert --upsertFields key1
That's the way mongoimport works. There's an existing new feature request for merge imports, but for now, you'll have to write your own import to provide merge behavior.
cross-collection workaround: forEach method can be run on a dummy collection and the resulting doc objects used to search/update your desired collection:
mongoimport.exe --collection mycoll --ftype csv --file first.csv --fields key,firstColumn,secondColumn
mongoimport.exe --collection dummy --ftype csv --file second.csv --fields key,third
db.dummy.find().forEach(function(doc) {db.mycoll.update({key:doc.key},{$set:{thirdcol:doc.third}})})
That's correct, mongoimport --upsert updates full documents.
You may achieve your goal by importing to a temporary collection and using the following Gist.
Load the script to Mongo Shell and run:
mergeCollections("srcCollectionName", "destCollectionName", {}, ["thirdColl"]);
I just had a very similar problem. There is a node module for mongo and jline is my command line node tool for stream processing JSON lines. So:
echo '{"page":"index.html","hour":"2015-09-18T21:00:00Z","visitors":1001}' |\
jline-foreach \
'beg::dp=require("bluebird").promisifyAll(require("mongodb").MongoClient).connectAsync("mongodb://localhost:27017/nginx")' \
'dp.then(function(db){
updates = {}
updates["visitors.hour."+record.hour] = record.visitors;
db.collection("pagestats").update({_id:record.page},{$set:updates},{upsert:true});});' \
'end::dp.then(function(db){db.close()})'
In your case you'd have to convert from csv to JSON lines first by piping it through jline-csv2jl. That converts each CSV line into a dictionary with names taken from the header.
I have added this example to the manual: https://github.com/bitdivine/jline/blob/master/bin/foreach.md
I haven't used jline with promises much but so far it's OK.
Disclaimer: I am the author of jline.