As example:
I have TSV file with data: {id:"", name:"", age:""} 100 records.
I import it to database into new collection:
mongoimport -d myDB -c people --type tsv C:\Users\User1\Downloads\PgWxXsCHH5rtmpOt4BXqZA.tsv --headerline
I decided that each record shoul have some custom field like e.g. rank so I add field to each record:
db.people.update({},{$set:{rank:0}},false,true)
I get new TSV file with updated data, for example same ids, only new ages.
Question is: how can I update same collection with new data, with pre-saving a custom field with its value. Also if TSV has new records, which are not present in collection they should be added and also same custom fields as old records but with empty or "0" value?
The command db.update is not a valid MongoDB command. The command to update a collection while setting a value for a new field is:
db.comments.update({}, { $set: { rank: 0 } }, false, true)
You do not need to define the custom fields for the new records except you require them.
Related
I am using mongoimport to import a large number of CSV records to MongoDb. Currently, by default, the _id field is being set to ObjectId, as shown:
"_id" : ObjectId("5fea8abcda05f098a1ac9857")
Is there a parameter that I can pass to the mongoimport command that will cause MongoDb to generate a new UUID instead for each document as it is imported, such as:
"_id" : UUID("0ef6bc1d-017d-4c85-9393-2d7832598d03")
Otherwise, if it's not possible to assign a UUID to the _id field on import, is there another query I can write to perform this update on the imported collection?
This seems to be not possible to switch to uuid when importing but you can do something like that:
mongoimport --db <your_db> --collection tmpCol --drop --type csv --file data.csv --headerline
And then:
db.tmpCol.find().forEach(function (document) {
document._id = UUID()
db.desiredCollection.insert(document)
})
Maybe relevant: https://forums.meteor.com/t/how-to-mongoimport-and-not-use-objectid-in-the--id-fields/23963/2
As far as I could find, mongoimport supports UUID with MongoDB Extended JSON (v2).
Extended json has Special rules for parsing $uuid fields.
According to documentation (and my tests), "_id" : UUID("0ef6bc1d-017d-4c85-9393-2d7832598d03") can be written as:
"_id" : { "$uuid": "0ef6bc1d-017d-4c85-9393-2d7832598d03" }
Unfortunately, this does not work with CSV (see Import UUID from CSV using “mongoimport”.
A ticket is opened to implement this functionality: TOOLS-2788.
My solution would be to use something like jq or yq, awk and bash to preprocess the csv files into json files with the UUID field written in the correct format, and then feed them to mongoimport`.
I have a database that contains a collection that has documents in it already. Now I'm trying to insert another .csv into the same database and collection. for example one document in the db.collection has:
Name: Bob
Age: 25
and an entry from the csv im tying to upload is like this:
Name: Bob
Age:27
How can I import the new csv without replacing any documents, just adding to the database so that both entries will be in the database.collection?
Assuming you're using mongoimport, take a look at the --mode option.
With --mode merge, mongoimport enables you to merge fields from a new
record with an existing document in the database. Documents that do
not match an existing document in the database are inserted as usual.
I would like to import data to a MongoDb document from a .tsv file using the record _id as defined in my file.
How would I go about using the _id as specified in my .tsv, specifying the MongoDb should use the provided _id rather than generating its own?
Example data set:
student firstName lastName
ab867499 example student
I want MongoDb to use the student column as _id rather than generate its own object_id as the key.
Here is what you can do:
mongoimport --db <your_db_name> --collection <your_collection_name> --type tsv --file <path_to_file> --fields _id,firstName,lastName
In this case you will want to make sure that the first line of your file does not contain the header row or simply drop the imported document for the header row after the import.
Also, make sure you have a line break at the end of your last line of data in your file since mongoimport will skip this last record otherwise.
I am trying to export data to a CSV file but for some reason I am not getting any data in the CSV file.
I have a DB called "test", and a collection called "people". The contents of the people collection is (json export works!):
{"_id":{"$oid":"55937ce0c64ddad5023a9570"},"name":"Joe Bloggs","position":"CE"}
{"_id":{"$oid":"55937d57c64ddad5023a9571"},"name":"Jane Bloggs","position":"CE"}
{"_id":{"$oid":"55937d62c64ddad5023a9572"},"name":"Peter Smith","position":"CE"}
{"_id":{"$oid":"55937d78c64ddad5023a9573"},"name":"Sarah Smith","position":"STL"}
I am trying to export this data into a CSV file with the following command:
mongoexport --type=csv -d test -c people --fieldFile c:\dev\peopleFields.txt --out c:\dev\people.csv
When I run this command, the response is:
2015-07-01T14:56:36.787+0800 connected to: localhost
2015-07-01T14:56:36.787+0800 exported 4 records
The contents of peopleFields.txt is:
ID
Name
Position
And the resulting output to the people.csv file is:
ID,Name,Position
"","",""
"","",""
"","",""
"","",""
Could someone please explain to me what I am doing wrong?
What you are missing here is that the --fieldFile option is not a "mapping" but just a "list" of all the fields you want to export from the collection.
So to actually "match" fields present in your collection the content should be:
_id
name
position
Since the names you have do not match any fields, you get four lines ( one per document ) of blank field output, for the number of fields you specify.
The mongoexport utility itself will not "map" to alternate names. If you want different names to how they are stored in your collection then you will have to alter the output yourself.
The same goes for the output as any ObjectId value will be output as that literal string.
You can use following command to export data in csv file:
mongoexport --db dbName --collection collectionName --type=csv --fields name,position --out fileName.csv
As per documentation,
1) The fieldFile allows you to specify fields to include in the export.
2) The file must have only one field per line, and the line(s) must end with the LF character (0x0A).
You are using different name (ID, Name, Position) in text file as that of in collection (_id, name, position)so you are getting empty fields exported.
I am importing data from a csv file using mongoimport. For this I execute following command
mongoimport -d {databaseName} -c {collectionName}--type csv --file {fileName} --headerline
It creates auto-generated _id field with the objectId value. and also creates index on it.
But I want that it should not create index on this. How can I make a compound index on _id and a field of that csv file ?
Quoting MongoDB documentation
MongoDB creates the _id index, which is an ascending unique index on the _id field, for all collections when the collection is created. You cannot remove the index on the _id field.
However you can create a compound indexes where your indexes can hold a reference to the _id field.
The syntax is db.test.ensureIndex({"_id": 1, "name": 1}) for example where name is another field in your document. More info