I want to insert data in a mongodb collection with a javascript loop.
Here is my code :
var collection = db.DATA.find();
for (var i = 0; i < 3; i++) {
db.PARSED_DATA.insert(collection[i].folders);
}
I have the error :
2017-08-04T10:17:06.390+0200 E QUERY [thread1] TypeError: collection[i] is undefined :
#(shell):2:9
When i execute in mongodb shell the commands :
var collection = db.DATA.find();
db.PARSED_DATA.insert(collection[0].folders);
db.PARSED_DATA.insert(collection[1].folders);
db.PARSED_DATA.insert(collection[2].folders);
I have the expected result and i still don't uderstand why my for loop doesn't work.
EDIT :
When i execute this inside Robo 3T i get the expected result :
var collection = db.DATA.find().toArray();
for (var i = 0; i < 3; i++) {
db.PARSED_DATA.insert(collection[i].folders);
}
Result in Robo 3T
But when i execute it inside my bash code with
mongo <myip>:27017 < db_parse.js
I get the error again
I achieve to create what i expect with these codes :
db_parse.sh
mongo <myip>:27017 < db_parse.js
db_parse.js
use <db_name>
var collection = db.DATA.find();
while(collection.hasNext()){
db.PARSED_DATA.insert(collection.next().folders);
}
I use this MongoDB manual.
Related
Surely, this is simple. I cannot figure this out. In Mongo shell I have the following command:
db.getCollection("CollectionName")
.findAndModify({query: {"Property.0.Element": {"$type" : 1}},
update: {$set: {"Property.0.Element":""}}
})
If I run this command several times, eventually it returns null and I know that I have changed all of the fields that I wanted to change. If however I run:
for(j = 0; j < 50;j++) {
var loc = "Property."+j+".Element";
db.getCollection("ShelbyCoAssessorDeepStaging")
.findAndModify({query: {loc : {"$type" : 1}},
update: {$set: {loc:""}}
})
}
Then I have null returned, but none of the values actually changed. Why is this? Note: I am running this in studio3T's intellishell against an atlas cluster at version 3.6.6.
You are trying to use dynamic keys for a query object. But JavaScript does not support this approach in object literals. Please consider the following example on the MongoDB shell or Studio 3T's IntelliShell:
> var fieldName = "foo"
> var query = { fieldName: 1 }
> printjsononeline(query)
{ "fieldName" : 1 }
In this small example, we create a variable that contains the name of the key that we want to set inside a query object. But JavaScript does not expect a variable as a key inside an object literal. The symbol is directly used as key name, it absolutely does not matter if the key is quoted or not.
If you want to have dynamic key names in JavaScript, you can use the simple bracket notation (or "member index access"):
> var fieldName = "foo"
> var query = {}
> query[fieldName] = 1
1
> printjsononeline(query)
{ "foo" : 1 }
If we apply this to your example, we get the following code to be run on the MongoDB shell or Studio 3T IntelliShell:
for(j = 0; j < 50;j++) {
var loc = "Property."+j+".Element";
var theQuery = {};
theQuery[loc] = {"$type" : 1};
var theUpdate = {$set: {}};
theUpdate["$set"][loc] = "";
db.getCollection("ShelbyCoAssessorDeepStaging")
.findAndModify({query: theQuery,
update: theUpdate
})
}
What is the fastest and safest strategy for adding a new field to over 100 million mongodb documents?
Background
Using mongodb 3.0 in a 3 node replica set
We are adding a new field (post_hour) that is based on data in another field (post_time) in the current document. The post_hour field is a truncated version of post_time to the hour.
I faced a similar scenario and in which I had create a script to update around 25 Million documents and it was taking a lot of time to update all the documents. To improve performance, I one by one inserted the updated document into a new collection and renamed the new collection.This approach helped because I was inserting the documents rather than updating them ('insert' operation is faster than 'update' operation).
Here is the sample script(I have not tested it):
/*This method returns postHour*/
function convertPostTimeToPostHour(postTime){
}
var totalCount = db.person.count();
var chunkSize = 1000;
var chunkCount = totalCount / chunkSize;
offset = 0;
for(index = 0; index<chunkCount; index++){
personList = db.persons.find().skip(offset).limit(chunkSize);
personList.forEach(function (person) {
newPerson = person;
newPerson.post_hour = convertPostTimeToPostHour(person.post_time);
db.personsNew.insert(newPerson); // This will insert the record in a new collection
});
offset += chunkSize;
}
When the above written script will get executed, the new collection 'personNew' will have the updated records with value of field 'post_hour' set.
If the existing collection is having any indexes, you need to recreate them in the new collection.
Once then indexes are created, you can rename the name of collection 'person' to 'personOld' and 'personNew' to 'person'.
The snapshot will allow to prevent duplicates in query result (as we are extending size) - can be removed if any trouble happen.
Please find mongo shell script below where 'a1' is collection name:
var documentLimit = 1000;
var docCount = db.a1.find({
post_hour : {
$exists : false
}
}).count();
var chunks = docCount / documentLimit;
for (var i = 0; i <= chunks; i++) {
db.a1.find({
post_hour : {
$exists : false
}
}).snapshot()
.limit(documentLimit)
.forEach(function (doc) {
doc.post_hour = 12; // put your transformation here
// db.a1.save(doc); // uncomment this line to save data
// you can also specify write concern here
printjson(doc); // comment this line to avoid polution of shell output
// this is just for test purposes
});
}
You can play with parameters, but as bulk is executed in 1000 records blocks, that looks optimal.
I have a mongodb collection like as follows . My job is to increase the "rate" of a particular document inside "ratings" key. I can do it by following command in mongo shell. db.amitava1.update({_id:1},{"$inc":{"ratings.0.rating":1 } } ) . . Here by 0 I access the first document in "ratings". But I need to use a variable in place of 0. The following does not work.
x = 0;
db.amitava1.update({_id:1},{"$inc":{"ratings.x.rating":1 } } );
Any help would be greatly appreciated. Thanks,
Try to do it with Template String, to parse x in the ratings.x.rating.
> var x = 0;
> var str = `ratings.${x}.rating`;
> db.amitava1.update({_id:1}, {$inc: {[str]: 1}})
What would be the equivalent of this line in groovy code when using the Mongo low level API?
db.countrycodes.findOne({"Country":"Antarctica"})
This line successfully finds the appropriate record for me in the Mongo shell but I tried many variations of it in my controller method and I keep getting NULL. Heres is my current attempt which is failing:
MongoClient mongoClient = new MongoClient("localhost", 27017)
DB db = mongoClient.getDB("twcdb");
DBCollection coll = db.getCollection('countrycodes')
println coll.find("Country":"Antarctica")
I know my collection and db is non NULL because when I do find() I do get a valid cursor back through which I can print the first record in collection. Here is the record I am trying to find:
{
"_id" : ObjectId("539848b2119918654e7e90b1"),
"Country" : "Antarctica",
"Alpha2" : "AQ",
"Aplha3" : "ATA",
"Numeric" : "10",
"FIPS" : "AY",
"IGA" : ""
}
Try this:
def cursor = coll.find()
def obj = cursor.next()
while (obj.Country != 'Antarctica') {
obj = cursor.next()
}
It is inefficient, you will have to traverse the whole collection everytime to find a record, but it will end up with 'obj' being the record you need.
Try below code and see if it works.
BasicDBObject query = new BasicDBObject("Country", "Antartica");
def cursor = coll.find(query)
try {
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
} finally {
cursor.close();
}
For more info look here: http://docs.mongodb.org/ecosystem/tutorial/getting-started-with-java-driver/
So I have a document that has references to foreign ObjectIDs that may point to other documents or collections.
For example this is the pseudo-structure of the document
{
_id: ObjectID(xxxxxxxx),
....
reference: ObjectID(yyyyyyyy)
}
I can't find anything that does not involve providing the collection and given that I don't know for sure on which collection to search, I am wondering if there is a way for me to find the document in the entire database and find the collection ObjectID(yyyyyyyy) belongs to.
The only possible way to do this is by listing every collection in the database and performing a db.collection.find() on each one.
E.g. in the Mongo shell I would do something like
var result = new Array();
var collections = db.getCollectionNames();
for (var i = 0; i < collections.length; i++) {
var found = db.getCollection(collections[i]).findOne({ "_id" : ObjectId("yyyyyyyy") });
if (found) {
result.push(found);
}
}
print(result);
You need to run your query on all collections in your database.
db.getCollectionNames().forEach(function(collection){
db[collection].find({ $or : [
{ _id : ObjectId("535372b537e6210c53005ee5") },
{ reference : ObjectId("535372b537e6210c53005ee5")}]
}).forEach(printjson);
});