I have a collection in mongo [4.0.9] which stores document like below
{ "_id" : 187489726, "mykey" : { "data" : [ { "id" : 2, "value" : "No" } ], "timestamp" : "2020-06-03 10:40:52.718" } }
If I fire queries like so, would it have performance impact because timestamp is string in the collection.
db.mycollection.find({"mykey.timestamp" : {$get : "2020-06-05 10:00:10.269"}},{"mykey": 1}).count()
I would have close 20 million records in DB.
Could not find any benchmark or documentation such a scenario. I can still convert all in ISO format.
Related
So this is my data in mongodb
{
"_id" : ObjectId("60f67dc955784b233692a0f2"),
"Id" : "9153",
"InfoList" : [
{
"itemId" : "42342",
"price" : 1009.0,
"date" : ISODate("2021-01-01T08:30:36.131Z")
}
]
}
{
"_id" : ObjectId("6105668a55784bd00ef3ebc6"),
"Id" : "894249",
"InfoList" : [
{
"itemId" : "42342",
"price" : 23.0,
"date" : ISODate("2021-01-01T08:30:36.131Z")
},
{
"itemId" : "3221",
"price" : 44554.0,
"date" : ISODate("2013-07-31T15:05:10.042Z")
}
]
}
I want to find all the items in InfoList for all the documents whose date is less than 2021-02-09 and then delete them.
This is the code that I am using
Query query = new Query();
query.addCriteria(Criteria.where("InfoList")
.elemMatch(Criteria
.where("date")
.lte(date)));
return mongoTemplate.findAllAndRemove(query,ProductInfo.class, CollectionName);
But this code is neither finding the documents which have date < 2021-02-01 nor deleting them. Any suggestions regarding what might be wrong here ?
I'm not familiar with Mongo query I know mongoose more
But do it like this 1-get all the of the object
It will return an array
On this array
2-do an if to check if the date is less than 2021-02-09 if yes remove it using its id
I have a MongoDB collection "Events" with 1 million documents similar to:
{
"_id" : 32423,
"testid" : 43212,
"description" : "fdskfhsdj kfsdjfhskdjf hksdjfhsd kjfs",
"status" : "error",
"datetime" : ISODate("2018-12-04T15:55:00.000Z"),
"failure" : 0,
}
Considering the documents were sorted based on datetime field (ascending), I want to check them in the chronical order one by one and pick only the records where the "failure" field was 0 in the previous document and it is 1 in the current document. I want to skip other records in between.
For example, if I also have the following records:
{
"_id" : 32424,
....
"datetime" : ISODate("2018-12-04T16:55:00.000Z"),
"failure" : 0,
}
,
{
"_id" : 32425,
....
"datetime" : ISODate("2018-12-04T17:55:00.000Z"),
"failure" : 1,
}
,
{
"_id" : 32426,
....
"datetime" : ISODate("2018-12-04T18:55:00.000Z"),
"failure" : 0,
}
I only want to collect the one with "_id:32425", and repeat the same policy for the following cases.
Of course, if I extract all the data at once, then I can process it using Python for instance. But, extracting all the records would be really time-consuming (1 million documents!).
Is there a way to do the above via MongoDB commands?
I have MongoDB Collection where some documents have arrays of objects. One of the fields of this objects is timestamp.
The problem is that historically some of timestamp values are Strings (e.g. '2018-02-25T13:33:56.675000') or Date and some of them are Double (e.g. 1528108521726.26).
I have to convert all of them to Double.
I've built the query to get all the documents with the problematic type:
db.getCollection('Cases').find({sent_messages: {$elemMatch:{timestamp: {$type:[2, 9]}}}})
And I also know how to convert Date-string to double using JS:
new Date("2018-02-18T06:39:20.797Z").getTime()
> 1518935960797
But I can't build the proper query to perform the update.
Here is an example of such a document:
{
"_id" : ObjectId("6c88f656532aab00050dc023"),
"created_at" : ISODate("2018-05-18T03:43:18.986Z"),
"updated_at" : ISODate("2018-05-18T06:39:20.798Z"),
"sent_messages" : [
{
"timestamp" : ISODate("2018-02-18T06:39:20.797Z"),
"text" : "Hey",
"sender" : "me"
}
],
"status" : 1
}
After the update it should be:
{
"_id" : ObjectId("6c88f656532aab00050dc023"),
"created_at" : ISODate("2018-05-18T03:43:18.986Z"),
"updated_at" : ISODate("2018-05-18T06:39:20.798Z"),
"sent_messages" : [
{
"timestamp" : 1518935960797.00,
"text" : "Hey",
"sender" : "me"
}
],
"status" : 1
}
As per your question, you are trying to fetch the record first.
db.getCollection('Cases').find({sent_messages: {$elemMatch:{timestamp: {$type:[2, 9]}}}})
Then convert date in JS:
new Date("2018-02-18T06:39:20.797Z").getTime()
And then this is an update query:
db.getCollection('Cases').updateOne({_id:ObjectId("6c88f656532aab00050dc023")}, { $set: { "sent_messages.$.timestamp" : "218392712937.0" }})
And if you want to update all records then you should write some forEach mechanism. I think you have already this implemented.
Hope this may help you.
Finally I just do it with JS code that can be run in mongo console:
db.getCollection('Cases').find({sent_messages: {$elemMatch:{timestamp: {$type:[2, 9]}}}}).forEach(function(doc) {
print('=================');
print(JSON.stringify(doc));
doc.sent_messages.forEach(function(msg){
var dbl = new Date(msg.timestamp).getTime();
print(dbl);
msg.timestamp = dbl;
});
print(JSON.stringify(doc))
db.Cases.save(doc);
} )
Thanks all for your help!
I have to migrate from MySQL to MongoDB , and i beginner in MongoDB, what is the best way to storing below data in MongoDB ?
should i create a document for each row?
should i save all row in a one document?
Which one is valid way in MongoDB?
{
"_id" : ObjectId("5659d56fef6c702fbc45cc1b")
"key" : "setting_update_id"
"value" : "1"
"extra" :
[
//some data
]
}
OR
{
"_id" : ObjectId("5659d56fef6c702fbc45cc1b")
"setting_update_id" : "1"
"extra" :
[
//some data
]
}
Ali,
As a beginner you would want to read the docs here. Each collection can be thought of roughly as a table in a relational database. And each document can be thought of as a row in the database. So each column of your table would be the keys of your document.
I would design it closer to the first one.
{
"_id" : ObjectId("5659d56fef6c702fbc45cc1b")
"key" : "setting_update_id"
"value" : "1"
"params" :
{
"extra" : "hello",
"foo" : "bar"
}
}
I have a document and I need to query mongodb database to return me all the documents which was inserted after current document.
Is it possible and how to do that query?
If you do not override the default _id field you can use that objectID (see the mongodb docs) to make a comparison by time. For instance, the following query will find all the documents that are inserted after curDoc has been inserted (assuming none overwrite the _id field):
>db.test.find({ _id : {$gt : curDoc._id}})
Note that these timestamps are not super granular, if you would like a finer grained view of the time that documents are inserted I encourage you to add your own timestamp field to the documents you are inserting and use that field to make such queries.
If you are using Insert time stamp as on of the parameter, you can query like below
> db.foo.find()
{ "_id" : ObjectId("514bf8bbbe11e483111af213"), "Name" : "abc", "Insert_time" : ISODate("2013-03-22T06:22:51.422Z") }
{ "_id" : ObjectId("514bf8c5be11e483111af214"), "Name" : "xyz", "Insert_time" : ISODate("2013-03-22T06:23:01.310Z") }
{ "_id" : ObjectId("514bf8cebe11e483111af215"), "Name" : "pqr", "Insert_time" : ISODate("2013-03-22T06:23:10.006Z") }
{ "_id" : ObjectId("514bf8eabe11e483111af216"), "Name" : "ijk", "Insert_time" : ISODate("2013-03-22T06:23:38.410Z") }
>
Here my Insert_time corresponds to the document inserted time, and following query will give you the documents after a particular Insert_time,
> db.foo.find({Insert_time:{$gt:ISODate("2013-03-22T06:22:51.422Z")}})
{ "_id" : ObjectId("514bf8c5be11e483111af214"), "Name" : "xyz", "Insert_time" : ISODate("2013-03-22T06:23:01.310Z") }
{ "_id" : ObjectId("514bf8cebe11e483111af215"), "Name" : "pqr", "Insert_time" : ISODate("2013-03-22T06:23:10.006Z") }
{ "_id" : ObjectId("514bf8eabe11e483111af216"), "Name" : "ijk", "Insert_time" : ISODate("2013-03-22T06:23:38.410Z") }
>