I want to store/update statistics for each year. I have the following model
{
entityid: ObjectId,
stats: [
{ year: 2018, value: 25 }
]
}
(This model is a bit simplified, in reality the year has also an array with months -> days -> hours. But the problem stays the same for the simplified model)
For updating I can simply use $inc like
db.statistics.updateOne(
{entityid, 'stats.year': 2018},
{$inc: { 'stats.$.year': 1}}
)
But now a problem arises when a new year begins because there will be no { year: 2019, value: 0 } inside the stats array. Upsert can not really be used because of the positional operator $.
The current solution is to check the result of the update query above if we actually modified a document. If no changes were applied we execute a push to insert the array element for the new year and execute the update again.
The solution feels like a hack and produces some problem with race conditions where multiple objects are pushed for the same year, although this can be fixed easily.
Can the update/push operation be performed in one go? Or is there a better database model to store this information?
You can either follow your hack or make database like this and use upsert on the year key while using $inc on value
{
entityid: ObjectId,
year: 2018,
value: 25
}
and use $group on entityid while fetching data if you want to group data.
Related
I want to create a partial index in MongoDB which excludes documents that are older than 30 days in the past from the current date.
Something like this (pseudo code):
partialFilterExpression: { date: { $gte: { $currentDate - 30 days }} }
The $currentDate should be the actual current date, in other words the date is a dynamic value, not a static value.
It this possible?
As far as I know, dynamic values in the filter expression are not currently supported with partial indexes & the filter expression only supports a subset of regular mongo query operators (and that subset does not include the $date operator).
One way to approximate your desired behavior is by creating a shouldIndex boolean field on your documents, have {shouldIndex: true} be your filter expression, and have a script update that field once a day for documents that are older than current_date - 30.
Yes that's possible,
you can use normal createIndex operation with partialFilterExpression which contains you condition.
I think code in you quenstion will only work
You can use
let today = new Date(); //your date condition
db.collection_name.createIndex({ field1 : type1, field2: type2},
partialFilterExpression: { date: { $gte: { today }} })
Read more about parital indexes here,
however it comes with some restrictions too.
This should be very simple. I have two collections, one of which holds two types of data (name, age), and the other should simply add the age values to an array (with no duplicates).
I "start" my collections like usual:
People = new Mongo.Collection('people')
Ages = new Mongo.Collection('ages')
Right now I'm working with seed data, but the question could easily extend to when I actually want to dynamically add data to the array. I seed it like so:
Meteor.startup(function() {
if (People.find().count() === 0) {
[
{
name: 'John',
age: '24' //Yes, I want to store it as strings.
},
{ ... } //more data
]
.forEach(function(person) {
People.insert(person)
Ages.update({ $addToSet: {age: person.age}}) //Not working
})
}
})
That last part there is what's not working. I guess I figured $addToSet would fix things for me, since the docs say:
If the field is absent in the document to update, $addToSet creates
the array field with the specified value as its element.
Now I suppose I have to create the field first, but I'm not sure where or how. I have a strong, strong feeling that I'm overlooking something ridiculously simple here...
If I got it right, your db should look like that when filled
Persons (_id, name, age)
1, John, 24
2, Pete, 21
3, Michele, 27
4, Sandy, 21
Ages (_id, ageset)
?, [ 24, 21, 27 ]
Solution1: Just insert one record on a fix key and then only update this one.
Have a look at this MeteorPad
Solution2: Using a local Meteor.Collection which is synced by server an gets DISTINCT field values from package mrt:mongodb-aggregation.
Have a look at this MeteorPad
Solution3: Using a server side synced Mongo.Collection to hold the distinct ages list.
Have a look at this MeteorPad
Remark: Checkout log infos on server process. There are timeouts to add, change and remove a record for test and updates (5 sec, 10 sec, 15 sec)
Now right now, I see that you're defining your People collection, but I don't see you actually defining "person" or "Age" anywhere. Maybe thats just due to how you've formatted your answer.
Either way though, I'm not entirely sure you'd be getting anything to happen. As far as I know, you'll need to select the documents each time through the loop, as you want to update them.
This is how I'm doing something similar in an app I'm working on:
Meteor.users.update({ _id: Meteor.userId() }, { $addToSet: { 'profile.viewedRequests' : this._id }});
The key there being that I'm selecting an individual document, before attempting to update it.
Its either that, or you need to switch to People.update.
I have two collections - shoppers (everyone in shop on a given day) and beach-goers (everyone on beach on a given day). There are entries for each day, and person can be on a beach, or shopping or doing both, or doing neither on any day. I want to now do query - all shoppers in last 7 days who did not go to beach.
I am new to Mongo, so it might be that my schema design is not appropriate for nosql DBs. I saw similar questions around join and in most cases it was suggested to denormalize. So one solution, I could think of is to create collection - activity, index on date, embed actions of user. So something like
{
user_id
date
actions {
[action_type, ..]
}
}
Insertion now becomes costly, as now I will have to query before insert.
A few of suggestions.
Figure out all the queries you'll be running, and all the types of data you will need to store. For example, do you expect to add activities in the future or will beach and shop be all?
Consider how many writes vs. reads you will have and which has to be faster.
Determine how your documents will grow over time to make sure your schema is scalable in the long term.
Here is one possible approach, if you will only have these two activities ever. One record per user per day.
{ user: "user1",
date: "2012-12-01",
shopped: 0,
beached: 1
}
Now your query becomes even simpler, whether you have two or ten activities.
When new activity comes in you always have to update the correct record based on it.
If you were thinking you could just append a record to your collection indicating user, date, activity then your inserts are much faster but your queries now have to do a LOT of work querying for both users, dates and activities.
With proposed schema, here is the insert/update statement:
db.coll.update({"user":"username", "date": "somedate"}, {"shopped":{$inc:1}}, true)
What that's saying is: "for username on somedate increment their shopped attribute by 1 and create it if it doesn't exist aka "upsert" (that's the last 'true' argument).
Here is the query for all users on a particular day who did activity1 more than once but didn't do any of activity2.
db.coll.find({"date":"somedate","shopped":0,"danced":{$gt:1}})
Be wary of picking a schema where a single document can have continuous and unbounded growth.
For example, storing everything in a users collection where the array of dates and activities keeps growing will run into this problem. See the highlighted section here for explanation of this - and keep in mind that large documents will keep getting into your working data set and if they are huge and have a lot of useless (old) data in them, that will hurt the performance of your application, as will fragmentation of data on disk.
Remember, you don't have to put all the data into a single collection. It may be best to have a users collection with a fixed set of attributes of that user where you track how many friends they have or other semi-stable information about them and also have a user_activity collection where you add records for each day per user what activities they did. The amount or normalizing or denormalizing of your data is very tightly coupled to the types of queries you will be running on it, which is why figure out what those are is the first suggestion I made.
Insertion now becomes costly, as now I will have to query before insert.
Keep in mind that even with RDBMS, insertion can be (relatively) costly when there are indices in place on the table (ie, usually). I don't think using embedded documents in Mongo is much different in this respect.
For the query, as Asya Kamsky suggest you can use the $nin operator to find everyone who didn't go to the beach. Eg:
db.people.find({
actions: { $nin: ["beach"] }
});
Using embedded documents probably isn't the best approach in this case though. I think the best would be to have a "flat" activities collection with documents like this:
{
user_id
date
action
}
Then you could run a query like this:
var start = new Date(2012, 6, 3);
var end = new Date(2012, 5, 27);
db.activities.find({
date: {$gte: start, $lt: end },
action: { $in: ["beach", "shopping" ] }
});
The last step would be on your client driver, to find user ids where records exist for "shopping", but not for "beach" activities.
One possible structure is to use an embedded array of documents (a users collection):
{
user_id: 1234,
actions: [
{ action_type: "beach", date: "6/1/2012" },
{ action_type: "shopping", date: "6/2/2012" }
]
},
{ another user }
Then you can do a query like this, using $elemMatch to find users matching certain criteria (in this case, people who went shopping in the last three days:
var start = new Date(2012, 6, 1);
db.people.find( {
actions : {
$elemMatch : {
action_type : { $in: ["shopping"] },
date : { $gt : start }
}
}
});
Expanding on this, you can use the $and operator to find all people went shopping, but did not go to the beach in the past three days:
var start = new Date(2012, 6, 1);
db.people.find( {
$and: [
actions : {
$elemMatch : {
action_type : { $in: ["shopping"] },
date : { $gt : start }
}
},
actions : {
$not: {
$elemMatch : {
action_type : { $in: ["beach"] },
date : { $gt : start }
}
}
}
]
});
i need to know if is possible to have a list of objects, where the objects are uniques by day.
I have a collection with this format:
{
domain: "google.com"
counters: [
{ day: "2011-08-03", metric1: 10, metric_2: 15 }
{ day: "2011-08-04", metric1: 08, metric_2: 07 }
{ day: "2011-08-05", metric1: 20, metric_2: 150 }
]
}
I tried something like that:
db.test.ensureIndex({ domain: 1, 'counters.day': 1 }, { unique: true }).
with upsert and $push, but this not works.
Then I tried with upsert and $addToSet. but i can't set the unique fields.
I need to push a new counter, if the day exists, it should be replaced.
Unique indexes working only for the root document, but not for the embedded. So that's mean that you can't insert two documents with same domain and counters.day. But you can insert into embedded counters duplicated rows.
I need to push a new counter, if the day exists, it should be
replaced.
When you trying to insert new embedded document you should check if document with such day exists and in case if it exists make an update, otherwise insert.
I'm trying to create a registration form with mongoose and MongoDB. I have a unique key, UserId, and every time I create a new entry I would like to take the greatest UserId in the database and increase it by one.
I tried with db.user.find({}).sort({userId: 1}); but it seems not to work.
Thanks
Masiar
What you want to do sounds more like a Schema for Relational Databases with an Auto Increment. I would recommend another solution.
At first you already have a unique id. It get automatically created and are in "_id" field. For me it seems you want to have a UserID for building relation, but you already ca use the value in _id.
The other thing why you want incremented ids could be that you create a webapplication and propably want "nicer" urls? For example. /user/1 instead of /user/abc48df...?
If that is the case i would prefer to create a unique constraint on a username. And instead of an id you use you username in the url "/user/john".
With this your urls are much nicer. And for building relation you can use _id. And you don't run into problems with fethcing the highest number first.
To create a unique index:
db.collection.ensureIndex({username: 1}, {unique: true})
You can do this to get the user with the current highest UserId:
db.user.insert( { UserId: 1 } )
db.user.insert( { UserId: 2 } )
db.user.insert( { UserId: 3 } )
db.user.find().sort( { UserId: -1 } ).limit(1)
It's worth noting that there isn't a way in MongoDB to fetch this value and insert a new user in a single atomic transaction, it only supports atomic operations on single documents. You'd need to take care that another operation didn't insert another user at the same time, you could end up with two users with the same UserId.
To iterate over the cursor and get put the returned doc in an array:
var myArray = [];
User.find().sort('UserId','descending').limit(1).each(function(err, doc) {
myArray.push(doc);
});