Imagine I have a document structure like this.
{
_id: ObjectId('internalId'),
externalId: 'externalId',
history: [
{
effective: ISODate('2000-02-01T00:00:00.000Z'),
property: 'new value'
},
{
effective: ISODate('2000-01-01T00:00:00.000Z'),
property: 'value'
}
]
}
Each time this document is read, all of the properties are merged together in historical order into a final state, possibly stopping at a specific point in time.
To add a new history item, I would need to perform something like this.
{
$push: {
history: {
property: 'even newer value',
effective: new Date()
}
},
$setOnInsert: {
externalId: externalId
}
}
I would like to find a way to make sure that an update that does not modify the actual merged history state is never stored. However it seems like this would require a separate read operation, and thus an (external) pessimistic lock to be held, while it was determined if a revision could proceed.
This feels like an incorrect design. Help!
Related
My issue is rather specific so I'll try to explain my setup first.
I have a collection called clients, which is a master list of all clients. The model for it is:
{
id: String,
organizationId: Number,
networkId: String,
deviceSerial: String,
}
(irrelevant properties removed)
I also have a collection called clienttransactions, which is a list of when clients have gone online or offline. So each time a client comes online, it adds a record saying it came online (online: true), and vice-versa for when a client goes offline (online: false). The model for that looks like this:
{
clientId: String,
deviceSerial: String,
networkId: String,
organizationId: Number,
ts: Number,
online: Boolean
}
ts is a unix timestamp in seconds. Also if you're wondering why I need all those foreign keys on each record, it's because of the way the API where I get this data from works. So just ignore that.
issue:
Given a deviceSerial, networkId, and organizationId, I want to find all clients that were online at any point between a given time frame (given a start time and end time in epoch seconds).
Possible edge case: There could be times when a client came online before the given start time, and stayed online until after the given end time. In this case, there will be no transaction record within the time frame, but the client should still be seen as online.
Accounting for this case is what I'm having the most trouble with, since I can't simply just search for online transactions between the time frame. If there are no transactions for a client in the time frame, then I need to search outside the time frame to see if the last transaction made before the start time for that client was an online one.
I'm not super well-versed on the aggregation pipeline yet, so this is as far as I got:
const startTime = 1550601742;
const endTime = 1550599341;
ClientTransaction.aggregation([
{
$match: {
organizationId: 600381,
networkId: 'N_651896046061895525',
deviceSerial: 'Q2MN-3CUN-6GQM',
ts: {$lt: endTime}
}
},
{
$group: {
_id: '$clientId',
lastStatus: {
$max: '$ts'
},
online: {
$last: '$online'
}
}
}
]);
I think I'm halfway there with this. It finds all transactions for unique clients before the end time, but stops before process of checking if the client was actually online during the time frame.
You are looking for all clients whose latest activity is an online activity before start time or has online/offline activity between start and end time.
So something like should work
ClientTransaction.aggregation([
{ $match: {
organizationId: 600381,
networkId: 'N_651896046061895525',
deviceSerial: 'Q2MN-3CUN-6GQM',
ts: {$lte: endTime}
}
},
{ $sort:{"clentId":1, "ts":-1 } },
{ $group: {
_id: '$clientId',
latest: {
$first: '$$ROOT'
}
}},
{ $match:{
$or:[
{"latest.online":true,"latest.ts":{$lt:startTime}},
{"latest.ts":{$gte:startTime, $lte:endTime}}
]
}}
]);
I have an object in my document like this:
revisions: {
<revisionId>: {
...
someProperty: { ... },
},
...
}
I would like to remove someProperty from every nested object in revisions object (i.e. iterate through all the object revisionId keys). I'm asking because if it's not possible I'd rather convert this to an array than do it on the server and possibly overwrite any updates in the mean time.
I tried $unset but I'm only aware of the $[] operator which only works for arrays e.g.
$unset: {
'revisions.$[].someProperty': 1
}
Thanks
You're almost there:
{ $unset: { someProperty: "" } }
The value doesn't matter from memory
You could then use a cursor to iterate through each doc and remove the unwanted property.
db.collection.find().forEach(<function>)
EDIT: Sorry, realising it's a nested model with arbitrary key for the top level property, makes it more tricky.
t.forEach(function( row) {
var newRevisions = [];
row.revisions.fields.forEach( function( revision ){
delete revision.someProperty;
newRevisions.push(revision);
} )
t.update(
{ _id: row._id },
{ "$set": { "revisions": newRevisions} }
);
});
Huge caveat, totally untested, but should give you starting point.
For each row
For each revision in revisions object
Remove the someProperty property
Set the revisions property back the collection based on _id
I have a users in MongoDB and each user has an interface allowing them to set their current state of hunger being a combination of "hungry", "not hungry", "famished", "starving", or "full"
Each user can enter a multiple options for any period of time. For example, one use case would be "in the morning, record how my hunger is" and the user can put "not hungry" and "full". They can record how their hunger is at any time in the day, and as many times as they want.
Should I store the data as single entries, and then group the data by a date in MongoDB later on when I need to show it in a UI? Or should I store the data as an array of the options the user selected along with a date?
It depends on your future queries, and you may want to do both. Disk space is cheaper than processing, and it's always best to double your disk space than double your queries.
If you're only going to map by date then you'll want to group all users/states by date. If you're only going to map by user then you'll want to group all dates/states by user. If you're going to query by both, you should just make two Collections to minimize processing. Definitely use an array for the hunger state in either case.
Example structure for date grouping:
{ date: '1494288000',
time-of-day: [
{ am: [
{ user: asdfas, hunger-state: [hungry, full] },
{ user: juhags, hunger-state: [full] }
],
pm: [
{ user: asdfas, hunger-state: [hungry, full] },
{ user: juhags, hunger-state: [full] }
]}]}
It depends on how you are going to access it. If you want to report on a user's last known state, then the array might be better:
{
user_id: '5358e4249611f4a65e3068ab',
timestamp: '2017-05-08T17:30:00.000Z',
hunger: ['HUNGRY','FAMISHED'],
}
The timestamps of multiple records might not align perfectly if you are passing in the output from new Date() (note the second record is 99 ms later):
{
user_id: '5358e4249611f4a65e3068ab',
timestamp: '2017-05-08T17:30:00.000Z',
hunger: 'HUNGRY',
}
{
user_id: '5358e4249611f4a65e3068ab',
timestamp: '2017-05-08T17:30:00.099Z',
hunger: ['FAMISHED',
}
You should probably look at your data model though and try to get a more deterministic state model. Maybe:
{
user_id: '5358e4249611f4a65e3068ab',
timestamp: '2017-05-08T17:30:00.000Z',
isHungry: true,
hunger: 'FAMISHED',
}
I have a model that is scattered all around the application. I have a redux state tree:
{
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
I need to keep a reference to mongoDb __v in my state too. Where is the best place to place it?
I was thinking about a separate branch model_metadata that would keep the metadata about docs (_id, __v, ...).
{
model_metadata: { <------------------------ HERE
model: {
_id: id,
__v: 2
}
}
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
Is it a valid approach or would you recommend a different one?
Every reducer only can access its own part of state, so when you do
combineReducers({
one,
another
});
and access state in one, it is equivalent to doing store.getState().one, and the same for another. So, you need to split the data in page property of state into two parts: actual data and metadata. Just like the object you retrieve from Mongo.
The point in having metadata and actual data being processed by the same reducer is that every time a reducer function is performed, you have everything you need about your object in state argument of that function. Splitting the data into two different reducers would make things way more complicated.
So, the new data representation in page would look like
{
model_metadata: { <------------------------ HERE
model: {
_id: id,
__v: 2
}
}
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
while connecting to page would look like
connect(state => ({
page: state.page
})(...)
I'm trying to come up with a way to update the values in an array of objects in mongo. I have a collection which looks like
[
{ CourseName: '',
Sessions: [
{
_id: null, //oops I didn't set this in the import
Name: 'blah',
Location: 'moon'
}]
}
]
Now I need to set the _id field. I tried the documented approach of doing
db.Course.update({'Sessions._id': null}, {$set:{'Sessions.$._id': ObjectId()}}, false, true)
But I ran into this bug http://jira.mongodb.org/browse/SERVER-1055 which meant that I couldn't do it. Is there some syntax which will allow me just to itterate over the collection and update each record by hand? I tried a few things like
db.Course.find().forEach(
function(course)
{
course.Sessions.forEach(function(session)
{
session._id=ObjectId();
course.Save(session); //Don't know how to save a single object
});
});
but they didn't work. I'm looking for some way to just update that value in each session.
I think what you want is:
db.Course.find().forEach(
function(course)
{
course.Sessions.forEach(function(session)
{
session._id=ObjectId();
});
db.Course.save(course);
});
However, you can run into problems saving stuff into a collection you're in the middle of iterating over, so I'd suggest loading a bunch of documents into an array, processing them, loading another batch, etc.