Auto populate date in MongoDB on insert - mongodb

MongoDB provides a way to update a date field by the system on update operations: https://docs.mongodb.com/manual/reference/operator/update/currentDate/. Is there any equivalent to this for insert operations?

You may try to do a few things if you do not want to handle this from code (I have executed the code below directly on mongo shell):
If you want to use $currentDate use update with upsert = true:
db.orders.update(
{"_id":ObjectId()},
{
$currentDate: {
createtime: true
}
},
{ upsert: true }
)
It will now generate the objectid on app server instead of date/time (unless you use raw command).
Use new timestamp or date object directly:
db.orders.insert(
"createtime": new Timestamp()
)
The problem with most driver will be then to make sure the new object is created on mondodb server- not on the machine where the code is running. You driver hopefully allows to run raw insert command.
Both will serve the purpose of avoiding time differences/ time sync issue between application server machines.

The $currentDate is an update operator which populates date field with current date through update operation.
To auto populate date field while insertion of new MongoDB document,please try executing following code snippet
var current_date=new Date();
db.collection.insert({datefield:current_date})
In above code snippet the statement
new Date()
creates a new JavaScript Date object which consists of a year, a month, a day, an hour, a minute, a second, and milliseconds

If you want to populate this value when running it in the server side, and are concerned about it being passed by the client, you can add properties to the data object that is being used in the insert statement only when it will be saved. This way, you can guarantee that it will be added every time with the server's date, not the client's:
Client side
...
let data = { info1: 'value1', info2: 'value2'}
someApi.addInfo(data);
...
Server side
function addInfo(data){
...
data['creationDate'] = new Date();
db.collection.insertOne(data);
...
}
Result will be:
{
info1: 'value1',
info2: 'value2',
creationDate: ISODate("2018-09-15T21:42:13.815Z")
}
If you are passing multiple values for insertion (using insertMany) you have to loop over the items and add this property for all of them.
You can also use this approach when updating documents if for some reason you can't use the $currentDate operator, just be sure you are not replacing any existing properties in the data passed to mongodb.

Since mongo 3.6 you can use 'change stream':
https://emptysqua.re/blog/driver-features-for-mongodb-3-6/#change-streams
to use it you need to create a change stream object by the 'watch' query:
def update_ts_by(change):
update_fields = change["updateDescription"]["updatedFields"].keys()
print("update_fields: {}".format(update_fields))
collection = change["ns"]["coll"]
db = change["ns"]["db"]
key = change["documentKey"]
if len(update_fields) == 1 and "update_ts" in update_fields:
pass
else:
client[db][collection].update(key, {"$set": {"update_ts": datetime.now()}})
client = MongoClient("172.17.0.2")
db = client["Data"]
change_stream = db.watch()
for change in change_stream:
print(change)
update_ts_by(change)
Note, to use the change_stream object, your mongodb instance should run as 'replica set'.
It can be done also as a 1-node replica set (almost no change then the standalone use):
Mongo DB - difference between standalone & 1-node replica set

Related

Cloudant View Unusual Conditional Behavior with Dates

Found some peculiar behavior was wondering if anyone could help me to understand it so to avoid similar issues in the future.
Creating a cloudant view I want to return only records with a timestamp of the current day.
I was having a hard time getting it to work and found the difference is in having a space before the end of the if condition.
See below for working and not working
if (new Date(Date.parse(doc.details.timestamp)).setHours(0,0,0,0) === new Date().setHours(0,0,0,0) ){
Works to check the current date against the Cloudant doc date
if (new Date(Date.parse(doc.details.timestamp)).setHours(0,0,0,0) === new Date().setHours(0,0,0,0)){
Does not work to check the date against the Cloudant doc date
Full working view below for context
function (doc) {
if (doc.details.location){
if (new Date(Date.parse(doc.details.timestamp)).setHours(0,0,0,0) === new Date().setHours(0,0,0,0) ){
emit(doc.details.location.toLowerCase(), { "location": doc.details.location.toLowerCase(), "status": doc.details.status, "user": doc.details.username, "time": doc.details.timestamp})
}
}
}
All the best,
Scott.
I suspect this may be to do with the execution date/time of the if statement rather than the space involved.
The map function needs to be deterministic regardless of execution time.
It looks like you are guessing that the map function is running at query time (so new Date would emit today's date). Instead, it runs at indexing time, so the value of new Date is whatever the datetime is when the indexing happens. As indexing runs at a different time to document insertion (sometime between insert and when the view is queried), using any form of value that changes with time will produce unpredictable results.
I suspect therefore the space is co-incidental and instead the output of new Date is changing, and so altering what's emitted into your view.
For your problem -- querying for things "today" -- I think you want to instead emit a key like [location, year, month, day]. Your map function would look like:
function (doc) {
if (doc.details.location) {
var l = doc.details.location.toLowerCase();
var d = Date.parse(doc.details.timestamp);
emit([l, d.getFullYear(), d.getMonth(), d.getDate()], {
"location": l,
"status": doc.details.status,
"user": doc.details.username,
"time": doc.details.timestamp
});
}
}
As JavaScript uses 0-based indexes for the month, to query for all items at location Bristol on today, 2-Feb-2017, you'd use key=["bristol",2017,1,2] to in your query to the view.

How to set a field value only when document modified?

I'm updating many documents using a bulk operation, but I only want to bump the timestamp of documents that are changed by the new values.
Currently my bulk operation looks something like this:
var updates = db.collection.initializeUnorderedBulkOp();
// ...
updates.find( someQuery ).update( {
$set: someValues,
$currentDate: { modified:true }
} );
updates.execute();
Even if someValues doesn't modify any fields, the modified field gets set. Is there a way to provide a list of additional updates to be performed only when the original update results in a change?
I'm running MongoDB 2.6
I want to avoid negating all the object values in the query one by one.

spring data mongo - mongotemplate count with query hint

The mongo docs specify that you can specify a query hint for count queries using the following syntax:
db.orders.find(
{ ord_dt: { $gt: new Date('01/01/2012') }, status: "D" }
).hint( { status: 1 } ).count()
Can you do this using the mongo template? I have a Query object and am calling the withHint method. I then call mongoTemplate.count(query); However, I'm pretty sure it's not using the hint, though I'm not positive.
Sure, there are a few forms of this including going down to the basic driver, but assuming using your defined classes you can do:
Date date = new DateTime(2012,1,1,0,0).toDate();
Query query = new Query();
query.addCriteria(Criteria.where("ord_dt").gte(date));
query.addCriteria(Criteria.where("status").is("D"));
query.withHint("status_1");
long count = mongoOperation.count(query, Class);
So you basically build up a Query object and use that object passed to your operation, which is .count() in this case.
The "hint" here is the name of the index as a "string" name of the index to use on the collection. Probably something like "status_1" by default, but whatever the actual name is given.

Mongo - exclude entries in one collection from another find()

I have a local collection that logs when a user has viewed an entry. It stores the ID of the entry and the time it was viewed:
viewedDate = new Date();
notifications.insert({
'viewed': this.data._id,
'viewedDate': viewedDate
});
I want to exclude any of the 'viewed' ids in this collection from another find() (I basically want to count how many entries haven't been viewed)
How can I use the results of a notifications.find() to exclude results from another find()? If I assign the notifications.find() to a variable, it returns all kinds of stuff as an object.
edit
OK, so if I use fetch() I can restrict what comes back - can I do something with this in a find()?
myNotes = notifications.find({}, {fields: {'viewedDate' :0, _id:0}}).fetch();
This returns
[
Object
viewed: "HqYcCma3qKseHALyv"
__proto__: Object
]
Thanks to some invaluable help from garilla_ in the Meteor IRC, I got it working, solution as follows:
myViewedOffers = notifications.find({},{fields: {'viewed':1}}).fetch();
myViewedArray = myViewedOffers.map(function(viewed){return viewed.viewed});
offerCount = Offers.find({_id: {$nin: myViewedArray}}).count();

MongoDb timestamp

i have created and want to now import a dummy collection. one of the fields in each item are "created" and "updated" fields. what can i put in the source/json file so that MongoDb will use the current date and time as the value on import?
this wont work
"created" : Date()
mongoimport is intended for importing data existing data in CSV, TSV, or JSON format. If you want to insert new fields (such as a created timestamp) you will have to set a value for them.
For example, if you want to set the created timestamp to the current time, you could get a unix timestamp from the command line (which will be seconds since the epoch):
$ date +%s
1349960286
The JSON <date> representation that mongoimport expects is a 64-bit signed integer representing milliseconds since the epoch. You'll need to multiply the unixtime seconds value by 1000 and include in your JSON file:
{ "created": Date(1349960286000) }
An alternative approach would be to add the created timestamps to documents after they have been inserted.
For example:
db.mycoll.update(
{created: { $exists : false }}, // Query criteria
{ $set : { created: new Date() }}, // Add 'created' timestamp
false, // upsert
true // update all matching documents
)
As Stennie correctly pointed out, you can not do this with just mongoimport or mongorestore: they are just for restoring your previously dumped data. Correct way of doing this is to restore the data and then to make update on the restored data.
With a new mongo 2.6 you can do this easily using $currentDate operation, which was created to update time to a current timestamp.
In your case you need something like
db.users.update(
{},
{
$currentDate: {
created: true,
updated: true
},
}
)