MongoDB: Creating a calculated field measuring the difference between date fields - mongodb

Context
This question uses the same collection/ document schema construct from my other question: MongoDB: Creating calculated fields using the switch function (column aliasing)
Data Schema
Note: cal_date1 is the calculated value shown in my other post linked above. When performing these steps they are done in a pipeline so that (I hope) the calculated columns can be thought of as part of the general document schema as shown below.
{
_id:ObjectId("619756f12c115f24df503c26"),
uniqueid:"12345678",
date1:"2021-11-02 20:04:50.253",
date2:"2021-11-03 18:10:57.520",
date3:"2021-11-08 07:08:00.000",
date4:"2021-11-08 14:40:00.000",
date5:"2021-11-08 08:34:00.000",
cal_date1: "2021-11-03 18:10:57.520"
}
Questions
How can I calculate the difference in date values between two particular dates (with some conditional logic behind it).
Extension
Is there a way of calculate the working days between two dates where "working" is defined as dates that are inclusive of Mon-Fri (no Sat, Sun)?
Tried
I have been playing with the $dateDiff operator inside a switch function as shown below however come across the error unknown operator: $cal_date1
db.collection.aggregate([
{
$project:
{
"uniqueid": 1,
"date1": 1,
"date2": 1,
"date3": 1,
"date4":1,
"date5": 1,
"cal_date1": {
$switch: {
branches: [
{ case: {$ne:["$date2",null]}, then: "$date2"},
{ case: {$ne:["$date3",null]}, then: "$date3"},
{ case: {$ne:["$date4",null]}, then: "$date4"},
{ case: {$ne:["$date5",null]}, then: "$date5"}
],
default: "blank"
}
},
"cal_date2": {
$switch: {
branches: [
{ case: {$ne:["$date4",null]}, then: "$date4"},
{ case: {$ne:["$date4",null]}, then: "$date5"}
],
default: "blank"
}
},
"cal_date3": {
$switch: {
branches: [
{ case: {$ne:["$date5",null]}, then: "$date5"}
],
default: "blank"
}
}
}
}
])
--updated code
"cal_days_between_date1_caldate1": {
$switch:
{
branches: [
{case: { $eq: ["$date1", null]}, then: "blank"},
{case: { $eq: ["$cal_date1", "blank"]}, then: "blank"}
],
default: {
$dateDiff: {
startDate: {
$dateFromString: {
dateString: "$date1"
}
},
endDate: {
$dateFromString: {
dateString: "$cal_date1"
}
},
unit: "day"
}
}
}
}
Update: 2021-11-24: T3:08pm UTC
Have altered the case expression slightly to try and get a boolean output and now have a new error of unknown operator: $dateDiff
Update: 2021-11-30: T8:36pm UTC
Have added in the code which contains the referenced fields (top of code chunk) along with the additions of the $dateFromString operator. The calculated date code is meant to act as the next step in the pipeline which I output along with the other fields.

The problem is that you are trying to use Date operators on fields that are strings.
You should transforms your strings to dates before with $dateFromString. Check this playground and docs.
Also there was some mistakes in the $eq arrays
db.collection.aggregate({
"$project": {
"cal_days_between_date1_caldate1": {
$switch: {
branches: [
{
case: {
$eq: [
"$date1",
null
]
},
then: "blank"
},
{
case: {
$eq: [
"$cal_date1",
"blank"
]
},
then: "blank"
}
],
default: {
$dateDiff: {
startDate: {
$dateFromString: {
dateString: "$date1"
}
},
endDate: {
$dateFromString: {
dateString: "$cal_date1"
}
},
unit: "day"
}
},
}
}
}
})

Related

how to set particular field in array of object in mongodb aggregation or pipeline query along with conditional update of the other field in document?

{
id:1,
projectName:'Project1'
start:"17-04-2022",
end:"12-05-2020",
tasks:[
{
taskId:1,
taskName:'first task',
taskStart:'20-04-2022',
taskEnd:'25-04-2022'
},
{
taskid:2,
taskName:'second task',
taskStart:'25-04-2022',
taskEnd:'30-04-2022'
},
]
},
{
id:2,
projectName:'Project2'
start:"27-04-2022",
end:"22-05-2020",
tasks:[
{
taskId:1,
taskName:'first task',
taskStart:'30-04-2022',
taskEnd:'12-05-2022'
},
{
taskid:2,
taskName:'second task',
taskStart:'28-04-2022',
taskEnd:'15-05-2022'
},
]
}
how to update field "taskEnd" of tasks array with document id=1 and taskID=1 AND while updating if data from the frontend which is used to update the "taskEnd" exceeds "end" field, update the "end" field also with same value.
Thanks in Advance,
I know how to update specific field for the array of objects and conditional update of field with two separate queries.
can it be done with single query?
You can do this with a pipeline update instead of using an update document.
I'm assuming you are getting the input date in the same format as the other dates.
var input_date = "11-05-2020";
db.collection.update({
id: 1
},
[
{
$addFields: {
end: {
$cond: [
{
$gt: [
{
$dateFromString: {
dateString: "$end"
}
},
{
$dateFromString: {
dateString: input_date
}
}
]
},
"$end",
input_date
]
},
tasks: {
$map: {
input: "$tasks",
as: "elem",
in: {
$cond: [
{
$eq: [
"$$elem.taskId",
1
]
},
{
$setField: {
field: "taskEnd",
input: "$$elem",
value: input_date
}
},
"$$elem"
]
}
}
}
}
}
])

MongoDB: Creating calculated fields using the switch function (column aliasing)

Note
I am completely new to MongoDB so my terminology might not be fully correct.
Context
I have some data with various dates in MongoDB and I am trying to use a $switch function to provide column aliasing to form some new calculated columns however I am running into some issues (perhaps syntax or wrong implementation) and want to know why and how to resolve.
Question
Basically I want to create a new calculated date column which takes the value of the a date field if its not null, else it takes the value of another date field if its not null and so on, however it defaults to some message "blank" if there is no values in any of those fields.
Schema
A single document with the collection follows this rough structure
{
_id:ObjectId("619756f12c115f24df503c26")
uniqueid:"12345678"
date1:"2021-11-02 20:04:50.253"
date2:"2021-11-03 18:10:57.520"
date3:"2021-11-08 07:08:00.000"
date4:"2021-11-08 14:40:00.000"
date5:"2021-11-08 08:34:00.000"
}
Tried
As part of an aggregation pipeline I tried using the $project and then returning all the columns along with the new calculated columns however it comes up with an error at this stage saying stage must be a properly formatted document leading me to believe I either my syntax is wrong or perhaps using the wrong operators to do this.
If I remove the calculated column code then the preview seems to work (further convincing me its something to do with my calculated column implementation).
db.collection.aggregate([
{
$project:
{
"uniqueid": 1,
"date1": 1,
"date2": 1,
"date3": 1,
"date4":1,
"date5": 1,
"cal_date1": {
$switch: {
branches: [
{ case: {"$date2": {$ne: null}}, then: "$date2"},
{ case: {"$date3": {$ne: null}}, then: "$date3"},
{ case: {"$date4": {$ne: null}}, then: "$date4"},
{ case: {"$date5": {$ne: null}}, then: "$date5"}
],
default: "blank"
}
},
"cal_date2": {
$switch: {
branches: [
{ case: {"$date4": {$ne: null}}, then: "$date4"},
{ case: {"$date5": {$ne: null}}, then: "$date5"}
],
default: "blank"
}
},
"cal_date3": {
$switch: {
branches: [
{ case: {"$date5": {$ne: null}}, then: "$date5"}
],
default: "blank"
}
}
}
}
])
Update1: 2021-11-24 T12:15pm UTC
I have added in a missing curly bracket at the end of each of the case expressions. The error now is unknown operator: $date2
change this case: {"$date4": {$ne: null}}, to case: {$ne:["$date4",null]}, in all of aggregate
Description
It seems that the syntax of my statement is what was causing the issue (not sure why). If someone could elaborate that would be good.
Answer
Here is the code:
db.collection.aggregate([
{
$project:
{
"uniqueid": 1,
"date1": 1,
"date2": 1,
"date3": 1,
"date4":1,
"date5": 1,
"cal_date1": {
$switch: {
branches: [
{ case: {$ne:["$date2",null]}, then: "$date2"},
{ case: {$ne:["$date3",null]}, then: "$date3"},
{ case: {$ne:["$date4",null]}, then: "$date4"},
{ case: {$ne:["$date5",null]}, then: "$date5"}
],
default: "blank"
}
},
"cal_date2": {
$switch: {
branches: [
{ case: {$ne:["$date4",null]}, then: "$date4"},
{ case: {$ne:["$date4",null]}, then: "$date5"}
],
default: "blank"
}
},
"cal_date3": {
$switch: {
branches: [
{ case: {$ne:["$date5",null]}, then: "$date5"}
],
default: "blank"
}
}
}
}
])
Notice that the {"$date_value": {$ne: null}} within each statement has been changed to {$ne:["$date_value",null]} possibly due to the $ne operator requiring 2 arguments?

MongoDB Aggregation pipeline, update an existing elements in an array after string split

Sample Doc :
{
bioupdate: [
{
date: "02/03/2020",
ts: "1583133621197-02/03/2020_15:20:21",
status: "1"
},
{
date: "02/03/2020",
ts: "1583135570542-02/03/2020_15:52:50",
status: "1"
},
{
date: "02/03/2020",
ts: "1583135586272-02/03/2020_15:53:06",
status: "0"
},
{
date: "21-03-2020:17:35:08",
ts: 1584783308231,
status: "1"
}
]
}
Below is the code I've tried with aggregation pipeline splitting the string with first '-' and take the first element which is epoch timestamp and save it to the same field to an existing array.
db.novelmodel.aggregate([
{$match: {pin: "JAIN"}},
{
$project: {
pin: 1,
bioupdate: {
$filter: {
input: "$bioupdate",
as: "bioupdateArray",
cond: { $and: [
{$arrayElemAt:[{$split:["$$bioupdateArray.ts", "-"]}, 0]}
] }
}
}
}
},
{$out:"novelmodel"}
]);
It gives me an error message: "errmsg" : "$split requires an expression that evaluates to a string as a first argument, found: double".I'm not sure how filter to take only the date which has delimiter '-' in a string
Your issue should be the last document which has ts as type NumberLong() instead of string, which is what throwing an error, Try below query :
db.collection.aggregate([
/** Re-create 'bioupdate' with updated data */
{
$addFields: {
bioupdate: {
$map: {
input: "$bioupdate", // Iterate on 'bioupdate' array
in: {
$cond: [
{ $eq: [{ $type: "$$this.ts" }, "string"] }, // Check if 'ts' field in current object is of type string
/** If YES, Split it & get first element from array, it will be string if you want to keep is as number convert using '$long' */
{
$mergeObjects: [
"$$this",
{
ts: {
$toLong: {
$arrayElemAt: [{ $split: ["$$this.ts", "-"] }, 0]
}
}
}
]
},
/** If NO, Just to make sure convert existing to long & merge with actual object & push entire object back to array */
{ $mergeObjects: ["$$this", { ts: { $toLong: "$$this.ts" } }] }
]
}
}
}
}
}
]);
Test : MongoDB-Playground

How to use '$let' in MongoDB Aggregation Query in Scala?

I am trying to write a mongoDB aggregation query in Scala.
How do I write Scala code to use "$let" in '$project' stage?
I am wondering if Variable should be used. Not sure how?
'$project': {
'myprojitem' :{
'$let': {
'vars' : { 'myVariable1': { '$or': [...] } }
'in' : {
'$cond': [
'$$myVariable1',
{ ... },
{ ... },
]
}
}
I figured out the answer. Hopefully it helps someone.
val doc : Document = Document("{
'$let': {
'vars' : { 'myVariable1': { '$or': [...] } },
'in' : { '$cond': ['$$myVariable1',{ ... },{ ... } ]
}
}")
var pipeline = mutable.Buffer[Bson]()
pipeline += Aggregates.project(Projections.fields(
Projections.computed("myprojitem",doc)
))
Basically, every { name : expression } can be written as :
Document("name" -> expression)
Or
Document( "{name : expression}")
$let is used to bind variables together to a results obj. The syntax follows the rule:
{
$let:
{
vars: { <var1>: <expression>},
in: <expression>
}
}
for mere details you should take a look at $let (aggregation) definition from mongodb manual
Here is a text book example just to make more sense:
Consider the following data:
{ _id: 1, price: 10, tax: 0.50, applyDiscount: true }
{ _id: 2, price: 10, tax: 0.25, applyDiscount: false }
And imagine that we want to generate a result for the finalTotal in a way that:
Where Disc = 10% if applyDiscount: true and 0 otherwise.
So we need now to create the aggregation on the data to construct this equation. So we can get a results like:
{ _id: 1, finalTotal: 9.45 }
{ _id: 2, finalTotal: 10.25 }
We can do this by doing:
$project: {
finalTotal: {
$let: {
vars: {
total: { $add: [ '$price', '$tax' ] },
discounted: { $cond: { if: '$applyDiscount', then: (0.9, else: 1 } }
},
in: { $multiply: [ "$$total", "$$discounted" ] }
}
}
}
We can break this down:
Step 1. adding price to tax together to a variable called total
total: { $add: [ '$price', '$tax' ] },
Step 2. transforming the condition in numbers (variable discounted)
discounted: { $cond: { if: '$applyDiscount', then: 0.9, else: 1 } }
Step 3. performing the operation $multiply operation between the constructed $$total and $$discounted
in: { $multiply: [ "$$total", "$$discounted" ] }

mongodb - $dateFromString to support multiple format

I have a string field in mongodb which should be converted to a date field.
The format of the string is like the following:
2014 - Only year, default month and day are 01 and 01, so it should be converted to date '2014-01-01'
2014-01 - With year and month, which should also be converted to date '2014-01-01'
2014-01-01 - Full date
$dateFromString in the following syntax doesn't seem to work:
$dateFromString: {
dateString: '$order.date',
format: '%Y-%m-%d',
}
How can I make $dateFromString to support multiple format?
What you could do is to add a new field via $addFields and then for its value create few if conditions using the $cond pipeline operator matching each of your date lengths (via $strLenCP) and concatenating the remaining parts (via $concat). Then since all of your date fields will now match the format %Y-%m-%d it should work ... like this:
db.getCollection('<YourCol>').aggregate([{
$addFields: {
dateFixed: {
$cond: {
if: { $eq: [{ $strLenCP: "$date"}, 4] }, // <-- "2011"
then: { $concat: ["$date", "-01-01"] },
else: {
$cond: {
if: { $eq: [{ $strLenCP: "$date" }, 7] }, // <-- "2011-01"
then: { $concat: ["$date", "-01"] },
else: "$date" // <-- "2011-01-01"
}
}
}
}
}
},
{
$project: {
date: {
$dateFromString: {
dateString: 'dateFixed',
format: '%Y-%m-%d'
}
}
}
}
])
You can see it working here