Background
momentjs 2.8.3
angularjs
collating dates in a date table
Problem
Trevor wishes to get the global timespan of dates in a date table, where each record contains a start date and an end date.
Goal
The goal is to get a global timespan, such that the earliest part of the timespan reflects the earliest date in any row the table, and the latest part of the timespan reflects the latest date in any row the table.
Trevor does not know in advance how the dates are arranged in the table, other than they are all formatted as 'YYYY-MM-DD'
Trevor is sold on momentjs as the most effective js library for handling this kind of problem, but he is open to using any others.
Details
The data is all encoded in JSON and structured as below.
```
dataroot {
"datedemo_data_table": [
{
"datebeg": "2014-01-15",
"dateend": "2014-02-15"
},
{
"datebeg": "2014-03-15",
"dateend": "2015-01-01"
},
{
"datebeg": "2015-06-15",
"dateend": "2015-07-20"
},
{
"datebeg": "2012-08-15",
"dateend": "2013-08-15"
},
{
"datebeg": "2013-01-15",
"dateend": "2013-01-16"
}
],
"datedemosummary_data_dict": {
"x": "x",
"ds_soonst_date": "",
"ds_latest_date": ""
}
}
```
The goal is to populate the ds_soonst_date and ds_latest_date with the correct date values.
Questions
Is momentjs the best library for a task such as this?
Are there any performance implications for large data tables (over 10k records)?
You actually don't need moment (or any library) for this. Since the values are in YYYY-MM-DD format, they are sortable as strings. Simple array/object manipulation will work.
var data = JSON.parse('{"datedemo_data_table":[{"datebeg":"2014-01-15","dateend":"2014-02-15"},{"datebeg":"2014-03-15","dateend":"2015-01-01"},{"datebeg":"2015-06-15","dateend":"2015-07-20"},{"datebeg":"2012-08-15","dateend":"2013-08-15"},{"datebeg":"2013-01-15","dateend":"2013-01-16"}],"datedemosummary_data_dict":{"x":"x","ds_soonst_date":"","ds_latest_date":""}}');
var firstBegDate = data.datedemo_data_table
.map(function(x){return x.datebeg;})
.sort().shift();
var lastEndDate = data.datedemo_data_table
.map(function(x){return x.dateend;})
.sort().pop();
As far as performance goes - if you have 10k items in a single JSON, that's probably an issue right there. You will always have O(n) performance with any approach unless you use an index to reduce the data to start with.
Answer
momentjs is an excellent choice as it is a well-documented and feature-full library.
The performance question is not addressed here, perhaps someone else can chime in on that.
Nevertheless with a small table of a few values, you can get a quick result by doing a collation of the dates into a single javascript array, and then extracting the max and min using the relevant functions from momentjs.
This can be done easily with the following:
Solution
var fmt = 'YYYY-MM-DD'
,ddtemp = $scope.dataroot.datedemosummary_data_dict
,aatemp_dates = []
;
$scope.dataroot.datedemo_data_table.forEach(function(currow, ixx, arr) {
aatemp_dates.push(moment(currow.datebeg,fmt));
aatemp_dates.push(moment(currow.dateend,fmt));
},ddtemp);
ddtemp.ds_soonst_date = (moment.min(aatemp_dates).format(fmt));
ddtemp.ds_latest_date = (moment.max(aatemp_dates).format(fmt));
Result
dataroot {
"datedemo_data_table": [
{
"datebeg": "2014-01-15",
"dateend": "2014-02-15"
},
{
"datebeg": "2014-03-15",
"dateend": "2015-01-01"
},
{
"datebeg": "2015-06-15",
"dateend": "2015-07-20"
},
{
"datebeg": "2012-08-15",
"dateend": "2013-08-15"
},
{
"datebeg": "2013-01-15",
"dateend": "2013-01-16"
}
],
"datedemosummary_data_dict": {
"x": "x",
"ds_soonst_date": "2012-08-15",
"ds_latest_date": "2015-07-20"
}
}
See also
momentjs #min
momentjs #max
momentjs range addon library by gf3 https://github.com/gf3/moment-range
Related
I'm trying to match the values greater than and less than a datetime parameter which holds the value for the current date time after it's been formatted with momentjs.
The datetime variable
datetime = moment.utc().format('YYYY-MM-DDTHH:mm:ssZ')
The values I'm trying to match exist in a mongo's document as seen here in the picture
The code I used for matching :
if (_params.datetime) {
_mongoParams.push({
$and: [
{
_rewards: {
_endDatetime: { $lte: new Date(datetime) }
},
},
{
_rewards: {
_startDatetime: { $gte: new Date(datetime) }
},
}
],
})
}
The problem is I'm not getting any results when I try to make requests with the datetime parameter. I tried placing the datetime variable inside an ISODate constructor but it didn't work too.
How can I possibly match the dates ?
I'm trying to fetch second max date json data from an json column..
Here is jsonb column
--------
value
--------
{
"id": "90909",
"records": [
{
"name":"john",
"date": "2016-06-16"
},
{
"name":"kiran",
"date": "2017-06-16"
},
{
"name":"koiy",
"date": "2018-06-16"
}
]
}
How to select the second maximum date json object..
expected output:-
{
"name":"kiran",
"date": "2017-06-16"
}
and if we have only one object inside the records means that will be the second max date
and any suggestions would also helpful..
My main suggestion would be this: If your data is structured, do not store it in a JSON. It will be much easier to work with it if you structure it as relational tables.
But anyhow, here's one way to get the second-latest-date object. First unpack the array, then sort by the date and take the second to last:
SELECT obj.*
FROM your_table, jsonb_array_elements(value->'records') obj
ORDER BY obj->'date' DESC
LIMIT 1 OFFSET 1;
value
-----------------------------------------
{"date": "2017-06-16", "name": "kiran"}
(1 row)
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqu08dsyxz98whc is one out of possibly many tx which have whc substring in output address,
https://blockchair.com/bitcoin-cash/transaction/ce4b6388c3b57dc188bfafde87d7af28ee3ba210d0a3223a3bc86f6083337459
I would like to find such outputs for 2018-08 using bitdb query lang similar to mongodb,
{
"v": 3,
"q": {
"find": {
"out.e.a": { "$regex": "whc$" },
"blk.t": {
"$gte": "2018-08-01T00:00:00Z",
"$lte": "2018-08-31T00:00:00Z"
}
},
"limit": 1
}
}
Unfortunately I'm not getting any result for such query
Is there some syntax issue which prevents correct results?
Try to query using Unix timestamps instead of ISO Dates.
To convert from ISO Date to unix timestamp in javascript you can do:
var myDate = new Date(ISODate("2015-10-25T00:00:00.000Z"));
var myTimeStamp = myDate.getTime() / 1000;
I'm working on a Meteor application and I have data for a weekly timetable of the format -
events:
event:
day: "Monday"
time: "9am"
location: "A"
event:
day: "Monday"
time: "10am"
location: "B"
There are numerous entries for each day. Can I run a query that will return an object of the format -
day: "Monday"
events:
event:
time: "9am"
location: "A"
event:
time: "10am"
location: "B"
I could store the object in the second format but prefer the first for ease of deleting and updating individual events.
I also want to return them ordered by day of week if there's a nice way to do that.
Several options:
You can use an aggregation command but be warned, you will loose reactivity: it means that except when you reload your template, you will not get external updates. You would also need to use a package to add the aggregation command to Meteor in order to achieve that.
My personal favorite: you don't need to aggregate (and loose reactivity) to achieve your data transformation. You can use a simple Collection.find() query and extend/reduce/modify it using a clever mix of cursor.Observe() and conditional modifications. Have a look at this answer, it did the trick for me (I needed a sum with black listing of some fields, but you can easily adapt it to your group/sorting case) : https://stackoverflow.com/a/30813050/3793161. Comment if you need more details on this
If you plan to have several servers, be warned that each server will have to observe so it may lead to an unnecessary load. So my third solution is either use collection hooks or methods to update an additional field containing every event for each day/user (whatever you need).See #David Weldon answer about that here: https://stackoverflow.com/a/31190896/3793161. In you case, it would probably mean to re-think your database structure to fit your needs (i.e. adding more fields so you ca update them on insert.
EDIT Here are some thoughts on your comment:
If you stick to what you described in the question, you would need seven documents, one per day, with an events field where you put all the events. My second solution is good if you need to rework a collection structure before sending it. However, in your case, you just need an object week with 7 sub-objects matching the days of the week.
I advise you to possible approaches:
use the aggregation in a method, as described by #chridam. Be warned that you will not be able to directly get a sorted array, as stated in mongo $group documentation
$group does not order its output documents
So you need to sort them (by day and by hour within each day) using, for example _.sortBy() before you return your method result. By the way, if you want to know what is going on in your method call, clientside, here is how you should write the call:
Meteor.call("getGroupedDailyEvents", userId, function(error, result){
if(error){
console.log("error", error);
}
if(result){
//do whatever you need
}
});
Make the data sorting client-side. You are looking for an overkill solution because, afaik, you don't need to filter any data to keep it from the user, and you are going to send the data anyway (just with another structure). This is much easier to make a simple helper in your template like this:
Template.displaySchedule.helpers({
"monday_events": function() {
return _.sortBy (events.find({day:"Monday"}).fetch(), "time")
},
//add other days
);
It assumes the format of your time field is sortable this way. If not, you just need to create a function to sort them accordingly to their formats or change the original format into something better suited.
The rest (HTML) would just be to iterate on Monday events using a {{#each monday_events}}
To achieve the desired result, use the aggregation framework where the $group pipeline operator groups all the input documents and apply the accumulator expression $push to the group to get the events array.
Your pipeline would look like this:
var Events = new Mongo.Collection('events');
var pipeline = [
{
"$group": {
"_id": "$day",
"events": {
"$push": {
"time": "$time"
"location": "$location"
}
}
}
},
{
"$project": {
"_id": 0, "day": "$_id", "events": 1
}
}
];
var result = Events.aggregate(pipeline);
console.log(result)
You can add the meteorhacks:aggregate package to implement the aggregation in Meteor:
Add to your app with
meteor add meteorhacks:aggregate
Since this package exposes .aggregate method on Mongo.Collection instances, you can define a method that gets the aggregated result array. For example
if (Meteor.isServer) {
var Events = new Mongo.Collection('events');
Meteor.methods({
getGroupedDailyEvents: function () {
var pipeline = [
{
"$group": {
"_id": "$day",
"events": {
"$push": {
"time": "$time"
"location": "$location"
}
}
}
},
{
"$project": {
"_id": 0, "day": "$_id", "events": 1
}
}
];
var result = Events.aggregate(pipeline);
return result;
}
});
}
if (Meteor.isClient) {
Meteor.call('getGroupedDailyEvents', logResult);
function logResult(err, res) {
console.log("Result: ", res)
}
}
I have a document structure like
{
"startDate": ISODate("2015-01-01T00:00:00Z"),
"endDate" : ISODate("2015-01-10T00:00:00Z"),
"foo" : "bar"
}
Is it possible to expand the date range like this?
{
"dates": [
ISODate("2015-01-01T00:00:00Z"),
ISODate("2015-01-02T00:00:00Z"),
ISODate("2015-01-03T00:00:00Z"),
ISODate("2015-01-04T00:00:00Z"),
ISODate("2015-01-05T00:00:00Z"),
ISODate("2015-01-06T00:00:00Z"),
ISODate("2015-01-07T00:00:00Z"),
ISODate("2015-01-08T00:00:00Z"),
ISODate("2015-01-09T00:00:00Z"),
ISODate("2015-01-10T00:00:00Z")
]
}
As far as I understood you want to add field dates for all your documents. Here is an approach I would use (you can do this in mongoshell):
1) iterate over all the documents modifying them
db.coll.find()..snapshot().forEach(function(o){
o.dates = func(o.startDate, o.endDate);
db.coll.save(o);
});
2) where you function func is something similar to this answer (you need to modify it a little bit because it looks like you need only dates without time included there.