My actual documents are more complex than this but simplifying them like so will explain the problem I want to solve. I have daily and weekly documents.
Daily Document: ObjectId, Type, Count, Date
Weekly Documents: ObjectId, Type, Count, StartDate, EndDate
If I wanted a daily report I can run a query that will select documents with Date field value between range X to Y and Type equal to 'daily'. I can do the same thing for Weekly reports and it all works.
The problem:
For weekly reports if the start date is not the first day of the week and the end date is not exactly the last day of the week, selecting documents with Type 'weekly' will produce inaccurate reports since weekly documents store the data for the entire week. This may seem strange but Google Analytics lets you do it:
In the above screenshot Jul 3rd isn't the beginning of that week, nor is Jul 17th the end of that week. But Google Analytics lets you see the data as you want.
Possible Solution:
One possible solution is to produce a daily report for the overflowing days and subtract it from the weekly report.
The question:
Is there a nicer solution to solving the problem I described? I'm open to redesigning the documents
For weekly reports if the start date is not the first day of the week
and the end date is not exactly the last day of the week, selecting
documents with Type 'weekly' will produce inaccurate reports since
weekly documents store the data for the entire week. This may seem
strange but Google Analytics lets you do it:
Because they don't store documents like that.
The way this works (I reckon) is that Google just summerises the daily documents and ignores your "week" range and applies their own range of saying:
get as many full weeks as possible and aggregate the daily documents that
come under that range
and then:
Just throw the others ontop making each the end and start point
I wouldn't try to mix week and daily documents here I would just query over daily documents only and aggregate client side.
Related
Does anyone know how to retrieve data by month using Mongodb query? For example, if I only want to retrieve the date range for the past month, or for the past two month, how do I do that? I see examples for getting data for the last N day (e.g for past 5 days using this new Date(new Date().setDate(new Date().getDate()-5) or N hours, but not seeing examples for only retrieving data for the last N month. Thanks,
Yes, You can retrieving data based on month like this.
new Date().setMonth(new Date().getMonth()-1) or N months.
And you can also get past month start and end date using moment like this.
moment().subtract(1,'months').startOf('month').format('YYYY-MM-DD')
it will provide previous month start date i.e 2021-01-01
moment().subtract(1,'months').endOf('month').format('YYYY-MM-DD');
it will provide previous month end date i.e 2021-01-31
Here I'm saving the date range using golang. Suppose we have to save the all monday comes between the range of the 1-may-2018 to 14-july-2018.
How we will find all the monday between these range using golang and on the other hand we have set the start_time (8:00 A.M.) and the end_time (6:00 P.M.) of the first two coming monday in the database but on the third monday we have a change in the schedule that there is a time change like start_time (9:00 A.M.) and end_time (5:00 P.M.). Then how I will make my database to make this situation in practically using the golang.
Can Anybody help me for this to solve this solution. I made a database for and I do ppr work on it and make some fields shown below:-
Fields for Schedule //Schedule is a collection name
Id (int)
Day (string)
Start_hours (int)
Start_minutes (int)
End_hours (int)
End_minutes (int)
Start_date (timestamp)
End_date (timestamp)
How I will select monday between the selected range and how will I do the situation I explained above can anybody give guidance to me to make this situation easier. Thank you if this is a basic question then I'm really sorry.
I'd make something like this.
Find the first Monday date from the date range (see for example How do I get the first Monday of a given month in Go?
Mondays happen every week, so you can easily find the rest of dates by adding 7 days till the end date
Store all the Monday dates you found with the start and end times
I wouldn't bother with hours and minutes as you can easily get them from the timestamps in Go. Here is the simplified DB structure I would make
Fields for Schedule //Schedule is a collection name
Id (int)
Day (string)
Date (timestamp) // the actual date
Start (timestamp)
End (timestamp)
You don't need any more fields. You can get the day of the week (Day (string) in your structure, e.g. Monday) from the Date field too, but I believe if you want to query the collection by different days, this might speed things up, but be careful if you need to adjust for time zones. If you work with more than one, then store everything in UTC and you may have an extra filed Timezone, cos a date could be Monday for one zone and Sunday for another.
So, the Schedule will hold weekdays and start and end times for each of them. I'm not sure if you need to store initial date ranges, the Schedule collection will hold that range as well, form the first record to the last one. In my mind, I'd initially populate the collection with a given date range, then later on, I can modify it by adding new days, or deleting them.
When you query this collection with some start and end date range for the Date field, if your first result comes newer than 7 days from the start, this means you miss 1 or more entries from the start. If the last result comes older than 7 days from the range end, this means you miss some entries prior to the range end.
There is nothing specific to Go, in my opinion, Go works well with dates and you don't need any special date structures in your DB.
I'm new to Neo4j so maybe I'm just completely wrong on this, but I'll give it a try!
Our data is mostly composed by reservations, users and facilities stored as nodes.
I need both to count the total reservations that occurred in a specific time frame and the overall income (stored as reservation.income) in this timeframe.
I was thinking to overcome the problem by creating the date as a node, in this way I can assign a relationship [:PURCHASED_ON] to all the reservations that occurred on a specific date.
As far as I've understood, creating the date as a node could give me a few pros:
I could split the date from dd/mm/yyyy and store them as integer properties, in this way I could use mathematical operators such as > and <
I could create a label for the node representing each month as a word
It should be easier to sum() the income on a day or a month
Basicly, I was thinking about doing something like this
CREATE (d:Day {name:01/11/2016 day: TOINT(01), month: TOINT(11), year: TOINT(2016)}
I have seen that a possible solution could be to create a node for every year, every month (1-12) and every day (1-31), but I think that would just complicate terribly the architecture of my Graph since every reservation has an "insert_date" (the day it's created) and then the official "reservation_date" (the day it's due).
Am I onto something here or is it just a waste of time? Thanks!
You may want to look at the GraphAware TimeTree library, as date handling is a complex thing, and this seems to be the logical conclusion of the direction you're going. The TimeTree also has support for finding events attached to your time tree between date ranges, at which point you can perform further operations (counting, summing of income, etc).
There are many date/time functions in the APOC plugin that you should take a look at.
As an example, you can use apoc.date.fields (incorrectly called by the obsolete name apoc.date.fieldsFormatted in the APOC doc) to get the year, month, day to put in your node:
WITH '01/11/2016' AS d
WITH apoc.date.fields(d, 'MM/dd/yyyy') AS f
CREATE (d:Day {name: d, day: f.days, month: f.month, year: f.years});
NOTE: The properties in the returned map have names that are oddly plural. I have submitted an issue requesting that the names be made singluar.
I have a fact table housing different granularity (date grain)
Monthly
Daily
The month data can be accessed by filtering by end of month date or using YYYYMM date format. In OBIEE RPD repo, the fact is set to LAST Aggregation.
I want to perform Year to Date analysis. And I want to sum only month end dates.
Using function TODATE(Measure), it tends to sum up all the data through out the month e.grain
Date Amount YTD TODate(Amount)
31/01/2106 100 100
28/02/2016 200 300
14/03/2016 50 350*
31/03/2016 100 450
I want YTD to ignore 50 and return 400, so also any other dates that falls within any month. And if if I Select 14/03/2016 I want 350 to return.
Thanks.
Alter the table to add a flag, something that flags Y if the record is at the specified monthly grain, and N if the record is not at the specified monthly grain.
In the logical layer, create two distinct LTSs with the first filtering on the flag for Y. This will be where you will calculate and source all your to date measures. The second LTS can either be filtered to N, or can be left to all the data depending on what you want to do with it.
The performance increases should come from the fact that any month measures you build off that monthly LTS will only hit records flagged as month, and will bypass all that other data that is not relevant. So if a user runs a report only asking for monthly measures, the query will automatically filter to that specific data.
What will happen is if a user selects your to date measure and a specific date measure on the same report, OBIEE should fire off two separate queries to get the data and stitch together based on common dimensions.
Could someone create this in the front end? Probably. You would have to do some sort of PERIODROLLING function, and tell it to aggregate at the month level, but I am afraid it may still roll those days up into a larger than desired number. A TODATE function will not work here.
In my application I have to store data month wise and year wise. So for this, I have to store the data along with date into database.
My requirement is how to store in terms of date and how to retrieve data with group by month and year. In my app I am showing a table of years and months, based on selected month and year. I have to show the data in a dashboard.
My problem is in storing and retrieving date data types.
Use the following syntax
SELECT * FROM DATABASE WHERE REQUIREDDATEFIELD LIKE '%2011-01%';
2011 is supposed to be the year
01 is supposed to be the month
DATABASE is supposed to be your mysql database name
REQUIREDDATEFIELD is supposed to be the field you are hoping to sort from month and year.
like '%2011-01%' is supposed to be meaning, all the records containing 2011-01 in the given field. It could be in the beginning or the end or in the middle of a large text, so having % in both the beginning and end of the search criteria is a good habit.
You just select either for a specific month or year or month and year. Or if you want all, you use GROUP BY.
I know this answer is quite vague and generic, but that's because your question is vague. You probably need to be more specific. Explain not only what you want to do, but what you have tried, and in which way that didn't work.