I am developing a social calendar application and want to make a feature for recuring events. I decided to use the following schhema as this might be the least memory intense for transfering (using meteorJS)
{
title: "Some Title",
/* ... some other fields for the event, e.g. location, name of venue, decription etc, a lot of data */
date: new Date(), // JS Date the event takes place first time or date for single event
repeatMethod: { //Every 2nd Monday in month, is used for generating repeatingEvents array
type: "monthly"
day: "Monday"
value: "2"
},
repeatingEvents: [
{
date: /*date for the first month */,
additionalStuff....
},
{
date: /* date for the second month */,
},
{
date: /*Date for the third month */,
canceled: true
}
]
}
I have several of these events (Around 100 Recuring events that repeat every week and around 50 events that repeat every month) and some single events that have no recurence.
Problem: I need to be able to list all events in a list like this:
Event 1 Occurence 1
Event 2 Occurence 1
Event 3 Single Event
Event 1 Occurence 2
Event 2 Occurence 2
Event 3 Occurence 1
...
but I want also to be able to limit the start and enddate of this list.
My first approach was to use single datasets per event but that blows up the database extremely. It also would be needed to store a complete event just for marking it canceled.
What way can I effectively find all events in a special timespan and list them ordered by date?
You're worrying about space when you should be worrying about seek times & user experience. Space is cheap, computations (and definitely user retention) aren't.
I suggest 2 collections, Events, Occurrences.
Store the eventId on the Occurrence document & index it.
So an event doc has _id, venue, title, etc.
An occurrence doc has _id, eventId, datetime, venue, title, etc.
Note:
Occurrence repeats venue because what happens if the regular venue isn't available a particular week? As a user, if changing the venue for 1 week changes the venue for all weeks, I'm gonna be grumpy.
We also repeat title (and all other fields, except the repetition info) so we don't need to access 2 collections for our queries. Denormalization is healthy, especially in a NoSQL environment.
This enables the user to change a field (like title) for a particular week (occurrence). If they want to change that field for ALL upcoming weeks, or maybe change the repeat structure, then they edit the event itself & that change to the event doc is propagated to all future occurrences.
A side effect of this structure is a really easy query for what you want: Occurrences.find({eventId: event, datetime: {$gt: date, $lt: date}).
Related
I'm trying to write to a Firebase grandchild node based on the value of one of its siblings. This value is unique for each node. Specifically, each day of the year returns a different note. My db structure is currently
= notes
=== unique key
===== day of year
===== note string
===== etc
I want to be able to query by day of year and edit the note string. What's the best way to do this? Should I make the day of the year the key? I was going to use multiple reads and a write, but it seems like there should be a more efficient way.
In your case, it seems like the push ID doesn't benefit you because notes are already unique based on the day. It makes more sense to do a structure with the key as the day of the year, as you suggested.
notes: {
day of year: {
note string
etc
}
}
I'm writing a web app using Firestore that needs to be able to show "Today's Popular Posts" and I'm having trouble getting the queries right when considering users in different timezones.
The dates are stored in the DB as UTC 0, then adjusted to the current user's UTC offset in the client via Moment.js. This works correctly.
When adding a new post I use firebase.firestore.FieldValue.serverTimestamp() to store the current server timestamp in a field called timestamp, like so:
const collectionRef = db.collection('posts');
collectionRef.add({
name: "Test Post",
content: "Blah blah blah",
timestamp: firebase.firestore.FieldValue.serverTimestamp(),
likeCount: 0
});
Then on the server I have a Cloud Function that runs on create and adds another field to the document called datestamp which is the the UTC 0 timestamp, but adjusted so that the time is the beginning of the day. The function looks like this:
exports.updatePostDate = functions.firestore
.document('posts/{postID}')
.onCreate((event) => {
const db = admin.firestore();
const postRef = db.doc('post/'+event.params.postID);
const postData = event.data.data();
const startOfDay = moment(postData.timestamp).startOf('day').toDate();
return postRef.update({
datestamp: startOfDay
});
});
Storing a timestamp where the time is always the beginning of the day enables me to write a query like this for finding all posts and ordering by popularity on a given day:
const startOfDayUTC = moment.utc().startOf('day').toDate();
const postQuery = db.collection('posts')
.orderBy('likeCount', 'desc')
.orderBy('timestamp', 'desc')
.where('datestamp', '==', startOfDayUTC)
.limit(25);
The problem is, depending on the user's UTC offset, this can display posts with two different dates when parsing the post's timestamp field. So even though the query is correctly fetching all the posts where the datestamp is say, 2018-01-30T00:00:00Z, the timestamp's date might not be the same once parsed. Here's an example of two posts:
Post 2:
likeCount: 1
timestamp (UTC 0): 2018-01-30T06:41:58Z
timestamp (parsed to UTC-8): 2018-01-29T22:41:58-08:00
datestamp (UTC 0): 2018-01-30T00:00:00Z
Post 1:
likeCount: 0
timestamp (UTC 0): 2018-01-30T10:44:35Z
timestamp (parsed to UTC-8): 2018-01-30T02:44:35-08:00
datestamp (UTC 0): 2018-01-30T00:00:00Z
So you can see, while the posts have the same datestamp, after adjusting the timestamp to the local UTC, the timestamp fields can end up being on two different days.
If anyone has a solution to this I would be very grateful.
I think it is better to avoid functions in this case as you can perform compound queries now. You can simply use
query.where(date > lastMidnight).where(data < now).get().then(...)
so to speak to limit data which only belongs to one day and try to keep all your time variables in UTC 0 and just find the start point and the current time both client side and convert them to UTC0.
//get local time from midnight to now (local)
const now = new Date();
const lastMidnight = now.setHours(0,0,0,0);
//then convert those to UTC0 to pass on in your query to firestore
const lastMidNightUTC = new Date(lastMidnight + now.getTimezoneOffset() * 60000).toString();
const nowInUTC = new Date(now + now.getTimezoneOffset() * 60000).toString();
and you can get your data (remember you need to make an index or just run the query once and firebase SDK will generate a link to create the index in dev tools -> console , for you)
query.where(date > lastMidNightUTC).where(data < now).get().then(...)
I came up with a solution that I'm really not happy with... But it works!
The problem is fundamentally one post can be on more than one date, depending on the user's location. And since for this case we also want to order by a field other than timestamp we can't use a range query to select posts on a given date, because your first .orderBy must be on the field you're using a range query on (see Firestore docs).
My solution is to map localized datestamps to their corresponding UTC offset. The object contains every UTC offset as a key, and the post's datestamp in that offset's time.
An example post looks like this:
posts/{somepostid}
{
name: "Test Post",
content: "Blah blah blah",
timestamp: Mon Jan 29 2018 21:37:21 GMT-0800 (PST),
likeCount: 0,
utcDatemap: {
0: "2018-01-30,
...
-480: "2018-01-29",
...
}
}
The field utcDatemap is the the one we use in our queries now:
const now = moment();
const datestamp = now.format("YYYY-MM-DD");
const utcOffset = now.utcOffset();
const utcDatemapField = 'utcDatemap.'+utcOffset;
const postQuery = db.collection('posts')
.orderBy('likeCount', 'desc')
.orderBy('timestamp', 'desc')
.where(utcDatemapField, '==', datestamp)
.limit(25);
Now posts can show up on two different days, depending on where the user is querying from. And we can still convert the regular old timestamp to the user's local time on the client.
This is definitely not an ideal solution. For the above query I needed to create composite indexes for every single key in utcDatemap. I'm not sure what the rules of thumb are with composite indexes, but I'm thinking having 39 indexes for something simple like this is probably not great.
Additionally I checked it out using the roughSizeOfObject function from thomas-peter's answer on this post and the utcDatemap object, with all it's string datestamps clocked in at roughly 780 bytes, and it's not like 0.78kb is a lot, but you do need to be mindful of how much data you're transferring with a service like Firestore (0.78kb is a lot for a date).
I'm learning/reading up on Firestore and have Google'd to see how it deals with times, so discount my answer appropriately.
It looks as though Firestore converts times to UTC and stores them as its own Timestamp datatype. If so, then it's critical to know that this is a destructive conversion.
Though UTC is useful for comparing instants in time, it means that the wall-clock time as observed by the app user is lost forever. Some countries like the UK are in one of two timezones during the year, Daylight Savings Time and British Summer Time.
You can convert back to the user's observed time, but the problem is that the rules change over the years. You'd have to keep a record of all the different rule changes for all the timezones and countries of the world.
What was the time at the time?
The question is, what time did the user think an event happened ...at the time. This can have legal ramifications. You may need to go back through historic data to prove a person acted at a certain time as they observed it.
The solution is to capture the user's observed offset in an additional field. That way, you can always use this to convert.
Regarding the OPs problem, this seems somewhat philosophical for a web app. Does "today" mean the event, such as a post, must have happened within the user's Monday? Or just posts on today's date? Or posts within the last 24h?
An important thing to remember is that dates are the same all around the world, even when they begin and end at different instants.
What's Elvis got to do with all this?
Christmas Day is 25th everywhere. If I say something happened on Christmas Day and I'm in the USA, and then someone in Australia wants to see all the world's posts made on Christmas Day, then they need to query for posts where observedDate == 25th Dec.
Think about it. Such posts were all made on Christmas Day, even though it might have been Boxing Day for me in England at the instant that they posted.
Elvis died on 16th August. In the UK our radio stations don't all wait until it's the 16th in the timezone of the place of his death to start playing his records.
Another interesting one is whether something happened in Q1 or Q2 of a company's reporting year. Is a sale recognised as on the date at the point-of-sale in the store in New York, or in the database in LA?
The observed date is interesting.
My point is, think deeply about this and store both a normalised instant like UTC, but also the user's observed date, time and offset. Then you have all the information you'll need for the future.
Finally, consider adding observed year or week numbers, or day ordinals, H1/H2 for financial data, since it can be super useful for composing rapid queries, depending on your use-cases.
I'm new to Neo4j so maybe I'm just completely wrong on this, but I'll give it a try!
Our data is mostly composed by reservations, users and facilities stored as nodes.
I need both to count the total reservations that occurred in a specific time frame and the overall income (stored as reservation.income) in this timeframe.
I was thinking to overcome the problem by creating the date as a node, in this way I can assign a relationship [:PURCHASED_ON] to all the reservations that occurred on a specific date.
As far as I've understood, creating the date as a node could give me a few pros:
I could split the date from dd/mm/yyyy and store them as integer properties, in this way I could use mathematical operators such as > and <
I could create a label for the node representing each month as a word
It should be easier to sum() the income on a day or a month
Basicly, I was thinking about doing something like this
CREATE (d:Day {name:01/11/2016 day: TOINT(01), month: TOINT(11), year: TOINT(2016)}
I have seen that a possible solution could be to create a node for every year, every month (1-12) and every day (1-31), but I think that would just complicate terribly the architecture of my Graph since every reservation has an "insert_date" (the day it's created) and then the official "reservation_date" (the day it's due).
Am I onto something here or is it just a waste of time? Thanks!
You may want to look at the GraphAware TimeTree library, as date handling is a complex thing, and this seems to be the logical conclusion of the direction you're going. The TimeTree also has support for finding events attached to your time tree between date ranges, at which point you can perform further operations (counting, summing of income, etc).
There are many date/time functions in the APOC plugin that you should take a look at.
As an example, you can use apoc.date.fields (incorrectly called by the obsolete name apoc.date.fieldsFormatted in the APOC doc) to get the year, month, day to put in your node:
WITH '01/11/2016' AS d
WITH apoc.date.fields(d, 'MM/dd/yyyy') AS f
CREATE (d:Day {name: d, day: f.days, month: f.month, year: f.years});
NOTE: The properties in the returned map have names that are oddly plural. I have submitted an issue requesting that the names be made singluar.
I have one iphone application in which recurring events happens.the duration are like every day,every 2nd day,every 3rd day..,every week on mon,every week on tues,...,every second week on mon,..,every month on 1sr,every month on 2nd...,every second month on 1st,every second month on 2nd,... For some events there is end date but some events occurs forever.so how can i insert them automatically in sqlite3 database.eg.If an event repeats every 3rd day.how can i store it automatically after 3 days.or should i store all the events at the time of creation.If i store all the evnets at time of creation then the events that repeats forever.upto what duration i should store the value of them in database.
For this i have thought 2 approaches.one is storing just one occurance with repeated duration like every day.but in my application edit and delete functionality is also there.suppose event has one field event description then it can be different for different dates if user edit the events.events are displayed datewise on screen for a particular month and user can navigate to any previous and next month for current ,next and previous years.So if i use only single occurance then where should those edited or deleted events should be stored.
And if i take second approach store each occurance in database.Then upto what duration i should store the events which has no enddate.or is there a way that insert is automatically performed after specified duration.
Two ways, one an easy hack, one more difficult but correct :)
(1) Store each individual event for the next 10 years or so (or however long you want to!)
Good :
Easy and quick to implement
Bad :
Your db will get big very quickly.
What if the user wants to edit the details - how do you edit all of them at once?
What if the user wants to cancel / move the event- you have to delete all of them!
(2) Store some sort of 'event recurrence' information in the database with each event i.e. 'Every tuesday'
Good :
Only one event in the database, regardless of how many times it repeats.
Easy for the user to edit / delete the event - there's only one row in the database.
Bad:
More complicated event object - each event must have a list of 'when this event happens' information.
Makes very complicated event timings hard - i.e. 'last friday of every month'
Takes longer to implement
EDIT : My personal choice
I would choose option (2) - it takes longer to implement but I think option (1) will et you into trouble in the future.
I would have a data model like
Event has many Occurances
where your Event is the thing that the user has created with a description, start date etc and an Occurance is some sort of object that will say 'every friday' or 'not on the 4th'.
As part of creating an event, the user will say 'occurs once on Friday 13th' or 'occurs every Wednesday'. That information is used to create an array of Occurance objects.
Occurance would be a protocol that simply has the method occursOn: so you can have lots of different types of occurance (and you can add new types as your app gets more complicated).
The Event object would have a method something like occursOn: where you give it an NSDate and it returns if it occurs on that day. It does this by asking each of it's occurances in turn to see if they apply to that day.
To deal with deleted events, just add an Occurance to the end of the Event's array that overrides the others - i.e. 'not on Friday 13th'.
For example :
(1)
A user creates an event called 'My Event' starting on 1st Jan, occurring every Friday.
Your app would store
Event
description : 'My Event',
start date : 1st Jan 2011
occurances :
WeeklyOccurance
day : Friday
where WeeklyOccurance implements the <Occurance> protocol
(2)
The user asks to show the week's events, starting on Sunday the 8th Jan 2011
The app would :
For each day in the week
For each event in the database
if occursOn: this day
show the event on the ui
and for our event 'My Event', it would implement occursOn: like
- (BOOL)occursOn:(NSDate *)date
is this date before this event starts
if it is, return NO
set remembered = NO
for each occurance
does this occurance say 'yes','no' or '?' for this date?
if 'yes' set remembered YES
if 'no' return NO
if '?' continue the loop
return remembered
Because WeeklyOccurace only knows that it occurs on Fridays, it would return 'yes' for Fridays and '?' for all other days so the ui would show 'My Event' on Friday and not on any other days.
To add different types of occurance, just implement the <Occurance> protocol in different ways.
(3)
The user says actually it should be on every Friday apart from the 22nd
The app would create another Occurance, this time a NotOnThisDayOccurance and add it to the end of the Event's array i.e.
Event
description : 'My Event',
start date : 1st Jan 2011
occurances :
WeeklyOccurance
day : Friday
NotOnThisDayOccurance
day: 22nd Jan 2011
Now, if the users asks to display the weekly events, 'My Event' would look do this :
Ask the WeeklyOccurance if it's valid for friday the 22nd - this would return yes.
Ask the NotOnThisDayOccurance if it's valid for friday the 22nd - this would override the previous result and say NO
Therefore, the event would not show up on the 22nd but would show up on all the other fridays.
I have this list of recurrent events on a table at my app. Something like iCal. The user can set up events like:
meeting next tuesday 3 pm
meet wife every wednesday 6 pm
take pills every day 1 pm
take children saturday and sunday 9 am
The question is: how do I sort this to discover which will be the next event?
I am not sure the best way to create the core data entities.
Should I create the days of weeks as a second managed object on core data and create a to-many relationship to the first managed object representing the events (so each event can have one or more days of week objects). Should I create one managed object with boolean entities representing the days of week?
I can imagine different ways of doing this on core data, but I am not seeing how one can extract from core data a list sorted by event, I mean, a list that shows first the events close to today's date and last the event more distant.
What I ask here is not code, but rather, ideas on how this should be constructed... what do you guys suggest as the best way to put this together.
thanks.
What I would do is have your "Event" entity in Core Data have a date and a recurrence attribute to it. Set the date to when the event occurs and recurrence to how many seconds until the next time it occurs. When the event happens, update the date with -dateByAddingTimeInterval with the current date and the recurrence interval.
You can then sort by date to get your next event. If you want to show the event multiple times, like for a month calendar layout, loop on -dateByAddingTimeInterval: with the recurrence interval and mark it on your view during each iteration.