I have a relatively large spreadsheet (300 rows, 30 columns) that I color based on the values in the spreadsheet. I'm doing accessing the API minimally using only two accesses:
getValues(...) to access all the values of the data range.
setBackgrounds(...) to set all the backgrounds of the data range.
This runs in about half a second or less. However, it gets in the way if I make it run on every edit using onEdit(), but I also don't want it to be updated at regular time intervals when I'm not editing it, seems like a waste. Is there a good way to make the script run in a "delayed" way, updating at regular time intervals while I'm editing?
Firstly, I would say you should look at Google Sheets' conditional formatting (Format > Conditional formatting menu item in Sheets) -- you may be able to do much of what you need without involving Apps Script at all.
Failing that, you can set up a regular time-based trigger to check for edits and change the backgrounds appropriately. You can support this trigger with a separate onEdit() trigger to record what has changed internally. The flow goes like this:
A change is made and onEdit() triggers
The onEdit() trigger only records the changed cell locations to a local variable or Cache
A time-based trigger fires every minute/hour/whenever
The time-based trigger checks the cache for edited cells, alters their backgrounds, then clears them from the cache
That said, depending on your workflow this approach may not be much better than simply using a time trigger to change the cells directly.
Related
I'm working with code that uses a tumbling window of one day, and would like to send early results to a different DataStream on an hourly basis.
I understand that triggers are a way to go here, but don't really see how it would work.
The current code is as follows:
myStream
.keyBy(..)
.window(TumblingEventTimeWindows.of(Time.days(1)))
.aggregate(new MyAggregateFunction(), new MyProcessWindowFunction())
In my understanding, I should register a trigger, and then on its onEventTime method get a hold of a TriggerContext and I can send data to the labeled output from there. But how do I get the current state of MyAggregateFunction from there? Or would I need to my own computation here inside of onEventTime()?
Also, the documentation states that "By specifying a trigger using trigger() you are overwriting the default trigger of a WindowAssigner.". Would my one day window then still fire correctly, or do I need to trigger it somehow differently?
Another way of doing this is creating two different operators - one that windows by 1 hour, and another that windows by 1 day. Would triggers be a preferred approach to that?
Rather than using a custom Trigger, it would be simpler to have two layers of windowing, where the hourly results are further aggregated into daily results. Something like this:
hourlyResults = myStream
.keyBy(...)
.window(TumblingEventTimeWindows.of(Time.hours(1)))
.aggregate(new MyAggregateFunction(), new MyProcessWindowFunction())
dailyResults = hourlyResults
.keyBy(...)
.window(TumblingEventTimeWindows.of(Time.days(1)))
.aggregate(new MyAggregateFunction(), new MyProcessWindowFunction())
hourlyResults.addSink(...)
dailyResults.addSink(...)
Note that the result of a window is not a KeyedStream, so you will need to use keyBy again, unless you can arrange to leverage reinterpretAsKeyedStream (docs).
Normally when I get to more complex behavior like this, I use a KeyedProcessFunction. You can aggregate (and save in state) hourly and daily results, set timers as needed, and use a side output for the hourly results versus the regular output for the daily results.
There are quite a few questions here. I will try to ask all of them. First of all, if You specify Your own trigger using trigger() this means You are going to effectively override the default trigger and thus the window may not work the way it would by default. So, if You for example if You create the 1 day event time tumbling window, but override a trigger so that it fires for every 20th element, it will never fire based on event time.
Now, after Your custom trigger fires, the output from MyAggregateFunction will be passed to MyProcessWindowFunction, so It will work the same as for the default trigger, you don't need to access the MyAggregateFunction from inside the trigger.
Finally, while it may be technically possible to implement trigger to trigger partial results every hour, my personal opinion is that You should probably go with the two separate windows. While this solution may create a slightly larger overhead and may result in a larger state, it should be much clearer, easier to implement, and finally much more error resistant.
I am deleting row from a sheet, On a sheet I have daily job which needs to recognize the deleted records, I need a way to recognize them using smartsheet api or sdk..
Thanks in advance..
I don't believe this scenario (identifying deleted rows) is explicitly supported by the API at this time. Seems like you could still use the API to achieve your goal though, with a bit more work (code) on your part.
Your code would have to get the sheet data (i.e., all sheet rows) at a regular interval and save that data somewhere -- then each time job runs, get the sheet data again and compare that data to the data you saved the previous time the job ran (to identify any rows that had been deleted).
Edit 9/26: Added Webhooks info
Note that with the approach I've described above, any rows that had been added AND deleted during the interval between job runs would not be detected. If it's important to identify each and every time a row is deleted, a better (and much more efficient) approach would be to use Webhooks. By using webhooks, your application subscribes to notifications for a specified sheet, and then would receive a callback (HTTP POST) from Smartsheet any time the sheet changes. Your application would need to inspect the information in each callback it receives to identify 'deleted row' events (eventType = deleted and objectType = row).
A simple way to do this is to add a column with a checkmark named "delete" or something similar, then with automation you can move the row to another sheet when the flag is detected, the row will be removed from the original sheet, but you will have a record of the deleted row in a different sheet that you can read or do what ever you need to do, this will also prevent deletions by mistake and you can even restore the row back if you need to. I don't think you need much code to implement this solution.
Central to a project I'm working on is a highlighting-mechanic that can be applied to certain items on the website. The idea is, that this highlighted-status is only active for a certain amount of time.
I'm trying to find the most performant way to achieve this (in querying, setting status, checking status and revoking it)
A first approach would be to set simply set a value 'highlighted:true' to the item. This seems to be the most performant way to query for highlighted items. The Drawback I see here, is that there also needs to be stored a date for the highlighting-action, but furthermore there needs to run an interval to check on the highlighted items and potentially revoke their highlighted status. Also the exact moment when the item stops beeing highlighted can't be determined exactly, since its depending on the interval of the check-function.
A second approach would be to mainly store the date of the highlighting-action and run the query against it. It seems that the query of highlighted objects is way less performant, since every item ever is beeing checked, and on top its not just a boolean, but a proper function that throws those differnt date-values around to check if it is still valid. On the upside there is no external cleanup-function neccessary and every highlighting period ends perfectly on time.
Would love to have your input on this. Is there maybe a clever pattern on this?
I have a spreadsheet that acquires some table data using the IMPORTHTML function, and for the first two days I was using it (refreshing twice daily) things were going fine. As of this morning, it is absolutely crawling. Went from taking ~15 seconds to load 30 rows to taking ~10 minutes. Can somebody lend aid on this?
Example formula:
=IMPORTHTML(
"http://www.muthead.com/16/players/prices/1508-markus-wheaton/playstation-4","table",2
)
As mentioned, the first couple of days it was able to refresh and process a list of 30 without any pauses. Now I get the 'Executing script' message for about ten minutes before it begins to do anything, and I haven't touched the source code since origin. I'm not sure what contributes to the performance of the IMPORTHTML statement...
I've run into similar loading issues when using IMPORTHTML, IMPORTDATA, etc. The best solution I've found is to write a trigger that will edit your formula so it is forced to refresh every hour or so.
Open up the script editor and put this in. Change 'A1' with the cell your IMPORTHTML function is in, and change foo to the URL you're trying to import.
function refreshData() {
var range = SpreadsheetApp.getActiveSpreadsheet().getRange('A1');
range.clear();
range.setFormula('=IMPORTHTML(foo)');
}
Then go to Edit > Current Project Trigger > Add Trigger, and set a refresh interval.
Hope this helps.
Please correct me if I am wrong.
What I know about triggers is that they are triggered by events (Insert, Update, Delete). So we can run a stored procedure etc.. in the trigger.
This will give the application a good responsiveness because the query that the user interacts with, is quite small and this "other" longer time taking stuffs are taken care by the server internally as a separate task.
But I do not know about how the the triggers are handled inside the server. What I exactly want to know is what would happen in scenarios as given below.
Take Insert after trigger. And take trigger is executing a longer stored procedure. Then in the middle of the trigger there can be another insert. What I want to know is what will happen to that second trigger. If possible can I make that second trigger ignore itself.
marc_s has given the correct answer. I will copy it for the sake of completeness.
TRIGGERS ARE SYNCHRONOUS
If you want to have a asynchronous functionality go for a SQL broker implimenation.
Triggers are triggered by events - and then they are executed - right now. Since you cannot control when and how often they are triggered, you should keep the processing in those triggers to an absolute minimum - I always try to make - at most - an entry into another table (an "Audit" table) or possibly put a "marker" row into a "command" table. But the actual processing of that info - running stored procedures etc. - should be left to an outside job - don't do extensive processing in a trigger! This will reliably KILL all your performance\responsiveness.