Currently using this script to monitor:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html
When the script is first run we receive the email, but after that the email never sends (even though the threshold has been breached continuously).
Amazon CloudWatch alarms will only trigger notifications when the state of the Alarm changes. It will not continuously send alarms when the state is Alert and it is not possible to configure such behavior.
One exception to this is triggering Auto Scaling changes -- it will continually try to trigger an Auto Scaling policy while the state is in Alarm.
This is an example of a custom metrics that reset itself every hour: Custom CloudWatch metrics with hourly reset. Therefore, the notifications are also repeated every hour, if the alarm has not been resolved.
Related
Is there an easy way to setup pagerduty to hold alerts to allow time for the cloud to self resolve?
my team often gets woken up for alerts that self resolve before we have a chance to address it. If Pagerduty could hold the alert for 5 minutes we would avoid unactionable alerts.
You can use Event Orchestration or Event Rules at a service level to control this. What you are looking for is to create an alert but pause notifications. This will suspend the alert for a time period of your choice, allowing the alert to resolve itself. If the alert doesn't resolve within the time period an incident will open as expected.
Event Orchestration (should be available on basic+ tiers)
Once you define the conditions for the Orchestration rule you want to set an incident action to pause notifications for 300 seconds (5 minutes).
https://support.pagerduty.com/docs/event-orchestration#incident-actions
If you have the event intelligence package you can also look at the auto-pause feature which detects and pauses transient alerts for you.
I have an MVC web site, a storage queue and a WebJob. Users can request the generation of a set of reports by clicking a button on the web page. This inserts a message into the storage queue. In the past, the WebJob ran continuously and processed those requests fine. But the demand and size of the reports has grown to the point where the WebJob is slowing down the web app. I would like to still place the request message in the queue, but delay processing of all requests until the evening, when the web app is mostly idle. This would allow me to continue using the WebJob code and QueueTrigger functionality without having to waste resources by moving to a dedicated Worker Role, etc. The reports don't need to be generated immediately, so a delay is acceptable.
I don't see a built-in way to set a time window on processing. The only thing I have found is a powershell cmdlet for starting and stopping WebJobs (Start-AzureWebsiteJob / Stop-AzureWebsiteJob). So I was thinking that I could create a scheduled powershell job that runs at midnight, starts the webjob, lets it run, and then runs again early in the AM and stops it.
Does anyone know of a better option than this? Anything more "official" that perhaps I could not find?
One possible solution would be to hide the messages in the queue for a certain amount of time when they are inserted.
If you're using AddMessage method, you can specify this timespan value in initialVisibilityDelay parameter.
What this will do is ensure that the messages are not immediately visible in the queue to be picked by WebJob and will become visible only when this timespan elapses.
Will such a solution work for you?
Maybe I didn't fully understand your question, but couldn't you use "Triggered" WebJob that is triggered by CRON schedule? You can then limit it to specific hours
0 * 20-22 * * *
This example will run every minute from 8pm to 10pm
I'm trying to set local notification in swift firing once at a definite date, but it seems there is no such option for repeat interval . By the default it triggers every 24 hours. Is there any way to submit a notification, which fires only once, so that user didn't have to even open the app or etc?
You have to set the repeatInterval of your UILocalNotification to 0:
If you assign a calendar unit such as weekly (NSCalendarUnitWeekOfYear) or yearly (NSCalendarUnitYear), the system reschedules the notification for delivery at the specified interval. Note that intervals of less than one minute are not supported.
The default value is 0, which means that the system fires the notification once and then discards it.
Regarding your second question: no! The user has to open the app at least once! Otherwise the app does not execute any of its code. Register the notification on first launch.
I want to disable a quartz job for a given period of time. Though we can use pauseJob/resumeJob, but the catch is on resumeJob it fires all the missed triggers(Yes, I don't want to change the default mis-fire policy, so that even if someone provides past time, job still runs).
Is there a way where we can disable a job for certain time, and on enabling it back it should NOT fire any missed triggers.
I have recently been thinking about possible architecture for a simple task reminder system. User will schedule a task and reminder in form of SMS/email/android needs to be sent to all stakeholders at some x minutes before the task is scheduled to be performed(much in the same way google calendar works). The problem here is to send the reminder at that precise point in time. Here are the two possible approaches I can think of:
Cron: I can setup a cron to run every minute. This will scan the table for notifications which need to be sent in the next minute and simply sends the notifications. But, precision is lost as there is always the chance of that +/-1 min error.
Work Queues: I can simply put a message with appropriate delay in a queue at the time task was scheduled. Workers will send the notification as and when they receive the message. I can add as many workers as I want in case my real time behavior starts getting affected because of load. There are still a few issues. How to choose the appropriate work queue? I have evaluated RabbitMq and Beanstalk. While Rabbitmq follows standard AMQP protocol and is widely suggested, it doesn't provide the delay functionality out of the box. There are ways to simulate this using dead-letter-exchanges but this will not work in my case because the delay needs to be variable. Beanstalk supports this but the problem is that beanstalk queue resides entirely in memory which I don't like(but can live with). Any possible alternatives?
Third Approach: ??????. I am sure a simple desktop notification tool does neither of the two. What technology do they use to achieve the same thing?
We had the same scenario and we use Redis for long schedules even now reminders for up to 2 years. You can use Sorted Set where the timestamp is the score.
We use Beanstalkd delay jobs for those kind of reminders where we know it's relatively short term couple of hours, and there is no cancellations, as removing from beanstalkd a delayed message you need to retain the job id in a database for later removal, and that is no viable.
Although you mention memory limit, we use persistence on both Redis/Beanstalkd