Is there a way to handle the event of a Service Fabric Node going down (NodeStatus change) in C#. Possibly an event I can listen for?
Assuming that you're running in Azure, you can use Azure Monitor Alerts to call a webhook, or a logic app to accomplish this.
For example:
In the Monitor - Alerts (classic) pane, select +Add activity log
alert. On the Add activity log alert page, select or enter the
requested information. In Criteria, make sure that you set the
following values: Event category: Select Service Health. Services:
Select Virtual Machine Scale Sets and Virtual Machines. Type: Select
Planned maintenance.
More info here and here.
Related
Azure Data Factory pipeline have been working fine for 2 years with Self-hosted Integration runtime (Azure VM 16GB)
In last few weeks pipeline have got very unreliable and data is no longer processed correctly.
Data Factory Activity using "MyAzureIntegrationRuntime" instance is failing with "timeout" error.
Some of activities get successfully completed, but most of them get failed.
ADF Monitor tell that node is "Unavailable". Typical ADF activity is Azure SQL lookup.
There is no event log errors in Virtual Machine. There seems to be enough CPU/RAM to execute IR activities.
Reboot of VM has once helped to recover connectivity and pipelines.
However latest VM reboot restored status to "Running" from “Unavailable”, but many pipeline activities get failed.
Integration Runtime is currently not in High Available Cluster.
There is single VM serving Sandbox, Dev, Test and Prod ADF. It has worked fine for last years except last 2 weeks.
How could I find what is the problem and fixed it?
How could I find what is the problem and fixed it?
For failed activities that are running on a self-hosted IR or a shared IR, the service supports viewing and uploading error logs. To get the error report ID, follow the instructions here, and then enter the report ID to search for related known issues.
On the Monitor page for the service UI, select Pipeline runs.
Under Activity runs, in the Error column, select the highlighted button to display the activity logs, as shown in the following screenshot:
The activity logs are displayed for the failed activity run.
For further assistance, select Send logs.
The Share the self-hosted integration runtime (IR) logs with Microsoft window opens.
Select which logs you want to send.
For a self-hosted IR, you can upload logs that are related to the failed activity or all logs on the self-hosted IR node.
For a shared IR, you can upload only logs that are related to the failed activity.
When the logs are uploaded, keep a record of the Report ID for later use if you need further assistance to solve the issue.
We are using Azure DevOps to deploy to a staging slot and then swap with production.
When there is an issue swapping it will keep trying for nearly 30 minutes.
Therefore I would like to put a timeout on the swap task, but if I do that it will stop the task in DevOps and leave the process happening in Azure.
I would like a way to force stop the process through a CLI, API, PowerShell or DevOps task.
Azure CLI doesn't seem to have anything
Kudu API can delete deployments but doesn't look to stop them (https://github.com/projectkudu/kudu/wiki/REST-API#deployment)
I have read that you can stop a process, but using a Linux Container App Service, I can't see that option. Azure-Web-sites: How to cancel a deployment?
Is there a way?
Please try to use the following command in Azure PowerShell task to cancel a pending swap:
Invoke-AzResourceAction -ResourceGroupName [resource group name] -ResourceType Microsoft.Web/sites/slots -ResourceName [app name]/[slot name] -Action resetSlotConfig -ApiVersion 2015-07-01
Here is the document and my sample, I added -Force -Confirm:$false at the end of the command:
Update
If any errors occur in the target slot (for example, the production slot) after a slot swap, restore the slots to their pre-swap states by swapping the same two slots immediately.
So, we don't need to stop it, just wait swap operation succeed.
When you submit swap slot request, you will get HttpStatus 202 code. On portal, when you click swap button, you will find that the browser has been requesting the url of location to get the status of swap.
As for when it ends, we can check the swap operation by polling.
If the swap operation time is too long, it is recommended to raise a support ticket and ask the engineer to check the reason.
Previous
You can use AzureAppServiceManage task.
Azure App Service Manage task
Use this task to start, stop, restart, slot swap, Swap with Preview, install site extensions, or enable continuous monitoring for an Azure App Service.
Tips
When you use rest api to swap slot, you can check location in response header.
When you submit swap slot request, you will get HttpStatus 202 code. On portal, when you click swap button, you will find that the browser has been requesting the url of location to get the status of swap.
I used the Sophos UTM 9.510 ha_standalone Cloudformation template (https://github.com/sophos-iaas/aws-cf-templates/blob/master/utm/9.510/standalone.template) and used defaults when possible. I did not use an existing ElasticIP, so it created it's own at (scrubbed) 50.12.12.123.
I gave a hostname at (for example) vpn.example.com and after creation, I created an A record for vpn.example.com to point to 50.12.12.123.
I don't have a license and just pay hourly for the AMI.
I understand that I should be able to hit https://vpn.example.com:4444 or https://50.12.12.123:4444 to see the admin panel. However, it times out and doesn't load anything.
When I deployed the stack, I got an email at the admin email I provided and it said REST daemon not running - restarted. I assume it restarted fine, since I have received no new emails, and the EC2 instance is running.
Has anyone else experienced this? Is there a step I'm missing? Aside from creating the Route53 record, I thought the Cloudformation Template should just work right out of the box.
The default security groups blocked traffic. I modified one of them to accept all traffic and the dashboard became accessible. I will now refine access further.
After creating a subscription in the Reporting Configuration, the emails are not sent at the specified time, but if you perform any actions on the site, the emails will be sent. This is the problem of the virtual machine on which the site or IIS is deployed. Or does the feature Kentico.
Prompt in what there can be a problem and how the system of sending of emails inin the Reporting Configuration works.
This is a correct behavior. Kentico checks for e-mails that are to be sent at the end of requests which means that if there are no requests, no e-mails are sent. If you need some tasks to be send at a specified time, you need to use Windows scheduler and configure task (e-mmail sending) to use it. See official documentation for more detail.s
Another option to Enn's, is to use a service like UpTime Robot to continually visit your page (like every 5 minutes), this not only generates a request but also will help keep your site awake and from going to sleep if you can't manually set the Worker Processor to never go to sleep.
Windows Scheduler is the most reliable, but UpTime robot has worked well for us.
https://uptimerobot.com/
Under the "Scheduled Tasks" module there is a schedule task called "Report subscription sender". This task runs every minute and is responsible for checking your subscriptions and sending e-mails based on your configuration. This task runs within the Kentico instance. By default IIS will put the site/app pool to sleep after x (default 20) minutes of idle and the scheduled tasks are no longer able to run. When you hit the site it wakes the process back up and the scheduled task is able to run again. You can go into IIS and configure the "Idle Time-out (minutes)" for the application pool see this link https://patrickdesjardins.com/blog/iis-no-sleep-idle-and-autostart-with-values for a pretty good illustration. You can also adjust the app pool recycle intervals but that is probably not necessary for your issue.
The other option, as mentioned by Enn, is to install the Kentico Scheduler Windows service which always runs and configure that scheduled task to run in the service.
I'm running an application that executes Windows Workflows as a service with some custom activities in them. When I run the activities as a logged on user in a shell that invokes the workflows with the activities, the activities have no problems performing as expected. When the activities get invoked while the user is running the service (e.g. not logged on, at least not graphically) the activities fail with "Access Denied" errors. What's the difference between running the code as a logged on user vs running it as a service user ? Is it that when it runs as a service, the user's profile doesn't get loaded ? If so, how can I force the user's profile to get loaded when running as a service ?
Specifically, the activities I'm working with perform remote management with classes in the System.Management.Automation namespace (i.e. powershell).
A bit late answer, but in case someone else is wondering.
I had a similar problem earlier and have done some research. The following is untested, but may be of help to others.
The difference between a process running as "Windows Service" and under a desktop environment is what Window Station it is assigned to. Normally a service runs under a different station than desktop users.
I say normally because a service can have the "SERVICE_INTERACTIVE_PROCESS" flag set. This will give it a desktop window station. See this article and this article for a more detailed explanation. Setting the "SERVICE_INTERACTIVE_PROCESS" can be done in the CreateService (C# use of that) call or manually by checking the "Allow service to interact with desktop" checbox under "LogOn" under properties of the specific service.