I'm running a Powershell script from a Powershell window.
Inside this script, I'm trying to connect to a service, and wish to stop the script if the connection fails on an exception.
I'm catching the exception but the script refuses to stop, but continues on.
I tried checking error output using **-ErrorAction** and **-ErrorVariable** and many other things.
But the issue is the script does not stop. The script is saved in a .ps1 file and I just run it from the shell windoe".\script.ps1"
Here's the code:
Write-Host "Attaching to cluster and retrieving credential" -ForegroundColor Gray
if ([string]::IsNullOrEmpty($clusterrg) -or [string]::IsNullOrEmpty($clustername)) {
Write-Host "Failed to attacth to cluster. Parameters missing." -ForegroundColor Red
Write-Host "Use -clusterreg CLUSTER_RESOURCE_GROUP -clustername CLUSTER_NAME in the command line`n" -ForegroundColor Red
Exit 1
} else {
try{
az aks get-credentials --resource-group $clusterrg --name $clustername
Write-Host "Done`n" -ForegroundColor Green
} catch {
Write-Error $Error[0]
exit 1
}
}
Would appreciate any help.
Thanks you!
You are using the Azure CLI, not the Azure PowerShell commands. So try/catch as well as the common parameters -ErrorAction and -ErrorVariable are not supported. You have to check the $LASTEXITCODE variable for errors.
az aks get-credentials --resource-group $clusterrg --name $clustername
if( 0 -ne $LASTEXITCODE ) {
Write-Error "Azure CLI failed with exit code $LASTEXITCODE"
exit 1
}
You can find possible Azure CLI exit codes documented here.
Related
I have 2 scripts, one serves as a Worker - uses variables to complete commands, second serves as a function Library. Worker via variable loads Library and uses functions from there.
As a user, when I run the script I would like to see the output in the console which I defined as the outcome for Library script.
Example Worker script:
Param(
[string]$server_01,
[string]$releaseDefinitionName,
[string]$pathRelease,
[string]$buildNumber,
[string]$command_01,
[string]$scheduledTask_01
)
$pathScriptLibrary = $pathRelease + "\" + $buildNumber + "\" + "_scripts"
. $pathScriptLibrary\_library.ps1
$user = xxx
$password = xxx
$cred = xxx
Invoke-Command -ComputerName $server_01 -Credential $cred -ErrorAction Stop -ScriptBlock {powershell $command_01}
Example Library script:
function Stop-ScheduledTasks{
Write-Output [INFO]: Stopping scheduled tasks... -ForegroundColor White
Get-ScheduledTask -TaskName "$scheduledTask_01" | ForEach {
if ($_.State -eq "Ready") {
Write-Output [WARNING]: Scheduled task $scheduledTask_01 was already stopped. -ForegroundColor Yellow
}
else {
Stop-ScheduledTask -TaskName "$scheduledTask_01"
Write-Output [OK]: Running task $scheduledTask_01 stopped. -ForegroundColor Green
}
}
}
function Start-ScheduledTasks{
Write-Output [INFO]: Starting scheduled tasks... -ForegroundColor White
Get-ScheduledTask -TaskName "$scheduledTask_01" | ForEach {
if ($_.State -eq "Running") {
Write-Output [WARNING]: Scheduled task $scheduledTask_01 already started. -ForegroundColor Yellow
}
else {
Start-ScheduledTask -TaskName "$scheduledTask_01"
Write-Output [OK]: Stopped scheduled task $scheduledTask_01 started. -ForegroundColor Green
}
}
}
Use case:
User starts the deployment by clicning the deploy button in Azure DevOps UI
The task using the Worker script takes function from Library script (in this case stops Scheduled Task) and performs it
User checks log on the Azure DevOps side and sees the custom output lines from Library script. (2 of them now - 1. starting with [INFO], 2. either starting with [WARNING] or [OK]).
Could you please advice a solution how to achieve that? Thank you.
NOTE: Those examples are run in Azure DevOps (on premise) release pipelines and desired outcomes are ment for users running those pipelines.
If you're trying to write to the azure devops pipeline log, then you should avoid using Write-Output. That does something subtly different; it adds to the function's return value.
So for example the Write-Output in the function Stop-ScheduledTask; that is roughly equivalent to you putting at the end of the function:
return "[WARNING]: Scheduled task $scheduledTask_01 was already stopped."
That might end up being printed to the pipeline log, or it might not; and importantly, it might completely mess up a function which is genuinely trying to return a simple value.
Instead of using Write-Output, I recommend using Write-Host. What that does is immediately write a line to the pipeline log, without affecting what a library function will return.
Write-Output "[WARNING]: Scheduled task $scheduledTask_01 was already stopped."
You can also use Write-Warning and Write-Error.
In order to scale Function Apps quickly we want to be able to deploy them via IaC and then deploy a code package onto it. Unfortunately this is not possible dynamically with YAML pipelines in Azure DevOps so I had to resort to using the Azure CLI.
Below you see the PowerShell script I came up with to deploy the code into the pool of Function Apps that I deployed through Terraform before-hand. To speed things up I turned on parallel processing of the ForEach-Object loop since there are no dependencies between the single instances. This also works fine to a certain extent but I am having troubles due to the quirkiness of the Azure CLI. Writing non-error information to StdErr seems to be by design. This combined with some other strange behavior leads to the following scenarios:
Running sequentially usually works flawlessly and I see any error output if a problem occurs. Also I don't need to set powerShellErrorActionPreference: 'continue'. This of course is slowing down the deployment significantly.
Running in parallel fails always without setting powerShellErrorActionPreference: 'continue'. The reason for the failure is not output to the console. This seems to happen even if no real error occurs as with continue there is no error output to the console as well. This wouldn't be an issue if the pipeline fails in the case of a real error (which should be handled by checking the state of the ChildJobs - but it doesn't.
So here I am between a rock and a hard place. Does anyone see the flaw in my implementation? Any suggestions are highly appreciated.
- task: AzureCLI#2
displayName: 'Functions deployment'
env:
AZURE_CORE_ONLY_SHOW_ERRORS: 'True'
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
ARM_CLIENT_ID: $(AzureApplicationId)
ARM_CLIENT_SECRET: $(AzureApplicationSecret)
ARM_SUBSCRIPTION_ID: $(AzureSubscriptionId)
ARM_TENANT_ID: $(AzureTenantId)
inputs:
azureSubscription: 'MySubscription'
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
Write-Output -InputObject "INFO: Get Function App names"
$appNames = terragrunt output -json all_functionapp_names | ConvertFrom-Json
Write-Output -InputObject "INFO: Loop over Function Apps"
$jobs = $appNames | ForEach-Object -Parallel {
$name = $_
try
{
Write-Output -InputObject "INFO: $name`: start slot"
az functionapp start --resource-group $(ResourceGroup) --name "$name" --slot Stage --verbose
Write-Output -InputObject "INFO: $name`: deploy into slot"
az functionapp deploy --resource-group $(ResourceGroup) --name "$name" --slot Stage --src-path "$(System.ArtifactsDirectory)/drop/MyCodePackage.zip" --type zip --verbose
Write-Output -InputObject "INFO: $name`: deploy app settings"
az functionapp config appsettings set --resource-group $(ResourceGroup) --name "$name" --slot Stage --settings "#$(Build.ArtifactStagingDirectory)/appsettings.json" --verbose
Write-Output -InputObject "INFO: $name`: swap slot with production"
az functionapp deployment slot swap --resource-group $(ResourceGroup) --name "$name" --slot Stage --action swap --verbose
}
catch
{
Write-Output -InputObject "ERROR: $name`: An error occured during deployment"
Write-Output -InputObject ($_.Exception | Format-List -Force)
}
finally
{
try
{
Write-Output -InputObject "INFO: $name`: stop slot"
az functionapp stop --resource-group $(ResourceGroup) --name "$name" --slot Stage --verbose
}
catch
{
Write-Output -InputObject "ERROR: $name`: could not stop slot"
}
}
} -AsJob
[int]$pollingInterval = 10
[int]$elapsedSeconds = 0
while ($jobs.State -eq "Running") {
$jobs.ChildJobs | ForEach-Object {
Write-Output -InputObject "---------------------------------"
Write-Output -InputObject "INFO: $($_.Name) output [$($elapsedSeconds)s]"
Write-Output -InputObject "---------------------------------"
$_ | Receive-Job
Write-Output -InputObject "---------------------------------"
Write-Output -InputObject ""
}
$elapsedSeconds += $pollingInterval
[Threading.Thread]::Sleep($pollingInterval * 1000)
}
$jobs.ChildJobs | Where-Object { $_.JobStateInfo.State -eq "Failed" } | ForEach-Object {
Write-Output -InputObject "ERROR: At least one of the deployments failed with the following reason:"
Write-Output -InputObject $_.JobStateInfo.Reason
}
if ($jobs.State -eq "Failed")
{
exit 1
}
else
{
exit 0
}
powerShellErrorActionPreference: 'continue'
workingDirectory: './infrastructure/environments/$(TerraFormEnvironmentName)'
Edit 1
To get all output from ChildJobs I had to alter the code like so:
[int]$pollingInterval = 10
[int]$elapsedSeconds = 0
$lastResultsRead = false
while ($jobs.State -eq "Running" -or !$lastResultsRead)
{
$lastResultsRead = $jobs.State -ne "Running"
$jobs.ChildJobs | ForEach-Object {
Write-Output -InputObject "---------------------------------"
Write-Output -InputObject "INFO: $($_.Name) output [$($elapsedSeconds)s]"
Write-Output -InputObject "---------------------------------"
$_ | Receive-Job
Write-Output -InputObject "---------------------------------"
Write-Output -InputObject ""
}
$elapsedSeconds += $pollingInterval
if (!$lastResultsRead)
{
[Threading.Thread]::Sleep($pollingInterval * 1000)
}
Hope this helps everyone that wants to achieve something similar.
So it seems that the mystery is solved.
TLDR;
If you want proper error handling, remove the --verbose from all Azure CLI calls as the verbose output is always written to StdErr even when setting the environment variable AZURE_CORE_ONLY_SHOW_ERRORS.
Explanation
I stumbled over the solution by adding an unrelated functionality to this script and noticed that in certain situations the last output of the ChildJobs is not being collected. I initially took that for a quirk of the Azure DevOps task but discovered that this also happens when I debug the output locally in VSCode.
That led me to add another condition for the while loop that would ensure to give me the final output. I'll update the script in my initial post accordingly. Finally equipped with the whole picture of what is going on in the ChildJobs I set up a separate test pipeline where I would run different test cases to find the culprit. Soon enough I noticed that taking away --verbose prevents the task from failing. This happened with AZURE_CORE_ONLY_SHOW_ERRORS set or not. So I gave the --only-show-errors option a go, which should have the same result as the environment variable though only on a single Azure CLI call. Due to the full output now at my disposal I could finally see the message that --verbose and --only-show-errors can't be used in conjunction. That settled it. --verbose had to go. All it adds is the information of how long the command ran anyway. I think we can do without it.
On an additional side-note: at the same time I discovered that ForEach-Object -Parallel {} -AsJob is making heavy use of PowerShell runspaces. That means that it cannot be debugged from within VSCode in the typical way. I found a video that might help in situations like this: https://www.youtube.com/watch?v=O-dksknPQBw
I hope this answer helps others that stumble over the same strange behavior. Happy coding.
Boiled down to the minimum I have a Powershell script that looks like this:
$ErrorActionPreference='Stop'
az group deployment create -g ....
# Error in az group
# More az cli commands
Even though there is an error in the az group deployment create, it continues to execute beyond the error. How do I stop the script from executing on error?
Normally, the first thing to try is to wrap everything in a try...catch block.
try {
$ErrorActionPreference='Stop'
az group deployment create -g ....
# Error in az group
# More az cli commands
}
catch {
Write-Host "ERROR: $Error"
}
Aaaaand it doesn't work.
This is when you scratch your head and realize that we are dealing with Azure CLI commands and not Azure PowerShell. They are not native PowerShell commands which would honor $ErrorActionPreference, instead, (as bad as it sounds), we have to treat each Azure CLI command independently as if we were running individual programs (in the back end, the Azure CLI is basically aliases which run python commands. Ironically most of Azure PowerShell commands are just PowerShell wrappers around Azure CLI commands ;-)).
Knowing that the Azure CLI will not throw a terminating error, instead, we have to treat it like a program, and look at the return code (stored in the variable $LASTEXITCODE) to see if it was successful or not. Once we evaluate that, we can then throw an error:
az group deployment create -g ....
if($LASTEXITCODE){
Write-Host "ERROR: in Az Group"
Throw "ERROR: in Az Group"
}
This then can be implemented into a try...catch block to stop the subsequent commands from running:
try {
az group deployment create -g ....
if($LASTEXITCODE){
Write-Host "ERROR: in Az Group"
Throw "ERROR: in Az Group"
}
# Error in az group
# More az cli commands
}
catch {
Write-Host "ERROR: $Error"
}
Unfortunately this means you have to evaluate $LASTEXITCODE every single time you execute an Azure CLI command.
You may use the automatic variable $?. This contains the result of the last execution, i.e. True if succeded or False if it failed: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_automatic_variables?view=powershell-5.1#section-1
Your code would look something like this:
az group deployment create -g ....
if(!$?){
Write-Error "Your error message"
# Handle your error
}
Unfortunately, and same to #HAL9256 answer, you will need to add this code every single time you execute azure-cli
Update - PS Version 7 error trapping for Az CLI (or kubectl)
This behavior changed significantly in PowerShell v7:
https://github.com/PowerShell/PowerShell/issues/4002
https://github.com/PowerShell/PowerShell/pull/13361
I ended up using the following solution instead which is consistent with PowerShell v5/7. It has the advantage of accepting pipeline input for commands like kubectl where stdio can be used for applying configuration etc.
I used scriptblocks since this integrates nicely into existing scripts without breaking syntax:
# no error
# Invoke-Cli {cmd /c echo hi}
# throws terminating error
# (with psv7 you can now call Get-Error to see details)
# Invoke-Cli {az bad}
# outputs a json object
# Invoke-Cli {az account show} -AsJson
# applies input from pipeline to kubernetes cluster
# Read-Host "Enter some json" | Invoke-Cli {kubectl apply -f -}
function Invoke-Cli([scriptblock]$script, [switch]$AsJson)
{
$ErrorActionPreference = "Continue"
$jsonOutputArg = if ($AsJson)
{
"--output json"
}
$scriptBlock = [scriptblock]::Create("$script $jsonOutputArg 2>&1")
if ($MyInvocation.ExpectingInput)
{
Write-Verbose "Invoking with input: $script"
$output = $input | Invoke-Command $scriptBlock 2>&1
}
else
{
Write-Verbose "Invoking: $script"
$output = Invoke-Command $scriptBlock
}
if ($LASTEXITCODE)
{
Write-Error "$Output" -ErrorAction Stop
}
else
{
if ($AsJson)
{
return $output | ConvertFrom-Json
}
else
{
return $output
}
}
}
Handling command shell errors in PowerShell <= v5
Use $ErrorActionPreference = 'Stop' and append 2>&1 to the end of the statement.
# this displays regular output:
az account show
# this also works as normal:
az account show 2>&1
# this demonstrates that regular output is unaffected / still works:
az account show -o json 2>&1 | ConvertFrom-Json
# this displays an error as normal console output (but unfortunately ignores $ErrorActionPreference):
az gibberish
# this throws a terminating error like the OP is asking:
$ErrorActionPreference = 'Stop'
az gibberish 2>&1
Background
PowerShell native and non-native streams, while similar, do not function identically. PowerShell offers extended functionality with streams and concepts that are not present in the Windows command shell (such as Write-Warning or Write-Progress).
Due to the way PowerShell handles Windows command shell output, the error stream from a non-native PowerShell process is unable (by itself) to throw a terminating error in PowerShell. It will appear in the PowerShell runspace as regular output, even though in the context of Windows command shell, it is indeed writing to the error stream.
This can be demonstrated:
# error is displayed but appears as normal text
cmd /c "asdf"
# nothing is displayed since the stdio error stream is redirected to nul
cmd /c "asdf 2>nul"
# error is displayed on the PowerShell error stream as red text
(cmd /c asdf) 2>&1
# error is displayed as red text, and the script will terminate at this line
$ErrorActionPreference = 'Stop'
(cmd /c asdf) 2>&1
Explanation of 2>&1 workaround
Unless otherwise specified, PowerShell will redirect Windows command shell stdio errors to the console output stream by default. This happens outside the scope of PowerShell. The redirection applies before any errors reach the PowerShell error stream, making $ErrorActionPreference irrelevant.
The behavior changes when explicitly specified to redirect the Windows command shell error stream to any other location in PowerShell context. As a result, PowerShell is forced to remove the stdio error redirection, and the output becomes visible to the PowerShell error stream.
Once the output is on the PowerShell error stream, the $ErrorActionPreference setting will determine the outcome of how error messages are handled.
Further info on redirection and PowerShell streams
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_redirection?view=powershell-7.2
The sample from HAL9256 didn't work for me either, but I did find a workaround:
az group deployment create -g ....
if($error){
Write-Host "ERROR: in Az Group"
#throw error or take corrective action
$error.clear() # optional - clear the error
}
# More az cli commands
I used this to try to deploy a keyvault, but in my environment, soft delete is enabled, so the old key vault is kept for a number of days. If the deployment failed, I'd run a key vault purge and then try the deployment again.
Building from HAL9256's and sam's answers, add this one-liner after each az cli or Powershell az command in a Powershell script to ensure that the catch block is hit when an error occurs:
if($error){ throw $error }
Then for the catch block, clear the error, e.g.
catch {
$ErrorMessage = $_.Exception.Message
$error.Clear()
Write-Warning "-- Operation Failed: $ErrorMessage"
}
I'm trying to Start a service using Powershell and using the code below
function CheckServiceStatus {
param($winupdate)
$getservice = Get-Service -Name $winupdate
if($getservice.Status -ne $running){
stop-service $winupdate
Start-Service $winupdate
Write-output "Starting" $winupdate "service"|out-file "C:\Users\Mani\Desktop\abc.txt"
Add-Content C:\Users\Mani\Desktop\abc.txt $getservice.Status
}
}
Read-Host -Prompt "Press Enter to exit"
#Variables
$winupdate = 'vsoagent.192.Shalem'
$running = 'Running'
CheckServiceStatus $winupdate
I got the following error:
Service 'ABC' cannot be stopped due to the following error: Cannot
open ABC service on computer '.'
I found some link here on our forum but couldn't resolve. Please suggest
If you want to start/stop/etc services, you need elevated privileges (run as Admin). You can still get information about the service, but that's it. If you want to include a more verbose error, include this:
$isadmin = [bool](([System.Security.Principal.WindowsIdentity]::GetCurrent()).groups -match "S-1-5-32-544")
if($isadmin){
Write-Error "You need elevated privileges to run this script"
exit(1)
}
...
Rest of your code
Or even better, if you're running Powershell 4.0 or higher (You can get it by checking $PSVersionTable), you can include
#Requires -RunAsAdministrator
At the top of your file, and you won't be able to run it without admin privilages.
I had this issue while running the command on PowerShell ISE. All I did was start the PowerShell ISE as an administrator.
Look into Event Viewer and find more details. In my case, I have found relevant info in Administrative Events, then Service Control Manager. The error was related to insufficient privileges given for the account. The service was creating a new file and this task failed. Of course, your error's details are probabably different, but that is the tip.
I wrote the PowerShell script to run the SQL scripts on target database server using RoundhousE.
Example code in PowerShell script:
$outsideTransactionDir = (Join-Path $migrateDir "OutsideTransaction")
Write-Host $(Get-Item $outsideTransactionDir).FullName
if ((Test-Path $outsideTransactionDir) -eq $true)
{
Write-Verbose "Running RoundhousE (Outside Transaction)" -Verbose
Write-Host "Running RoundhousE (Outside Transaction)"
ExecuteRoundhousE $(Get-Item $outsideTransactionDir).FullName $([String]::Format('-cs "{0}" -ni -ct 500 -rb "runBeforeUp" –-debug --env PROD --vf "db_version.xml" {1}', $settings.roundhouse.connectionstring, $r))
}
Write-Verbose "Running RoundhousE" -Verbose
Write-Host "Running RoundhousE"
ExecuteRoundhousE $(Get-Item $migrateDir).FullName $([String]::Format('-cs "{0}" --trx -ni -ct 500 -rb "runBeforeUp" -sp "StoredProcedure" -fu "UserDefinedFunction" -vw "View" -ix "Index" –-debug --env PROD --vf "db_version.xml" {1}', $settings.roundhouse.connectionstring, $r))
After executing the above lines of code through the Run PowerShell on Target Server task at VSTS release. Internally one text file will be created in specified server, it contains logs (changes) related to database SQL scripts execution, and also same logs will be displayed in VSTS release logs under specified task. But the issue is total logs will not be displayed In VSTS release logs only few logs will be displayed in VSTS release logs.