azure cli not stopping on error in PS script - powershell

Boiled down to the minimum I have a Powershell script that looks like this:
$ErrorActionPreference='Stop'
az group deployment create -g ....
# Error in az group
# More az cli commands
Even though there is an error in the az group deployment create, it continues to execute beyond the error. How do I stop the script from executing on error?

Normally, the first thing to try is to wrap everything in a try...catch block.
try {
$ErrorActionPreference='Stop'
az group deployment create -g ....
# Error in az group
# More az cli commands
}
catch {
Write-Host "ERROR: $Error"
}
Aaaaand it doesn't work.
This is when you scratch your head and realize that we are dealing with Azure CLI commands and not Azure PowerShell. They are not native PowerShell commands which would honor $ErrorActionPreference, instead, (as bad as it sounds), we have to treat each Azure CLI command independently as if we were running individual programs (in the back end, the Azure CLI is basically aliases which run python commands. Ironically most of Azure PowerShell commands are just PowerShell wrappers around Azure CLI commands ;-)).
Knowing that the Azure CLI will not throw a terminating error, instead, we have to treat it like a program, and look at the return code (stored in the variable $LASTEXITCODE) to see if it was successful or not. Once we evaluate that, we can then throw an error:
az group deployment create -g ....
if($LASTEXITCODE){
Write-Host "ERROR: in Az Group"
Throw "ERROR: in Az Group"
}
This then can be implemented into a try...catch block to stop the subsequent commands from running:
try {
az group deployment create -g ....
if($LASTEXITCODE){
Write-Host "ERROR: in Az Group"
Throw "ERROR: in Az Group"
}
# Error in az group
# More az cli commands
}
catch {
Write-Host "ERROR: $Error"
}
Unfortunately this means you have to evaluate $LASTEXITCODE every single time you execute an Azure CLI command.

You may use the automatic variable $?. This contains the result of the last execution, i.e. True if succeded or False if it failed: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_automatic_variables?view=powershell-5.1#section-1
Your code would look something like this:
az group deployment create -g ....
if(!$?){
Write-Error "Your error message"
# Handle your error
}
Unfortunately, and same to #HAL9256 answer, you will need to add this code every single time you execute azure-cli

Update - PS Version 7 error trapping for Az CLI (or kubectl)
This behavior changed significantly in PowerShell v7:
https://github.com/PowerShell/PowerShell/issues/4002
https://github.com/PowerShell/PowerShell/pull/13361
I ended up using the following solution instead which is consistent with PowerShell v5/7. It has the advantage of accepting pipeline input for commands like kubectl where stdio can be used for applying configuration etc.
I used scriptblocks since this integrates nicely into existing scripts without breaking syntax:
# no error
# Invoke-Cli {cmd /c echo hi}
# throws terminating error
# (with psv7 you can now call Get-Error to see details)
# Invoke-Cli {az bad}
# outputs a json object
# Invoke-Cli {az account show} -AsJson
# applies input from pipeline to kubernetes cluster
# Read-Host "Enter some json" | Invoke-Cli {kubectl apply -f -}
function Invoke-Cli([scriptblock]$script, [switch]$AsJson)
{
$ErrorActionPreference = "Continue"
$jsonOutputArg = if ($AsJson)
{
"--output json"
}
$scriptBlock = [scriptblock]::Create("$script $jsonOutputArg 2>&1")
if ($MyInvocation.ExpectingInput)
{
Write-Verbose "Invoking with input: $script"
$output = $input | Invoke-Command $scriptBlock 2>&1
}
else
{
Write-Verbose "Invoking: $script"
$output = Invoke-Command $scriptBlock
}
if ($LASTEXITCODE)
{
Write-Error "$Output" -ErrorAction Stop
}
else
{
if ($AsJson)
{
return $output | ConvertFrom-Json
}
else
{
return $output
}
}
}
Handling command shell errors in PowerShell <= v5
Use $ErrorActionPreference = 'Stop' and append 2>&1 to the end of the statement.
# this displays regular output:
az account show
# this also works as normal:
az account show 2>&1
# this demonstrates that regular output is unaffected / still works:
az account show -o json 2>&1 | ConvertFrom-Json
# this displays an error as normal console output (but unfortunately ignores $ErrorActionPreference):
az gibberish
# this throws a terminating error like the OP is asking:
$ErrorActionPreference = 'Stop'
az gibberish 2>&1
Background
PowerShell native and non-native streams, while similar, do not function identically. PowerShell offers extended functionality with streams and concepts that are not present in the Windows command shell (such as Write-Warning or Write-Progress).
Due to the way PowerShell handles Windows command shell output, the error stream from a non-native PowerShell process is unable (by itself) to throw a terminating error in PowerShell. It will appear in the PowerShell runspace as regular output, even though in the context of Windows command shell, it is indeed writing to the error stream.
This can be demonstrated:
# error is displayed but appears as normal text
cmd /c "asdf"
# nothing is displayed since the stdio error stream is redirected to nul
cmd /c "asdf 2>nul"
# error is displayed on the PowerShell error stream as red text
(cmd /c asdf) 2>&1
# error is displayed as red text, and the script will terminate at this line
$ErrorActionPreference = 'Stop'
(cmd /c asdf) 2>&1
Explanation of 2>&1 workaround
Unless otherwise specified, PowerShell will redirect Windows command shell stdio errors to the console output stream by default. This happens outside the scope of PowerShell. The redirection applies before any errors reach the PowerShell error stream, making $ErrorActionPreference irrelevant.
The behavior changes when explicitly specified to redirect the Windows command shell error stream to any other location in PowerShell context. As a result, PowerShell is forced to remove the stdio error redirection, and the output becomes visible to the PowerShell error stream.
Once the output is on the PowerShell error stream, the $ErrorActionPreference setting will determine the outcome of how error messages are handled.
Further info on redirection and PowerShell streams
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_redirection?view=powershell-7.2

The sample from HAL9256 didn't work for me either, but I did find a workaround:
az group deployment create -g ....
if($error){
Write-Host "ERROR: in Az Group"
#throw error or take corrective action
$error.clear() # optional - clear the error
}
# More az cli commands
I used this to try to deploy a keyvault, but in my environment, soft delete is enabled, so the old key vault is kept for a number of days. If the deployment failed, I'd run a key vault purge and then try the deployment again.

Building from HAL9256's and sam's answers, add this one-liner after each az cli or Powershell az command in a Powershell script to ensure that the catch block is hit when an error occurs:
if($error){ throw $error }
Then for the catch block, clear the error, e.g.
catch {
$ErrorMessage = $_.Exception.Message
$error.Clear()
Write-Warning "-- Operation Failed: $ErrorMessage"
}

Related

Powershell exit does not stop script from running

I'm running a Powershell script from a Powershell window.
Inside this script, I'm trying to connect to a service, and wish to stop the script if the connection fails on an exception.
I'm catching the exception but the script refuses to stop, but continues on.
I tried checking error output using **-ErrorAction** and **-ErrorVariable** and many other things.
But the issue is the script does not stop. The script is saved in a .ps1 file and I just run it from the shell windoe".\script.ps1"
Here's the code:
Write-Host "Attaching to cluster and retrieving credential" -ForegroundColor Gray
if ([string]::IsNullOrEmpty($clusterrg) -or [string]::IsNullOrEmpty($clustername)) {
Write-Host "Failed to attacth to cluster. Parameters missing." -ForegroundColor Red
Write-Host "Use -clusterreg CLUSTER_RESOURCE_GROUP -clustername CLUSTER_NAME in the command line`n" -ForegroundColor Red
Exit 1
} else {
try{
az aks get-credentials --resource-group $clusterrg --name $clustername
Write-Host "Done`n" -ForegroundColor Green
} catch {
Write-Error $Error[0]
exit 1
}
}
Would appreciate any help.
Thanks you!
You are using the Azure CLI, not the Azure PowerShell commands. So try/catch as well as the common parameters -ErrorAction and -ErrorVariable are not supported. You have to check the $LASTEXITCODE variable for errors.
az aks get-credentials --resource-group $clusterrg --name $clustername
if( 0 -ne $LASTEXITCODE ) {
Write-Error "Azure CLI failed with exit code $LASTEXITCODE"
exit 1
}
You can find possible Azure CLI exit codes documented here.

How to determine if Write-Host will work for the current host

Is there any sane, reliable contract that dictates whether Write-Host is supported in a given PowerShell host implementation, in a script that could be run against any reasonable host implementation?
(Assume that I understand the difference between Write-Host and Write-Output/Write-Verbose and that I definitely do want Write-Host semantics, if supported, for this specific human-readable text.)
I thought about trying to interrogate the $Host variable, or $Host.UI/$Host.UI.RawUI but the only pertinent differences I am spotting are:
in $Host.Name:
The Windows powershell.exe commandline has $Host.Name = 'ConsoleHost'
ISE has $Host.Name = 'Windows PowerShell ISE Host'
SQL Server Agent job steps have $Host.Name = 'Default Host'
I have none of the non-Windows versions installed, but I expect they are different
in $Host.UI.RawUI:
The Windows powershell.exe commandline returns values for all properties of $Host.UI.RawUI
ISE returns no value (or $null) for some properties of $Host.UI.RawUI, e.g. $Host.UI.RawUI.CursorSize
SQL Server Agent job steps return no values for all of $Host.UI.RawUI
Again, I can't check in any of the other platforms
Maintaining a list of $Host.Name values that support Write-Host seems like it would be bit of a burden, especially with PowerShell being cross-platform now. I would reasonably want the script to be able to be called from any host and just do the right thing.
Background
I have written a script that can be reasonably run from within the PowerShell command prompt, from within the ISE or from within a SQL Server Agent job. The output of this script is entirely textual, for human reading. When run from the command prompt or ISE, the output is colorized using Write-Host.
SQL Server jobs can be set up in two different ways, and both support capturing the output into the SQL Server Agent log viewer:
via a CmdExec step, which is simple command-line execution, where the Job Step command text is an executable and its arguments, so you invoke the powershell.exe executable. Captured output is the stdout/sterr of the process:
powershell.exe -Command x:\pathto\script.ps1 -Arg1 -Arg2 -Etc
via a PowerShell step, where the Job Step command text is raw PS script interpreted by its own embedded PowerShell host implementation. Captured output is whatever is written via Write-Output or Write-Error:
#whatever
Do-WhateverPowershellCommandYouWant
x:\pathto\script.ps1 -Arg1 -Arg2 -Etc
Due to some other foibles of the SQL Server host implementation, I find that you can emit output using either Write-Output or Write-Error, but not both. If the job step fails (i.e. if you throw or Write-Error 'foo' -EA 'Stop'), you only get the error stream in the log and, if it succeeds, you only get the output stream in the log.
Additionally, the embedded PS implementation does not support Write-Host. Up to at least SQL Server 2016, Write-Host throws a System.Management.Automation.Host.HostException with the message A command that prompts the user failed because the host program or the command type does not support user interaction.
To support all of my use-cases, so far, I took to using a custom function Write-Message which was essentially set up like (simplified):
$script:can_write_host = $true
$script:has_errors = $false
$script:message_stream = New-Object Text.StringBuilder
function Write-Message {
Param($message, [Switch]$iserror)
if ($script:can_write_host) {
$private:color = if ($iserror) { 'Red' } else { 'White' }
try { Write-Host $message -ForegroundColor $private:color }
catch [Management.Automation.Host.HostException] { $script:can_write_host = $false }
}
if (-not $script:can_write_host) {
$script:message_stream.AppendLine($message) | Out-Null
}
if ($iserror) { $script:has_errors = $true }
}
try {
<# MAIN SCRIPT BODY RUNS HERE #>
}
catch {
Write-Message -Message ("Unhandled error: " + ($_ | Format-List | Out-String)) -IsError
}
finally {
if (-not $script:can_write_host) {
if ($script:has_errors) { Write-Error ($script:message_stream.ToString()) -EA 'Stop' }
else { Write-Output ($script:message_stream.ToString()) }
}
}
As of SQL Server 2019 (perhaps earlier), it appears Write-Host no longer throws an exception in the embedded SQL Server Agent PS host, but is instead a no-op that emits nothing to either output or error streams. Since there is no exception, my script's Write-Message function can no longer reliably detect whether it should use Write-Host or StringBuilder.AppendLine.
The basic workaround for SQL Server Agent jobs is to use the more-mature CmdExec step type (where Write-Output and Write-Host both get captured as stdout), but I do prefer the PowerShell step type for (among other reasons) its ability to split the command reliably across multiple lines, so I am keen to see if there is a more-holistic, PowerShell-based approach to solve the problem of whether Write-Host does anything useful for the host I am in.
Just check if your host is UserInteractive or an service type environment.
$script:can_write_host = [Environment]::UserInteractive
Another way to track the output of a script in real time is to push that output to a log file and then monitor it in real time using trace32. This is just a workaround, but it might work out for you.
Add-Content -Path "C:\Users\username\Documents\PS_log.log" -Value $variablewithvalue

Service cannot be started due to the following error: Cannot open service on computer '.'

I'm trying to Start a service using Powershell and using the code below
function CheckServiceStatus {
param($winupdate)
$getservice = Get-Service -Name $winupdate
if($getservice.Status -ne $running){
stop-service $winupdate
Start-Service $winupdate
Write-output "Starting" $winupdate "service"|out-file "C:\Users\Mani\Desktop\abc.txt"
Add-Content C:\Users\Mani\Desktop\abc.txt $getservice.Status
}
}
Read-Host -Prompt "Press Enter to exit"
#Variables
$winupdate = 'vsoagent.192.Shalem'
$running = 'Running'
CheckServiceStatus $winupdate
I got the following error:
Service 'ABC' cannot be stopped due to the following error: Cannot
open ABC service on computer '.'
I found some link here on our forum but couldn't resolve. Please suggest
If you want to start/stop/etc services, you need elevated privileges (run as Admin). You can still get information about the service, but that's it. If you want to include a more verbose error, include this:
$isadmin = [bool](([System.Security.Principal.WindowsIdentity]::GetCurrent()).groups -match "S-1-5-32-544")
if($isadmin){
Write-Error "You need elevated privileges to run this script"
exit(1)
}
...
Rest of your code
Or even better, if you're running Powershell 4.0 or higher (You can get it by checking $PSVersionTable), you can include
#Requires -RunAsAdministrator
At the top of your file, and you won't be able to run it without admin privilages.
I had this issue while running the command on PowerShell ISE. All I did was start the PowerShell ISE as an administrator.
Look into Event Viewer and find more details. In my case, I have found relevant info in Administrative Events, then Service Control Manager. The error was related to insufficient privileges given for the account. The service was creating a new file and this task failed. Of course, your error's details are probabably different, but that is the tip.

Jenkins powershell plugin always builds successfully

I'm using Jenkins PowerShell plugin to build a project.
However, I found that Jenkins always considers my build successful no matter what I type inside Windows PowerShell command.
Here's an example:
As you can see, asdf isn't a legal command. Jenkins should give me FAILURE after the build.
But the console output gives me:
Started by user admin
Building in workspace C:\Users\Administrator\.jenkins\jobs\Test\workspace
[workspace] $ powershell.exe -NonInteractive -ExecutionPolicy ByPass "& 'C:\Users\ADMINI~1\AppData\Local\Temp\hudson2092642221832331776.ps1'"
The term 'asdf' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\ADMINI~1\AppData\Local\Temp\hudson2092642221832331776.ps1:1 char:5
+ asdf <<<<
+ CategoryInfo : ObjectNotFound: (asdf:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
Finished: SUCCESS
I think the execution result of PowerShell should depend on $lastexitcode.
Is this a bug of PowerShell plugin?
As of 1.3, the plugin will not handle exceptions such as those from missing commands. You can do this yourself with try/catch:
try
{
asdf
}
catch
{
write-host "Caught an exception"
exit 1
}
See MSDN for more.
For me, I wanted the script to stop and fail in Jenkins soon as it hit an error. This was accomplished by adding this to the start of the script:
$ErrorActionPreference = "Stop"
This is discussed here: [How to stop a PowerShell script on the first error?][1]
[1]: How to stop a PowerShell script on the first error?. ..................
Per the latest version of the plugin (Version 1.3 Sept 18 2015), you must use $LastExitCode to fail a build.
Version 1.3 (Sept 18 2015)
PowerShell now runs in Non-Interactive mode to prevent interactive prompts from hanging the build
PowerShell now runs with ExcecutionPolicy set to "Bypass" to avoid execution policy issues
Scripts now exit with $LastExitCode, causing non-zero exit codes to mark a build as failed
Added help and list of available environment variables (including English and French translations)
I want to add here that I just ran into a quirk: you must have the powershell script end with exit and not return.
My jenkins pipe looked like:
script {
result = powershell(returnStatus: true, script: '''...if(error condition) { return 1 }''')
if(result) { error }
}
I was using if(error condition) { return 1 } and while the 1 was showing up as the return value in the jenkins console, it was not failing the build. When i used if(error condition) { exit 1 }, the build failed as expected.
I think this is a helpful addition to this thread - the need to use exit and not return. But I don't understand this part: the pipe is checking for result to be non-zero. What is the difference between exit and return in a powershell directive that makes if(result) { error } not work as expected when using return?
Update Feb-16-2021: Coming back to this to add some more notes from my experience: I think what I was doing and what a lot of people do that confuses things, is to use returnStdout or returnStatus and not check them and/or not fully understand what's coming back.
If you use either of those params, Jenkins will not do anything for you based on their value. You have to check them yourself and act accordingly. On the other hand, if you don't use them, Jenkins will recognize a failure code and fail the pipe if one comes back.
Think about it: If you set returnStatus, you're getting the exit code back from the step as a return value and not as something for Jenkins itself to worry about. If you set returnStdout, you're getting the stdout stream - and not any error codes or anything from stderr. So you must check the return for what you want or you will not get the behavior you are expecting.
What I've been doing for a while is actually not setting either of those params and making sure to set $ErrorActionPreference = 'Stop' at the start of any and all PowerShell scripts running in my pipes. That way any powershell failures automatically fail the pipe as expected, without having to check it.
This is how I implemented RRIROWER's solution. Hope it helps.
<yourscript>.ps1; exit $lastexitcode
Make sure your powershell scripts does exit with the desired value.
Run "exit <value>" as the last line.
Ultimately, I had to resort to the following configuration in Jenkins as none of the solutions here worked for me. Chris Nelson's answer got me on the right track. We're invoking chef-client remotely so we had to do a little magic to get the remote PS session to talk the local and then pass status on to Jenkins.
$res gives the output of chef-client.
$lastsuccess is true or false according to PS rules of engagment.
Of course, you'll have to supply your own evironment variables! :)
Write-host "Deploying $env:Computer with $env:Databag data bag... "
$secstr = ConvertTo-SecureString $env:Password -AsPlainText -Force
$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $env:User, $secstr
$s = New-PSSession -ComputerName $env:Computer -Credential $cred
$res = Invoke-Command -Session $s -ScriptBlock { try {chef-client} catch {exit 1}}
$lastsuccess = Invoke-Command -Session $s -ScriptBlock {$?}
Remove-PSSession $s
write-host " --- "
write-host $res
write-host " --- "
if($lastsuccess)
{
write-host "chef deployment completed"
exit 0
}
write-host "chef deployment had errors"
exit 1

How do you get powershell script output in the deployment log for a vNext Release Template?

This blog post is the only thing I have found that comes close to the problem but it doesn't explain how to configure the Deploy Using PS/DSC to run with the verbose option:
http://nakedalm.com/create-log-entries-release-management/
I can get this Agent-based Release Template to run the script:
Write-Debug "debug"
Write-Output "output"
Write-Verbose "verbose"
Write-Warning "warning"
The drilling down into deployment log for this release provides a log with the lines:
output
WARNING: warning
If I add -verbose to the Arguments field I also get a "VERBOSE: verbose" line in the log.
This is great, but I need the access to the System Variables ($Stage, $BuildNumber, etc). When I create a vNext template to run the same script (instructions are here: http://www.visualstudio.com/en-us/get-started/deploy-no-agents-vs.aspx), the log reports:
Copying recursively from \\vsalm\Drops2\TestBuild\TestBuild_20130710.3 to c:\Windows\DtlDownloads\my vnext component succeeded.
It is nice that this copying operation succeeded and all, but I'd like my script's output to be in this log as well. Does anyone have any idea about configuring a "Deploy Using PS/DSC" action so that the powershell script output is captured by Release Management?
For a vNext Release Template, try Write-Verbose with a -verbose switch if you want to see the powershell script output in the logs.
Eg. Write-Verbose "Some text" -verbose
Allow me to shamelessly plug my own blog article about this subject, because I found that it's not easy to get a script that does everything right.
The following script skeleton ensures that stdout output is logged without empty lines, and that processing is halted on the first error, in which case both the error details and the stdout output upto that point are visible in MSRM:
function Deploy()
{
$ErrorActionPreference = "Stop"
try
{
#
# Deployment actions go here.
#
}
catch
{
# Powershell tracks all Exceptions that occured so far in $Error
Write-Output "$Error"
# Signal failure to MSRM:
$ErrorActionPreference = "Continue"
Write-Error "Error: $Error"
}
}
pushd $Global:ApplicationPath
Deploy | Out-String | Write-Verbose -Verbose
popd
This is just the final result, the explanation behind it can be found here.