I'm relatively new to powershell scripting so I have been coding based on multiple examples that I have seen online.
I have a script that executes multiple batch files in parallel and each batch file contains a bcp command to execute. I'm trying to catch any errors that may occur running the batch file but it's not working as expected. I specifically forced an error on product.bat by having an invalid select syntax.
workflow Test-Workflow
{
Param ([string[]] $file_names)
$file_names = Get-Content "D:/EDW/data/informatica/ming/Powersh/bcplist.lst"
foreach -parallel ($line in $file_names)
{
try
{
Write-Output ("processing... " + $line + ".bat")
start-process D:/EDW/data/informatica/ming/Powersh/$line.bat -ErrorAction Stop -wait
}
catch
{
$ErrorMessage = $_.Exception.Message
$FailedItem = $_.Exception.ItemName
Write-Output $line : $ErrorMessage $FailedItem
}
}
}
bcplist.lst:
ing_channel
ing_product
ing_channel:
bcp "SELECT * FROM CHANNEL" queryout ing_channel.txt -T -S99.999.999.9,99999 -t"\t" -c -q
ing_product:
bcp "SELT * FROM PRODUCT" queryout ing_product.txt -T -S99.999.999.9,99999 -t"\t" -c -q
Any help or suggestion would be greatly appreciated.
Exceptions are only thrown/caught when terminating errors are thrown, which are only thrown by cmdlets, .NET libraries, or native code when P/Invoke is in play. In order to handle failures with external commands, such as checking whether a bat or exe succeeded, you will need to check the $LASTEXITCODE yourself. $LASTEXITCODE is the PowerShell equivalent of %ERRORLEVEL% in cmd.exe. Here is an example of some basic boilerplate code to check this for the ping command:
&ping nonexistant.domain.tld
if( $LASTEXITCODE -ne 0 ){
# Handle the error here
# This example writes to the error stream and throws a terminating error
Write-Error "Unable to ping server, ping returned $LASTEXITCODE" -EA Stop
}
Note that the -ErrorAction argument has a shorthand of -EA, so either the long or short form will work.
Related
Is there any sane, reliable contract that dictates whether Write-Host is supported in a given PowerShell host implementation, in a script that could be run against any reasonable host implementation?
(Assume that I understand the difference between Write-Host and Write-Output/Write-Verbose and that I definitely do want Write-Host semantics, if supported, for this specific human-readable text.)
I thought about trying to interrogate the $Host variable, or $Host.UI/$Host.UI.RawUI but the only pertinent differences I am spotting are:
in $Host.Name:
The Windows powershell.exe commandline has $Host.Name = 'ConsoleHost'
ISE has $Host.Name = 'Windows PowerShell ISE Host'
SQL Server Agent job steps have $Host.Name = 'Default Host'
I have none of the non-Windows versions installed, but I expect they are different
in $Host.UI.RawUI:
The Windows powershell.exe commandline returns values for all properties of $Host.UI.RawUI
ISE returns no value (or $null) for some properties of $Host.UI.RawUI, e.g. $Host.UI.RawUI.CursorSize
SQL Server Agent job steps return no values for all of $Host.UI.RawUI
Again, I can't check in any of the other platforms
Maintaining a list of $Host.Name values that support Write-Host seems like it would be bit of a burden, especially with PowerShell being cross-platform now. I would reasonably want the script to be able to be called from any host and just do the right thing.
Background
I have written a script that can be reasonably run from within the PowerShell command prompt, from within the ISE or from within a SQL Server Agent job. The output of this script is entirely textual, for human reading. When run from the command prompt or ISE, the output is colorized using Write-Host.
SQL Server jobs can be set up in two different ways, and both support capturing the output into the SQL Server Agent log viewer:
via a CmdExec step, which is simple command-line execution, where the Job Step command text is an executable and its arguments, so you invoke the powershell.exe executable. Captured output is the stdout/sterr of the process:
powershell.exe -Command x:\pathto\script.ps1 -Arg1 -Arg2 -Etc
via a PowerShell step, where the Job Step command text is raw PS script interpreted by its own embedded PowerShell host implementation. Captured output is whatever is written via Write-Output or Write-Error:
#whatever
Do-WhateverPowershellCommandYouWant
x:\pathto\script.ps1 -Arg1 -Arg2 -Etc
Due to some other foibles of the SQL Server host implementation, I find that you can emit output using either Write-Output or Write-Error, but not both. If the job step fails (i.e. if you throw or Write-Error 'foo' -EA 'Stop'), you only get the error stream in the log and, if it succeeds, you only get the output stream in the log.
Additionally, the embedded PS implementation does not support Write-Host. Up to at least SQL Server 2016, Write-Host throws a System.Management.Automation.Host.HostException with the message A command that prompts the user failed because the host program or the command type does not support user interaction.
To support all of my use-cases, so far, I took to using a custom function Write-Message which was essentially set up like (simplified):
$script:can_write_host = $true
$script:has_errors = $false
$script:message_stream = New-Object Text.StringBuilder
function Write-Message {
Param($message, [Switch]$iserror)
if ($script:can_write_host) {
$private:color = if ($iserror) { 'Red' } else { 'White' }
try { Write-Host $message -ForegroundColor $private:color }
catch [Management.Automation.Host.HostException] { $script:can_write_host = $false }
}
if (-not $script:can_write_host) {
$script:message_stream.AppendLine($message) | Out-Null
}
if ($iserror) { $script:has_errors = $true }
}
try {
<# MAIN SCRIPT BODY RUNS HERE #>
}
catch {
Write-Message -Message ("Unhandled error: " + ($_ | Format-List | Out-String)) -IsError
}
finally {
if (-not $script:can_write_host) {
if ($script:has_errors) { Write-Error ($script:message_stream.ToString()) -EA 'Stop' }
else { Write-Output ($script:message_stream.ToString()) }
}
}
As of SQL Server 2019 (perhaps earlier), it appears Write-Host no longer throws an exception in the embedded SQL Server Agent PS host, but is instead a no-op that emits nothing to either output or error streams. Since there is no exception, my script's Write-Message function can no longer reliably detect whether it should use Write-Host or StringBuilder.AppendLine.
The basic workaround for SQL Server Agent jobs is to use the more-mature CmdExec step type (where Write-Output and Write-Host both get captured as stdout), but I do prefer the PowerShell step type for (among other reasons) its ability to split the command reliably across multiple lines, so I am keen to see if there is a more-holistic, PowerShell-based approach to solve the problem of whether Write-Host does anything useful for the host I am in.
Just check if your host is UserInteractive or an service type environment.
$script:can_write_host = [Environment]::UserInteractive
Another way to track the output of a script in real time is to push that output to a log file and then monitor it in real time using trace32. This is just a workaround, but it might work out for you.
Add-Content -Path "C:\Users\username\Documents\PS_log.log" -Value $variablewithvalue
My snippet is something like this:
$msg=Remove-Item -Recurse -Force C:\users\bkp 2>&1
if ($LASTEXITCODE -eq 1)
{
"Encountered error during Deleting the Folder. Error Message is $msg. Please check." >> $LogFile
exit
}
The folder C:\users\bkp does not exist. Even though $msg gives me the error message $LASTEXITCODE is still 0. How do I capture as a flag?
You can use the $? automatic variable to determine the result of the last command. If you need access to the actual error, you can use the $Error automatic variable. The first item in the array is the last error thrown:
Remove-Item -Recurse -Force C:\users\bkp 2>&1
if( -not $? )
{
$msg = $Error[0].Exception.Message
"Encountered error during Deleting the Folder. Error Message is $msg. Please check." >> $LogFile
exit
}
$LASTEXITCODE is strictly for command line programs to return their status. Cmdlets that are built into PS, such as Remove-item return their errors in up to 3 ways. For warnings, they write messages (or other .NET objects) to the "warning stream". In PSv3 there is a straightforward way to redirect that stream to a file: cmdlet blah blah blah 3>warning.out. The second is via the error stream. That stream can be redirected as well ... 2>error.out, or more typically errors are caught with try/catch or trap, or written to a variable with the -ErrorVariable parameter (see help about_commonparameters). The third way is for errors to be "thrown". Unless caught (try/catch or trap), a thrown error will cause the script to terminate. Thrown errors generally are subclasses of the .NET class system.Management.Automation.ErrorRecord. An ErrorRecord provides a lot more information about an error than a return code.
If remove-item fails due to a file not found error, it writes a System.Management.Automation.ItemNotFoundException to the error stream. Using a try/catch you can filter for that specific error or other specific errors from remove-item. If you are just typing in PS commands from the command line you can enter $error[0]|select-object * to get a lot of info on the last error.
You could do this:
try {
Remove-Item -Recurse -Force C:\users\bkp 2>&1
} catch {
# oops remove-item failed. Write warning then quit
# replace the following with what you want to do
write-warning "Remove-item encounter error: $_"
return # script failed
}
I am calling sqlcmd from PowerShell to execute a T-SQL script. Currently I am using ":On Error exit" to exit the script if there is an error caused by the data used violating a constraint etc. This is handled by PowerShell detecting the $SqlcmdProcess.ExitCode of 1.
However, if there is a connectivity issue with the database, sqlcmd also gives an ExitCode of 1. Is there a way to set the :On Error ExitCode to something other than 1? I'm aware of using something like :Exit(SELECT 2) to do this, but I'd rather still use :On Error so I don't have to rewrite the script.
You could use the exit keyword in Powershell. Here's an example
Create a script called sqlcmdexit.ps1, with something like the following:
$result = sqlcmd -S"missing" -d master -Q "select ##servername"
if ($result[1] -like "*Error Locating Server/Instance Specified*" -or $result[1] -like "*Could not open a connection to SQL Server*") {
exit 99
}
Call script and observe exist code:
C:\Users\Public\bin>.\sqlcmdExit.ps1
Sqlcmd: Error: Microsoft SQL Server Native Client 10.0 : A network-related or instance-specific error has occurred while
establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and i
f SQL Server is configured to allow remote connections. For more information see SQL Server Books Online..
Sqlcmd: Error: Microsoft SQL Server Native Client 10.0 : Login timeout expired.
C:\Users\Public\bin>$LASTEXITCODE
99
I'm not aware of any way to set default ExitCode. Using start-process you could something similar:
$tempFile = [io.path]::GetTempFileName()
$exitCode = (Start-Process -FilePath "sqlcmd.exe" -ArgumentList #"
-S"missing" -d master -Q "select ##servername"
"# -Wait -NoNewWindow -RedirectStandardOutput $tempFile -Passthru).ExitCode
if ($exitCode -eq 1) {
$result = get-content $tempfile
if ($result[1] -like "*Error Locating Server/Instance Specified*" -or $result[1] -like "*Could not open a connection to SQL Server*") {
remove-item $tempFile
Exit 99
}
}
else {
remove-item $tempfile
Exit $exitCode
}
I want to run 15 instances of a script that pipelines 5 scripts together, and so far I'm missing the pixie dust. I've boiled the problem down to a test case with a master script that calls a slave script that in turn calls a sub_slave script (no pipelines are necessary to reproduce the failure.)
If master.ps1 calls slave.ps1 each background job hangs indefinitely at the point slave.ps1 calls to sub_slave.ps1. If I comment out the call in slave.ps1 to sub_slave.ps1, master.ps1 runs to completion. And if I start slave.ps1 directly, it runs the call to sub_slave.ps1 just fine. The problem seems intractable, though if I try the double-chain. I've not seen anything in the docs saying you cannot daisy chain these scripts past some arbitrary depth, but maybe I'm not reading deeply enough?
You'll see I'm tracking progress through the script by add-content to a simple text file.
d:\jobs\master.ps1
$indexes = #(0,1,2)
$initString = "cd $pwd"
$initCmd = [scriptblock]::create($initString)
set-content -path out.txt -value "Master started"
foreach ($index in $indexes)
{
add-content -path out.txt -value "job starting"
start-job -filepath slave.ps1 -InitializationScript $initCmd -ArgumentList #("master index $index originated")
}
add-content -path out.txt -value "master completed"
d:\jobs\slave.ps1
#requires -version 2.0
param (
[parameter(Mandatory=$false)]
[string]$message = "slave_originated"
)
begin
{
Set-StrictMode -version Latest
add-content -path out.txt -value "slave beginning in $pwd"
}
process
{
add-content -path out.txt -value "slave processing $message"
invoke-expression "D:\jobs\sub_slave.ps1 -message $message"
}
end
{
add-content -path out.txt -value "slave ending"
}
d:\jobs\sub_slave.ps1
#requires -version 2.0
param (
[parameter(Mandatory=$false)]
[string]$message = "sub_slave originated"
)
begin
{
Set-StrictMode -version Latest
add-content -path out.txt -value "sub_slave beginning"
}
process
{
add-content -path out.txt -value "sub_slave processing $message"
}
end
{
add-content -path out.txt -value "sub_slave ending"
}
When the code is run as written, without anything commented out, I get this in out.txt:
d:\jobs\out.txt
Master started
job starting
job starting
job starting
master completed
slave beginning in D:\jobs
slave processing master index 0 originated
slave beginning in D:\jobs
slave processing master index 1 originated
slave beginning in D:\jobs
slave processing master index 2 originated
Note that the very next command after each farewell message is the call to sub_slave.ps1 and that no message from sub_slave.ps1 appears at all. The system will hang with 3 powershell.exe processes running happily forever. Get-job reports the 3 processes are still Running and HasMoreData.
The only clue I've got is that if I crash dump the hung processes, I see:
Number of exceptions of this type: 2
Exception MethodTable: 79330e98
Exception object: 012610fc
Exception type: System.Threading.ThreadAbortException
Message: <none>
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131530
-----------------
Number of exceptions of this type: 2
Exception MethodTable: 20868118
Exception object: 01870344
Exception type: System.Management.Automation.ParameterBindingException
Message: System error.
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131501
-----------------
Perhaps I'm having some sort of parameter issue? If I remove all parameters from sub_slave.ps1 the behaviour is unchanged, so I kind of doubt that, but it's possible. I'm open to any idea.
You are calling sub_slave.ps1 wrong. This:
invoke-expression "D:\jobs\sub_slave.ps1 -message $message"
Should be this:
D:\jobs\sub_slave.ps1 -message $message
The message was getting evaluated and became multiple arguments.
Just taking a guess here, but having multiple threads/jobs write to the same file out.txt could be causing issues.
I have a simple powershell script that gets ran daily to compress and move some log files. How can i test that the command completes successfully before deleting the original log file.
set-location $logpath1
& $arcprg $pram $dest_file $source_file
Move-Item $dest_file $arcdir
If the Move-Item completes ok i want to remove-item $source_file
The completion status of the previous command can be accessed via the special variable $?.
Note that this works best with non-terminating errors (like you would get from Move-Item). Terminating errors are the result of a direct throw or an exception getting thrown in .NET and they alter the flow of your code. Best to use a trap or try/catch statement to observe those type of errors.
One other thing to watch out for WRT $? and console exes is that PowerShell assumes an exit code of 0 means success (i.e. $? is set to $true) and anything else means failure ($? set to $false). Unfortunately not all console exe's observe that exit code convention e.g. there may be multiple success codes and a single failure code (0). For those exes that don't follow the exit code rules, use $LastExitCode as pointed out in the comments to determine success or failure.
Depending on how parnoid you are and what component you are using for archiving, you can check the archive to confirm the file exixts. We use DotNetZip component to zip our archive log files (http://dotnetzip.codeplex.com/).
$zipFileObj = new-object Ionic.Zip.ZipFile($zipName);
[void] $zipFileObj.UpdateFile( "$fileName", "" ) # adds file if doesn't already exist
trap #catch an zip errors and Stop processing
{
write-error "Caught a system exception. Execution stopped"
write-error $("TRAPPED: " + $_.Exception.Message);
exit
}
if ( $zipFileObj.ContainsEntry( $fileName) )
{
remove-item $pathFile # delete file from file-system
}
else
{
# throw error
}