I'm building a wrapper around PowerShell scripts to handle logging in one unified way. The idea is to intercept messages from information/warning/error streams and log them in chronological order using different sinks (console/file/db etc). For that purpose, I'd like to utilize events that occur when some data is added to a corresponding stream. That's where I'm now:
$ps = [PowerShell]::Create("CurrentRunspace")
$ps.AddScript({
Write-Information 'Information'
Write-Warning 'Warning'
Write-Error 'Error'
exit -1
}) # in reality here should be script that is wrapped
$output = [System.Management.Automation.PSDataCollection[object]]::new()
$ps.Streams.Information.Add_DataAdded({
# $output.Add($_) is for example, in reality, any handler could be added
$ps.Streams.Information.ReadAll().ForEach{ $output.Add($_) }
})
$ps.Streams.Warning.Add_DataAdded({
# $output.Add($_) is for example, in reality, any handler could be added
$ps.Streams.Warning.ReadAll().ForEach{ $output.Add($_) }
})
$ps.Streams.Error.Add_DataAdded({
# $output.Add($_) is for example, in reality, any handler could be added
$ps.Streams.Error.ReadAll().ForEach{ $output.Add($_) }
})
# run script
$results = $ps.Invoke()
$exitCode = $LASTEXITCODE
# these lines are just for the test
Write-Host $exitCode
Write-Host "$([string]::Join(', ', $output))"
Expected content of $output:
-1
Information
Warning
Error
Actual content of $output:
-1
Error
Do you guys know why a subscription to the Warning and Information DataAdded event doesn't work?
Related
I would like to let the user to enter a specified key (e.g.: $ ) within 5 seconds.
If the user cannot enter the specified key, it will become time out.
$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")
This method allows the user to enter a key, but the console will continue to wait for input, which is not time out. How to achieve that?
You need to check [Console]::KeyAvailable in a while loop and timeout manually.
function Read-KeyWithTimeout {
param([int] $Timeout = 1000)
end {
$stopwatch = [System.Diagnostics.Stopwatch]::StartNew()
while (-not [Console]::KeyAvailable) {
if ($stopwatch.ElapsedMilliseconds -gt $Timeout) {
throw 'Timeout hit'
}
Start-Sleep -Milliseconds 50
}
$Host.UI.RawUI.ReadKey('NoEcho, IncludeKeyDown')
}
}
When [Console]::KeyAvailable returns true there is input in the buffer waiting to be read.
Note: On Unix this will still echo the input because of how System.Console is implemented.
I have noticed that objects implementing IDisposable in advanced functions aren't reliably disposed of when a "stop" signal (eg. pressing CTRL+C) is sent during execution. This is a pain when the object holds a handle to, for example, a file. If the stop signal is received at an inopportune time, the handle doesn't get closed and the file remains locked until the PowerShell session is closed.
Consider the following class and functions:
class f : System.IDisposable {
Dispose() { Write-Host 'disposed' }
}
function g {
param( [Parameter(ValueFromPipeline)]$InputObject )
begin { $f = [f]::new() }
process {
try {
$InputObject
}
catch {
$f.Dispose()
throw
}
}
end {$f.Dispose()}
}
function throws {
param ( [Parameter(ValueFromPipeline)] $InputObject )
process { throw 'something' }
}
function blocks {
param ( [Parameter(ValueFromPipeline)] $InputObject )
process { Wait-Event 'bogus' }
}
Imagine $f holds a handle to a file and releases it when its Dispose() method is called. My goal is that the lifetime of $f matches the lifetime of g. $f is disposed correctly when g is invoked in each the following ways:
g
'o' | g
'o' | g | throws
I can tell as much because each of these outputs disposed.
When the stop signal is sent while execution is occuring downstream of g, however, $f is not disposed. To test that, I invoked
'o' | g | blocks
which blocks at the Wait-Event inside blocks, then I pressed Ctrl+C to stop execution. In that case, Dispose() does not seem to get called (or, at least disposed is not written to the console).
In C# implementations of such functions it is my understanding that StopProcessing() gets called on a stop signal to do such cleanup. However, there seems to be no analog to StopProcessing available for PowerShell implementations of advanced functions.
How can I ensure that $f is disposed in all cases including a stop signal?
You can't if the function accepts pipeline input.
I don't think a robust way of achieving this is possible if the function accepts pipeline input. The reason is that any of the following could occur while code is executing upstream in the pipeline:
break, continue, or throw
terminating error
stop signal received
When these occur upstream, no part of the function can be caused to intervene. The begin{} and process{} blocks have either run to completion or not run at all, and the end{} block may or may not be run. The closest to an on-point solution I have found is the following:
function g {
param (
[Parameter(ValueFromPipeline)]
$InputObject
)
begin { $f = [f]::new() } # The local IDisposable is created when the pipeline is established.
process {
try
{
# flags to keep track of why finally was run
$success = $false
$caught = $false
$InputObject # output an object to exercise the pipeline downstream
# if we get here, nothing unusual happened downstream
$success = $true
}
catch
{
# we get here if an exception was thrown
$caught = $true
# !!!
# This is bad news. It's possible the exception will be
# handled by an upstream process{} block. The pipeline would
# survive and the next invocation of process{} would occur
# after $f is disposed.
# !!!
$f.Dispose()
# rethrow the exception
throw
}
finally
{
# !!!
# This finally block is not invoked when the PowerShell instance receives
# a stop signal while executing code upstream in the pipeline. In that
# situation cleanup $f.Dispose() is not invoked.
# !!!
if ( -not $success -and -not $caught )
{
# dispose only if finally{} is the only block remaining to run
$f.Dispose()
}
}
}
end {$f.Dispose()}
}
However, per the comments there are still cases where $f.Dispose() is not invoked. You can step through this working example that includes such cases.
Consider a pattern like usingObject {} instead.
If we limit usage to the case where the function responsible for cleanup does not accept pipeline input, then we can factor out the lifetime-management logic into a helper function similar to C#'s using block. Here is a proof-of-concept that implements such a helper function called usingObject. This is an example of how g could be substantially simplified when using usingObject to achieve robust invokation of .Dispose():
# refactored function g
function g {
param (
[Parameter(ValueFromPipeline)]
$InputObject,
[Parameter(Mandatory)]
[f]
$f
)
process {
$InputObject
}
}
# usages of function g
usingObject { [f]::new() } {
g -f $_
}
usingObject { [f]::new() } {
'o' | g -f $_
}
try
{
usingObject { [f]::new() } {
'o' | g -f $_ | throws
}
}
catch {}
usingObject { [f]::new() } {
'o' | g -f $_ | blocks
}
Seems like you can just add a finally{} block and dispose it there. You might also want to consider setting your ErrorActionPreference, since you are doing custom error handling.
$ErrorActionPreference = 'SilentlyContinue'
try
{
try
{
1/0
}
catch
{
throw New-Object System.Exception("Exception!")
}
finally
{
"You can dispose here!"
}
}
catch
{
$_.Exception.Message | Write-Output
}
I'm using Powershell Core v6-beta.5 using AppImage on Linux. Is there a way to find out the "new" location of a 301 redirect?
Invoke-WebRequest -Method HEAD http://SomethingThatThrows301.com/ -MaximumRedirection 0 throws an error (Response status code does not indicate success: 301 (Moved Permanently)).
While the error does mention that the move is a 301, I'd still like a proper object telling me that, and the new address.
Is there a way to do so?
Note: All code below works in both Windows PowerShell and PowerShell Core, on all supported platforms, with up to 50 redirections by default.
Assuming:
that you don't care about the specific 3xx redirection status code and
that you only need to know the ultimate target URL (there could be a chain of redirections)
use the following:
[System.Net.HttpWebRequest]::Create('http://cnn.com').GetResponse().ResponseUri.AbsoluteUri
This yields (note how the target URL has www.):
http://www.cnn.com
Below is the source code for advanced convenience function Get-UrlRedirection, which packages the functionality, offering both resolution to the ultimate target URL and an enumeration of the chain of redirection URLs.
Example calls:
> Get-UrlRedirection http://cnn.com
http://www.cnn.com
> Get-UrlRedirection -Enumerate http://microsoft.com/about
http://microsoft.com/about
https://microsoft.com/about
https://www.microsoft.com/about
https://www.microsoft.com/about/
https://www.microsoft.com/about/default.aspx
https://www.microsoft.com/en-us/about/
Function Get-UrlRedirection {
[CmdletBinding()]
Param (
[Parameter(Mandatory, ValueFromPipeline)] [Uri] $Url,
[switch] $Enumerate,
[int] $MaxRedirections = 50 # Use same default as [System.Net.HttpWebRequest]
)
process {
try {
if ($Enumerate) { # Enumerate the whole redirection chain, from input URL to ultimate target,
# assuming the max. count of redirects is not exceeded.
# We must walk the chain of redirections one by one.
# If we disallow redirections, .GetResponse() fails and we must examine
# the exception's .Response object to get the redirect target.
$nextUrl = $Url
$urls = #( $nextUrl.AbsoluteUri ) # Start with the input Uri
$ultimateFound = $false
# Note: We add an extra loop iteration so we can determine whether
# the ultimate target URL was reached or not.
foreach($i in 1..$($MaxRedirections+1)) {
Write-Verbose "Examining: $nextUrl"
$request = [System.Net.HttpWebRequest]::Create($nextUrl)
$request.AllowAutoRedirect = $False
try {
$response = $request.GetResponse()
# Note: In .NET *Core* the .GetResponse() for a redirected resource
# with .AllowAutoRedirect -eq $False throws an *exception*.
# We only get here on *Windows*, with the full .NET Framework.
# We either have the ultimate target URL, or a redirection
# whose target URL is reflected in .Headers['Location']
# !! Syntax `.Headers.Location` does NOT work.
$nextUrlStr = $response.Headers['Location']
$response.Close()
# If the ultimate target URL was reached (it was already
# recorded in the previous iteration), and if so, simply exit the loop.
if (-not $nextUrlStr) {
$ultimateFound = $true
break
}
} catch [System.Net.WebException] {
# The presence of a 'Location' header implies that the
# exception must have been triggered by a HTTP redirection
# status code (3xx).
# $_.Exception.Response.StatusCode contains the specific code
# (as an enumeration value that can be case to [int]), if needed.
# !! Syntax `.Headers.Location` does NOT work.
$nextUrlStr = try { $_.Exception.Response.Headers['Location'] } catch {}
# Not being able to get a target URL implies that an unexpected
# error ocurred: re-throw it.
if (-not $nextUrlStr) { Throw }
}
Write-Verbose "Raw target: $nextUrlStr"
if ($nextUrlStr -match '^https?:') { # absolute URL
$nextUrl = $prevUrl = [Uri] $nextUrlStr
} else { # URL without scheme and server component
$nextUrl = $prevUrl = [Uri] ($prevUrl.Scheme + '://' + $prevUrl.Authority + $nextUrlStr)
}
if ($i -le $MaxRedirections) { $urls += $nextUrl.AbsoluteUri }
}
# Output the array of URLs (chain of redirections) as a *single* object.
Write-Output -NoEnumerate $urls
if (-not $ultimateFound) { Write-Warning "Enumeration of $Url redirections ended before reaching the ultimate target." }
} else { # Resolve just to the ultimate target,
# assuming the max. count of redirects is not exceeded.
# Note that .AllowAutoRedirect defaults to $True.
# This will fail, if there are more redirections than the specified
# or default maximum.
$request = [System.Net.HttpWebRequest]::Create($Url)
if ($PSBoundParameters.ContainsKey('MaxRedirections')) {
$request.MaximumAutomaticRedirections = $MaxRedirections
}
$response = $request.GetResponse()
# Output the ultimate target URL.
# If no redirection was involved, this is the same as the input URL.
$response.ResponseUri.AbsoluteUri
$response.Close()
}
} catch {
Write-Error $_ # Report the exception as a non-terminating error.
}
} # process
}
In order to focus on the code, I've omitted the comment-based help above; here it is - simply paste it directly above the function definition:
<#
.SYNOPSIS
Gets a URL's redirection target(s).
.DESCRIPTION
Given a URL, determines its redirection target(s), as indicated by responses
with 3xx HTTP status codes.
If the URL is not redirected, it is output as-is.
By default, the ultimate target URL is determined (if there's a chain of
redirections), but the number of redirections that are followed is limited
to 50 by default, which you may change with -MaxRedirections.
-Enumerate enumerates the redirection chain and returns an array of URLs.
.PARAMETER Url
The URL whose redirection target to determine.
You may supply multiple URLs via the pipeline.
.PARAMETER MaxRedirections
Limits the number of redirections that are followed, 50 by default.
If the limit is exceeded, a non-terminating error is reported.
.PARAMETER Enumerate
Enumerates the chain of redirections, if applicable, starting with
the input URL itself, and outputs it as an array.
If the number of actual redirections doesn't exceed the specified or default
-MaxRedirections value, the entire chain up to the ultimate target URL is
enumerated.
Otherwise, a warning is issued to indicate that the ultimate target URL wasn't
reached.
All URLs are output in absolute form, even if the targets are defined as
relative URLs.
Note that, in order to support multiple input URLs via the pipeline, each
array representing a redirection chain is output as a *single* object, so
with multiple input URLs you'll get an array of arrays as output.
.EXAMPLE
> Get-UrlRedirection http://cnn.com
http://www.cnn.com
.EXAMPLE
> Get-UrlRedirection -Enumerate http://microsoft.com/about
http://microsoft.com/about
https://microsoft.com/about
https://www.microsoft.com/about
https://www.microsoft.com/about/
https://www.microsoft.com/about/default.aspx
https://www.microsoft.com/en-us/about/
.NOTES
This function uses the [System.Net.HttpWebRequest] .NET class and was
inspired by http://www.powershellmagazine.com/2013/01/29/pstip-retrieve-a-redirected-url/
#>
If you ignore the error being thrown, you will be able to examine the HTTP response. The new URL will be in the Location header.
Try something like the below.
$url="https://jigsaw.w3.org/HTTP/300/301.html"
$resp = Invoke-WebRequest -Method HEAD $url -MaximumRedirection 0 -ErrorAction Ignore
$code = $resp.StatusCode
Write-Output "URL: $url"
Write-Output "ErrorCode: $code"
if($code -eq 301) {
$loc = $resp.Headers.Location
Write-Output "New URL: $loc"
}
I have a fairly large powershell scripts with many (20+) functions which perform various actions.
Right now all of the code doesn't really have any error handling or retry functionality. If a particular task/function fails it just fails and continues on.
I would like to improve error handling and implement retries to make it more robust.
I was thinking something similar to this:
$tries = 0
while ($tries -lt 5) {
try{
# Do Something
# No retries necessary
$tries = 5;
} catch {
# Report the error
# Other error handling
}
}
The problem is that I have many many steps where I would need to do this.
I don't think it make sense to implement the above code 20 times. That seems really superfluous.
I was thinking about writing an "TryCatch" function with a single parameter that contains the actual function I want to call?
I'm not sure that's the right approach either though. Won't I end up with a script that reads something like:
TryCatch "Function1 Parameter1 Parameter2"
TryCatch "Function2 Parameter1 Parameter2"
TryCatch "Function3 Parameter1 Parameter2"
Is there a better way to do this?
If you frequently need code that retries an action a number of times you could wrap your looped try..catch in a function and pass the command in a scriptblock:
function Retry-Command {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true)]
[scriptblock]$ScriptBlock,
[Parameter(Position=1, Mandatory=$false)]
[int]$Maximum = 5,
[Parameter(Position=2, Mandatory=$false)]
[int]$Delay = 100
)
Begin {
$cnt = 0
}
Process {
do {
$cnt++
try {
# If you want messages from the ScriptBlock
# Invoke-Command -Command $ScriptBlock
# Otherwise use this command which won't display underlying script messages
$ScriptBlock.Invoke()
return
} catch {
Write-Error $_.Exception.InnerException.Message -ErrorAction Continue
Start-Sleep -Milliseconds $Delay
}
} while ($cnt -lt $Maximum)
# Throw an error after $Maximum unsuccessful invocations. Doesn't need
# a condition, since the function returns upon successful invocation.
throw 'Execution failed.'
}
}
Invoke the function like this (default is 5 retries):
Retry-Command -ScriptBlock {
# do something
}
or like this (if you need a different amount of retries in some cases):
Retry-Command -ScriptBlock {
# do something
} -Maximum 10
The function could be further improved e.g. by making script termination after $Maximum failed attempts configurable with another parameter, so that you can have have actions that will cause the script to stop when they fail, as well as actions whose failures can be ignored.
I adapted #Victor's answer and added:
parameter for retries
ErrorAction set and restore (or else exceptions do not get caught)
exponential backoff delay (I know the OP didn't ask for this, but I use it)
got rid of VSCode warnings (i.e. replaced sleep with Start-Sleep)
# [Solution with passing a delegate into a function instead of script block](https://stackoverflow.com/a/47712807/)
function Retry()
{
param(
[Parameter(Mandatory=$true)][Action]$action,
[Parameter(Mandatory=$false)][int]$maxAttempts = 3
)
$attempts=1
$ErrorActionPreferenceToRestore = $ErrorActionPreference
$ErrorActionPreference = "Stop"
do
{
try
{
$action.Invoke();
break;
}
catch [Exception]
{
Write-Host $_.Exception.Message
}
# exponential backoff delay
$attempts++
if ($attempts -le $maxAttempts) {
$retryDelaySeconds = [math]::Pow(2, $attempts)
$retryDelaySeconds = $retryDelaySeconds - 1 # Exponential Backoff Max == (2^n)-1
Write-Host("Action failed. Waiting " + $retryDelaySeconds + " seconds before attempt " + $attempts + " of " + $maxAttempts + ".")
Start-Sleep $retryDelaySeconds
}
else {
$ErrorActionPreference = $ErrorActionPreferenceToRestore
Write-Error $_.Exception.Message
}
} while ($attempts -le $maxAttempts)
$ErrorActionPreference = $ErrorActionPreferenceToRestore
}
# function MyFunction($inputArg)
# {
# Throw $inputArg
# }
# #Example of a call:
# Retry({MyFunction "Oh no! It happened again!"})
# Retry {MyFunction "Oh no! It happened again!"} -maxAttempts 10
Solution with passing a delegate into a function instead of script block:
function Retry([Action]$action)
{
$attempts=3
$sleepInSeconds=5
do
{
try
{
$action.Invoke();
break;
}
catch [Exception]
{
Write-Host $_.Exception.Message
}
$attempts--
if ($attempts -gt 0) { sleep $sleepInSeconds }
} while ($attempts -gt 0)
}
function MyFunction($inputArg)
{
Throw $inputArg
}
#Example of a call:
Retry({MyFunction "Oh no! It happend again!"})
Error handling is always going to add more to your script since it usually has to handle many different things. A Try Catch function would probably work best for what you are describing above if you want to have each function have multiple tries. A custom function would allow you to even set things like a sleep timer between tries by passing in a value each time, or to vary how many tries the function will attempt.
I am trying to make my own Jabber bot but i have run into a little trouble. I have gotten my bot to respond to messages, however, if I try to change the bot's presence then it seems as though all of the messages you send to the bot get delayed.
What I mean is when I run the script I change the presence so I can see that it is online. Then when I send it a message it takes three before the callback subroutine I have set up for messages gets called. After the thirrd message is sent and the chat subroutine is called it still process the first message I sent.
This really doesn't pose too much of a problem except that I have it set up to log out when I send the message "logout" and it has to be followed by two more messages in order to log out. I am not sure what it is that I have to do to fix this but i think it has something to do with iq packets because I have an iq callback set as well and it gets called two times after setting the presence.
Here is my source code:
#!/usr/bin/perl
use strict;
use warnings;
#Libraries
use Net::Jabber;
use DBI;
use DBD::mysql;
#--------------- Config Vars -----------------
# Jabber Client
my $jbrHostname = "DOMAINNAME";
my $jbrUserName = "USERNAME";
my $jbrPassword = "PASSWORD";
my $jbrResource = "RESOURCE";
my $jbrBoss = new Net::Jabber::JID();
$jbrBoss->SetJID(userid=>"USERNAME",server=>$jbrHostname);
# MySQL
my $dbHostname = "DOMAINNAME";
my $dbName = "DATABASENAME";
my $dbUserName = "USERNAME";
my $dbPassword = "PASSWORD";
#--------------- End Config -----------------
# connect to the db
my $dbh = DBI->connect("DBI:mysql:database=$dbName;host=$dbHostname",$dbUserName, $dbPassword, {RaiseError => 1}) or die "Couldn't connect to the database: $!\n";
# create a new jabber client and connect to server
my $jabberBot = Net::Jabber::Client->new();
my $status = $jabberBot->Connect(hostname=>$jbrHostname) or die "Cannot connect ($!)\n";
my #results = $jabberBot->AuthSend(username=>$jbrUserName,password=>$jbrPassword,resource=>$jbrResource);
if($results[0] ne "ok")
{
die "Jabber auth error #results\n";
}
# set jabber bot callbacks
$jabberBot->SetMessageCallBacks(chat=>\&chat);
$jabberBot->SetPresenceCallBacks(available=>\&welcome);
$jabberBot->SetCallBacks(iq=>\&gotIQ);
$jabberBot->PresenceSend(type=>"available");
$jabberBot->Process(1);
sub welcome
{
$jabberBot->MessageSend(to=>$jbrBoss->GetJID(),subject=>"",body=>"Hello There!",type=>"chat",priority=>10);
&keepItGoing;
}
$jabberBot->MessageSend(to=>$jbrBoss->GetJID(),subject=>"",body=>"Hello There! Global...",type=>"chat",priority=>10);
#$jabberBot->Process(5);
&keepItGoing;
sub chat
{
print "Chat Called!\n";
my ($sessionID,$msg) = #_;
$jabberBot->MessageSend(to=>$msg->GetFrom(),subject=>"",body=>"Chatting!",type=>"chat",priority=>10);
if($msg->GetBody() ne 'logout')
{
print $msg->GetBody()."\n";
&keepItGoing;
}
else
{
&killBot($msg);
}
}
sub gotIQ
{
print $_[1]->GetID()."\n";
&chat;
}
sub keepItGoing
{
print "Movin' the chains!\n";
my $proc = $jabberBot->Process(1);
while(defined($proc) && $proc != 1)
{
$proc = $jabberBot->Process(1);
}
}
sub killBot
{
$jabberBot->MessageSend(to=>$_[0]->GetFrom(),subject=>"",body=>"Logging Out!",type=>"chat",priority=>10);
$jabberBot->Process(1);
$jabberBot->Disconnect();
exit;
}
Thanks for your help!
You've got resource starvation because of your keepItGoing routine. In general, trying to use XMPP synchronously like this is not going to work. I suggest getting your callbacks set up, then just calling Process() in one loop.
The docs for Process() say:
Process(integer) - takes the timeout period as an argument. If no
timeout is listed then the function blocks until
a packet is received. Otherwise it waits that
number of seconds and then exits so your program
can continue doing useful things. NOTE: This is
important for GUIs. You need to leave time to
process GUI commands even if you are waiting for
packets. The following are the possible return
values, and what they mean:
1 - Status ok, data received.
0 - Status ok, no data received.
undef - Status not ok, stop processing.
IMPORTANT: You need to check the output of every
Process. If you get an undef then the connection
died and you should behave accordingly.
Each time you call Process(), 0 or more of your callbacks will fire. You never know which, since it depends on server timing. If you want for Process() to return before sending something, you're almost always thinking synchronously, rather than asych, which kills you in XMPP.
In your case, if you remove the call to keepItGoing from chat(), I bet things will work more like you expect.
Replace the line:
$jabberBot->Process(1);
with these:
while (defined($jabberBot->Process(1))) {
# Do stuff here
}