Incrementing a number each time it replaces a string - powershell

So I working with a script that looks like this. It works just fine the only issue is that it is counting the line numbers and I just wanted it to replace num each time with 1 2 3 4 etc insted it is looking like this 5 20 25 etc and that is because it seems to be counting and incremnting $c for each line not each time it replaces the string num.
$c=0
(Get-Content C:\Users\H\Desktop\3.txt) |
Foreach-Object {$_ -replace "num", $((++$c))} |
Out-File C:\Users\H\Desktop\4.txt

Try this:
$c = [ref] 0
$text = Get-Content 'C:\Users\H\Desktop\3.txt' -Raw
[regex]::Replace( $text, 'num', { (++$c.Value) } ) |
Set-Content 'C:\Users\H\Desktop\4.txt'
# With PowerShell 6+ you could write:
# (Get-Content 'C:\Users\H\Desktop\3.txt' -Raw) -replace 'num', { (++$c.Value) } |
# Set-Content 'C:\Users\H\Desktop\4.txt'
Copy-pastable demo:
$text = #'
num foo num
bar num bar
num baz num
'#
$c = [ref] 0
[regex]::Replace( $text, 'num', { (++$c.Value) } )
# With PowerShell 6+ you could write:
# $text -replace 'num', { (++$c.Value) }
Output:
1 foo 2
bar 3 bar
4 baz 5
Explanation:
Use a reference type ([ref]) variable instead of a plain variable. The scriptblock passed to [regex]::Replace() or -replace (PS 6+ only) runs in a child scope, so it can't modify variables from the parent scope directly. You can modify the members of a reference type, so this is why the [ref] trick works. Instead of [ref] you could also use a [PSCustomObject] or a [Hashtable] (which are both reference types) with a counter member.
The parenthesis around the expression ++$c.Value are required to output the value of the expression, which doesn't produce output by default. You already had that, I'm just explaining it for other visitors.
Using Get-Content with parameter -Raw can be faster, because the file's content gets output as a single multiline string instead of splitting it up into one string per line, at the expense of requiring more memory.
As for what you have tried:
$_ -replace "num", $((++$c))
You are passing an expression instead of a script block as the RHS argument to the -replace operator. Furthermore, a script block argument for the -replace operator is only supported beginning with PowerShell 6. For older versions you have to use the .NET function [Regex]::Replace.
This expression is evaluated once for each iteration of the ForEach-Object "loop", before the -replace operator is evaluated. So you are effectively just counting the lines of the file, not the number of occurences of the pattern.
Only a script block's execution can be delayed. It doesn't get called immediately in the place where you define it1, but when the function or operator that has a [ScriptBlock] parameter decides to call it, possibly multiple times as happens when the pattern has multiple matches. To define a script block, use {} as I did in my sample at the beginning.
[1] Unless you use the call or dot source operator, e.g. &{'foo'}

Related

How to make ForEach-Object start counting from 1 instead of 0

I have a nice string that looks like this:
Get-Azsubscription | ForEach-Object {"$($a).) $($_.Name)"; $a++}
It allows me to count how many subscriptions I have on Azure and it adds a number to it. The problem is that the counter starts from 0, which is in my case .) :
How to make that counter start to count from 1 and not from 0?
The ForEach-Object cmdlet takes an additional -Begin block you can use to initialize any variable you want to use:
Get-Azsubscription | ForEach-Object -Begin { $a = 1 } -Process {"$a.) $($_.Name)"; $a++}
To complement Mathias R. Jessen's effective solution:
Indeed you should explicitly initialize your $a variable with the desired starting value.
If $a is uninitialized, as it was in your case:
Using it inside an expandable string ("$a") makes it expand to the empty string.
The first time you apply ++ to it, it is fist implicitly initialized (as an [int] with) value 0, so that it receives value 1. Subsequent ++ operations then perform as usual.
While using a -Begin block with ForEach-Object in order to initialize the variable is conceptually clear, the caveat is that the variable is not scoped to that statement, and lives on afterwards.
Thus, the alternative is to simply initialize it before calling ForEach-Object.
You can also streamline the command a bit by incorporating the ++ operation into the expandable string, but note that you then need to enclose it in (...), in addition to $(...); the reason is that a ++ operation produces no output by default, except if you apply (...), the grouping operator, to pass the value through.
Therefore:
$a = 0 # Initialize the sequence number
Get-Azsubscription | ForEach-Object { "$((++$a)).) $($_.Name)" }
Note the initialization to 0 in combination with the prefix version of ++, the increment operator, so that incrementing to 1 and outputting that value happens on the first iteration.
A simplified example:
$a = 0 # Initialize the sequence number
[pscustomobject] #{ Name='foo' }, [pscustomobject] #{ Name='bar' } |
ForEach-Object { "$((++$a)).) $($_.Name)" }
Output:
1.) foo
2.) bar
Finally, a perhaps more PowerShell-idiomatic solution that constructs custom objects for display, which PowerShell automatically formats nicely for you:
$a = 0 # Initialize the sequence number
[pscustomobject] #{ Name='foo' }, [pscustomobject] #{ Name='bar' } |
ForEach-Object { [pscustomobject] #{ '#' = ++$a; Name = $_.Name } }
Output:
# Name
- ----
1 foo
2 bar

How to get 2 variables from a file PowerShell

I'm trying to make an script that take numbers from a file and then do some maths with them, but I don't know how to take from a single line 2 variables.
The file it must be something like this:
3 5
What I need is that one variable is for example $a be 3 and other $b be 5
$a=3
$b=5
The problem is that I found this
$Contents = Get-Content ".\file.txt"
$a = $Contents[0] -split(" ")[1]
$b = $Contents[1] -split(" ")[1]
but it doesen´t work with the second number, how can I do this?
To refer to the first line in the file, you want $Contents[0] ($Contents[1] would refer to the second line).
$a,$b = -split $Contents[0] -as [int[]]
Using -split in unary mode will make PowerShell split on any sequence of consecutive whitespace characters, and throw away any empty parts (this way it works when the iput has leading or trailing whitespace, like " 3 5 ").
The -as [int[]] operation will force PowerShell to attempt to convert the resulting string values to [int] values, so now you can meaningfully do integer arithmetic with them:
PS ~> $a + $b
8

How do I change foreach to for in PowerShell?

I want to print the word exist in a text file and print "match" and "not match". My 1st text file is: xxaavv6J, my 2nd file is 6J6SCa.yB.
If it is match, it return like this:
Match found:
Match found:
Match found:
Match found:
Match found:
Match found: 6J
Match found:
Match found:
Match found:
My expectation is just print match and not match.
$X = Get-Content "C:\Users\2.txt"
$Data = Get-Content "C:\Users\d.txt"
$Split = $Data -split '(..)'
$Y = $X.Substring(0, 6)
$Z = $Y -split '(..)'
foreach ($i in $Z) {
foreach ($j in $Split) {
if ($i -like $j) {
Write-Host ("Match found: {0}" -f $i, $j)
}
}
}
The operation -split '(..)' does not produce the result you think it does. If you take a look at the output of the following command you'll see that you're getting a lot of empty results:
PS C:\> 'xxaavv6J' -split '(..)' | % { "-$_-" }
--
-xx-
--
-aa-
--
-vv-
--
-6J-
--
Those empty values are the additional matches you're getting from $i -like $j.
I'm not quite sure why -split '(..)' gives you any non-empty values in the first place, because I would have expected it to produce 5 empty strings for an input string "xxaavv6J". Apparently it has to do with the grouping parentheses, since -split '..' (without the grouping parentheses) actually does behave as expected. Looks like with the capturing group the captured matches are returned on top of the results of the split operation.
Anyway, to get the behavior you want replace
... -split '(..)'
with
... |
Select-String '..' -AllMatches |
Select-Object -Expand Matches |
Select-Object -Expand Value
You can also replace the nested loop with something like this:
foreach ($i in $Z) {
if (if $Split -contains $i) {
Write-Host "Match found: ${i}"
}
}
A slightly different approach using regex '.Match()' should also do it.
I have added a lot of explaining comments for you:
$Test = Get-Content "C:\Users\2.txt" -Raw # Read as single string. Contains "xxaavv6J"
$Data = (Get-Content "C:\Users\d.txt") -join '' # Read as array and join the lines with an empty string.
# This will remove Newlines. Contains "6J6SCa.yB"
# Split the data and make sure every substring has two characters
# In each substring, the regex special characters need to be Escaped.
# When this is done, we join the substrings together using the pipe symbol.
$Data = ($Data -split '(.{2})' | # split on every two characters
Where-Object { $_.Length -eq 2 } | # don't care about any left over character
ForEach-Object { [Regex]::Escape($_) } ) -join '|' # join with the '|' which is an OR in regular expression
# $Data is now a string to use with regular expression: "6J|6S|Ca|\.y"
# Using '.Match()' works Case-Sensitive. To have it compare Case-Insensitive, we do this:
$Data = '(?i)' + $Data
# See if we can find one or more matches
$regex = [regex]$Data
$match = $regex.Match($Test)
# If we have found at least one match:
if ($match.Groups.Count) {
while ($match.Success) {
# matched text: $match.Value
# match start: $match.Index
# match length: $match.Length
Write-Host ("Match found: {0}" -f $match.Value)
$match = $match.NextMatch()
}
}
else {
Write-Host "Not Found"
}
Result:
Match found: 6J
Further to the excellent Ansgar Wiechers' answer: if you are running (above) Windows PowerShell 4.0 then you could apply the .Where() method described in Kirk Munro's exhaustive article ForEach and Where magic methods:
With the release of Windows PowerShell 4.0, two new “magic” methods
were introduced for collection types that provide a new syntax for
accessing ForEach and Where capabilities in Windows PowerShell.
These methods are aptly named ForEach and Where. I call
these methods “magic” because they are quite magical in how they work
in PowerShell. They don’t show up in Get-Member output, even if you
apply -Force and request -MemberType All. If you roll up your
sleeves and dig in with reflection, you can find them; however, it
requires a broad search because they are private extension methods
implemented on a private class. Yet even though they are not
discoverable without peeking under the covers, they are there when you
need them, they are faster than their older counterparts, and they
include functionality that was not available in their older
counterparts, hence the “magic” feeling they leave you with when you
use them in PowerShell. Unfortunately, these methods remain
undocumented even today, almost a year since they were publicly
released, so many people don’t realize the power that is available in
these methods.
…
The Where method
Where is a method that allows you to filter a collection of objects.
This is very much like the Where-Object cmdlet, but the Where
method is also like Select-Object and Group-Object as well,
includes several additional features that the Where-Object cmdlet
does not natively support by itself. This method provides faster
performance than Where-Object in a simple, elegant command. Like
the ForEach method, any objects that are output by this method are
returned in a generic collection of type
System.Collections.ObjectModel.Collection1[psobject].
There is only one version of this method, which can be described as
follows:
Where(scriptblock expression[, WhereOperatorSelectionMode mode[, int numberToReturn]])
As indicated by the square brackets, the expression script block is
required and the mode enumeration and the numberToReturn integer
argument are optional, so you can invoke this method using 1, 2, or 3
arguments. If you want to use a particular argument, you must provide
all arguments to the left of that argument (i.e. if you want to
provide a value for numberToReturn, you must provide values for
mode and expression as well).
Applied to your case (using the simplest variant Where(scriptblock expression) of the .Where() method):
$X = '6J6SCa.yB' # Get-Content "C:\Users\2.txt"
$Data = 'xxaavv6J' # Get-Content "C:\Users\d.txt"
$Split = ($Data -split '(..)').Where({$_ -ne ''})
$Y = $X.Substring(0, 6)
$Z = ($Y -split '(..)').Where{$_ -ne ''} # without parentheses
For instance, Ansgar's example changes as follows:
PS > ('xxaavv6J' -split '(..)').Where{$_ -ne ''} | % { "-$_-" }
-xx-
-aa-
-vv-
-6J-

What are some of the most useful yet little known features in the PowerShell language [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
A while back I was reading about multi-variable assignments in PowerShell. This lets you do things like this
64 > $a,$b,$c,$d = "A four word string".split()
65 > $a
A
66 > $b
four
Or you can swap variables in a single statement
$a,$b = $b,$a
What little known nuggets of PowerShell have you come across that you think may not be as well known as they should be?
The $$ command. I often have to do repeated operations on the same file path. For instance check out a file and then open it up in VIM. The $$ feature makes this trivial
PS> tf edit some\really\long\file\path.cpp
PS> gvim $$
It's short and simple but it saves a lot of time.
By far the most powerful feature of PowerShell is its ScriptBlock support. The fact that you can so concisely pass around what are effectively anonymous methods without any type constraints are about as powerful as C++ function pointers and as easy as C# or F# lambdas.
I mean how cool is it that using ScriptBlocks you can implement a "using" statement (which PowerShell doesn't have inherently). Or, pre-v2 you could even implement try-catch-finally.
function Using([Object]$Resource,[ScriptBlock]$Script) {
try {
&$Script
}
finally {
if ($Resource -is [IDisposable]) { $Resource.Dispose() }
}
}
Using ($File = [IO.File]::CreateText("$PWD\blah.txt")) {
$File.WriteLine(...)
}
How cool is that!
A feature that I find is often overlooked is the ability to pass a file to a switch statement.
Switch will iterate through the lines and match against strings (or regular expressions with the -regex parameter), content of variables, numbers, or the line can be passed into an expression to be evaluated as $true or $false
switch -file 'C:\test.txt'
{
'sometext' {Do-Something}
$pwd {Do-SomethingElse}
42 {Write-Host "That's the answer."}
{Test-Path $_} {Do-AThirdThing}
default {'Nothing else matched'}
}
$OFS - output field separator. A handy way to specify how array elements are separated when rendered to a string:
PS> $OFS = ', '
PS> "$(1..5)"
1, 2, 3, 4, 5
PS> $OFS = ';'
PS> "$(1..5)"
1;2;3;4;5
PS> $OFS = $null # set back to default
PS> "$(1..5)"
1 2 3 4 5
Always guaranteeing you get an array result. Consider this code:
PS> $files = dir *.iMayNotExist
PS> $files.length
$files in this case may be $null, a scalar value or an array of values. $files.length isn't going to give you the number of files found for $null or for a single file. In the single file case, you will get the file's size!! Whenever I'm not sure how much data I'll get back I always enclose the command in an array subexpression like so:
PS> $files = #(dir *.iMayNotExist)
PS> $files.length # always returns number of files in array
Then $files will always be an array. It may be empty or have only a single element in it but it will be an array. This makes reasoning with the result much simpler.
Array covariance support:
PS> $arr = '127.0.0.1','192.168.1.100','192.168.1.101'
PS> $ips = [system.net.ipaddress[]]$arr
PS> $ips | ft IPAddressToString, AddressFamily -auto
IPAddressToString AddressFamily
----------------- -------------
127.0.0.1 InterNetwork
192.168.1.100 InterNetwork
192.168.1.101 InterNetwork
Comparing arrays using Compare-Object:
PS> $preamble = [System.Text.Encoding]::UTF8.GetPreamble()
PS> $preamble | foreach {"0x{0:X2}" -f $_}
0xEF
0xBB
0xBF
PS> $fileHeader = Get-Content Utf8File.txt -Enc byte -Total 3
PS> $fileheader | foreach {"0x{0:X2}" -f $_}
0xEF
0xBB
0xBF
PS> #(Compare-Object $preamble $fileHeader -sync 0).Length -eq 0
True
Fore more stuff like this, check out my free eBook - Effective PowerShell.
Along the lines of multi-variable assignments.
$list = 1,2,3,4
While($list) {
$head, $list = $list
$head
}
1
2
3
4
I've been using this:
if (!$?) { # if previous command was not successful
Do some stuff
}
and I also use $_ (current pipeline object) quite a bit, but these might be more known than other stuff.
The fact that many operators work on arrays as well and return the elements where a comparison is true or operate on each element of the array independently:
1..1000 -lt 800 -gt 400 -like "?[5-9]0" -replace 0 -as "int[]" -as "char[]" -notmatch "\d"
This is faster than Where-Object.
Not a language feature but super helpful
f8 -- Takes the text you have put in already and searches for a command that starts with that text.
Tab-search through your history with #
Example:
PS> Get-Process explorer
PS> "Ford Explorer"
PS> "Magellan" | Add-Content "great explorers.txt"
PS> type "great explorers.txt"
PS> #expl <-- Hit the <tab> key to cycle through history entries that have the term "expl"
Love this thread. I could list a ton of things after reading Windows Powershell in Action. There's a disconnect between that book and the documentation. I actually tried to list them all somewhere else here, but got put on hold for "not being a question".
I'll start with foreach with three script blocks (begin/process/end):
Get-ChildItem | ForEach-Object {$sum=0} {$sum++} {$sum}
Speaking of swapping two variables, here's swapping two files:
${c:file1.txt},${c:file2.txt} = ${c:file2.txt},${c:file1.txt}
Search and replace a file:
${c:file.txt} = ${c:file.txt} -replace 'oldstring','newstring'
Using assembly and using namespace statements:
using assembly System.Windows.Forms
using namespace System.Windows.Forms
[messagebox]::show('hello world')
A shorter version of foreach, with properties and methods
ps | foreach name
'hi.there' | Foreach Split .
Use $() operator outside of strings to combine two statements:
$( echo hi; echo there ) | measure
Get-content/Set-content with variables:
$a = ''
get-content variable:a | set-content -value there
Anonymous functions:
1..5 | & {process{$_ * 2}}
Give the anonymous function a name:
$function:timestwo = {process{$_ * 2}}
Anonymous function with parameters:
& {param($x,$y) $x+$y} 2 5
You can stream from foreach () with these, where normally you can't:
& { foreach ($i in 1..10) {$i; sleep 1} } | out-gridview
Run processes in background like unix '&', and then wait for them:
$a = start-process -NoNewWindow powershell {timeout 10; 'done a'} -PassThru
$b = start-process -NoNewWindow powershell {timeout 10; 'done b'} -PassThru
$c = start-process -NoNewWindow powershell {timeout 10; 'done c'} -PassThru
$a,$b,$c | wait-process
Or foreach -parallel in workflows:
workflow work {
foreach -parallel ($i in 1..3) {
sleep 5
"$i done"
}
}
work
Or a workflow parallel block where you can run different things:
function sleepfor($time) { sleep $time; "sleepfor $time done"}
workflow work {
parallel {
sleepfor 3
sleepfor 2
sleepfor 1
}
'hi'
}
work
Three parallel commands in three more runspaces with the api:
$a = [PowerShell]::Create().AddScript{sleep 5;'a done'}
$b = [PowerShell]::Create().AddScript{sleep 5;'b done'}
$c = [PowerShell]::Create().AddScript{sleep 5;'c done'}
$r1,$r2,$r3 = ($a,$b,$c).begininvoke()
$a.EndInvoke($r1); $b.EndInvoke($r2); $c.EndInvoke($r3) # wait
($a,$b,$c).Streams.Error # check for errors
($a,$b,$c).dispose() # cleanup
Parallel processes with invoke-command, but you have to be at an elevated prompt with remote powershell working:
invoke-command localhost,localhost,localhost { sleep 5; 'hi' }
An assignment is an expression:
if ($a = 1) { $a }
$a = $b = 2
Get last array element with -1:
(1,2,3)[-1]
Discard output with [void]:
[void] (echo discard me)
Switch on arrays and $_ on either side:
switch(1,2,3,4,5,6) {
{$_ % 2} {"Odd $_"; continue}
4 {'FOUR'}
default {"Even $_"}
}
Get and set variables in a module:
'$script:count = 0
$script:increment = 1
function Get-Count { return $script:count += $increment }' > counter.psm1 # creating file
import-module .\counter.psm1
$m = get-module counter
& $m Get-Variable count
& $m Set-Variable count 33
See module function definition:
& $m Get-Item function:Get-Count | foreach definition
Run a command with a commandinfo object and the call operator:
$d = get-command get-date
& $d
Dynamic modules:
$m = New-Module {
function foo {"In foo x is $x"}
$x=2
Export-ModuleMember -func foo -var x
}
flags enum:
[flags()] enum bits {one = 1; two = 2; three = 4; four = 8; five = 16}
[bits]31
Little known codes for the -replace operator:
$number Substitutes the last submatch matched by group number.
${name} Substitutes the last submatch matched by a named capture of the form (?).
$$ Substitutes a single "$" literal.
$& Substitutes a copy of the entire match itself.
$` Substitutes all the text from the argument string before the matching portion.
$' Substitutes all the text of the argument string after the matching portion.
$+ Substitutes the last submatch captured.
$_ Substitutes the entire argument string.
Demo of workflows surviving interruptions using checkpoints. Kill the window or reboot. Then start PS again. Use get-job and resume-job to resume the job.
workflow test1 {
foreach ($b in 1..1000) {
$b
Checkpoint-Workflow
}
}
test1 -AsJob -JobName bootjob
Emacs edit mode. Pressing tab completion lists all the options at once. Very useful.
Set-PSReadLineOption -EditMode Emacs
Any command that begins with "get-", you can leave off the "get-":
date
help
End parsing --% and end of parameters -- operators.
write-output --% -inputobject
write-output -- -inputobject
Tab completion on wildcards:
cd \pro*iles # press tab
Compile and import a C# module with a cmdlet inside, even in Osx:
Add-Type -Path ExampleModule.cs -OutputAssembly ExampleModule.dll
Import-Module ./ExampleModule.dll
Iterate backwards over a sequence just use the len of the sequence with a 1 on the other side of the range:
foreach( x in seq.length..1) { Do-Something seq[x] }

Powershell pitfalls

What Powershell pitfalls you have fall into? :-)
Mine are:
# -----------------------------------
function foo()
{
#("text")
}
# Expected 1, actually 4.
(foo).length
# -----------------------------------
if(#($null, $null))
{
Write-Host "Expected to be here, and I am here."
}
if(#($null))
{
Write-Host "Expected to be here, BUT NEVER EVER."
}
# -----------------------------------
function foo($a)
{
# I thought this is right.
#if($a -eq $null)
#{
# throw "You can't pass $null as argument."
#}
# But actually it should be:
if($null -eq $a)
{
throw "You can't pass $null as argument."
}
}
foo #($null, $null)
# -----------------------------------
# There is try/catch, but no callstack reported.
function foo()
{
bar
}
function bar()
{
throw "test"
}
# Expected:
# At bar() line:XX
# At foo() line:XX
#
# Actually some like this:
# At bar() line:XX
foo
Would like to know yours to walk them around :-)
My personal favorite is
function foo() {
param ( $param1, $param2 = $(throw "Need a second parameter"))
...
}
foo (1,2)
For those unfamiliar with powershell that line throws because instead of passing 2 parameters it actually creates an array and passes one parameter. You have to call it as follows
foo 1 2
Another fun one. Not handling an expression by default writes it to the pipeline. Really annoying when you don't realize a particular function returns a value.
function example() {
param ( $p1 ) {
if ( $p1 ) {
42
}
"done"
}
PS> example $true
42
"done"
$files = Get-ChildItem . -inc *.extdoesntexist
foreach ($file in $files) {
"$($file.Fullname.substring(2))"
}
Fails with:
You cannot call a method on a null-valued expression.
At line:3 char:25
+ $file.Fullname.substring <<<< (2)
Fix it like so:
$files = #(Get-ChildItem . -inc *.extdoesntexist)
foreach ($file in $files) {
"$($file.Fullname.substring(2))"
}
Bottom line is that the foreach statement will loop on a scalar value even if that scalar value is $null. When Get-ChildItem in the first example returns nothing, $files gets assinged $null. If you are expecting an array of items to be returned by a command but there is a chance it will only return 1 item or zero items, put #() around the command. Then you will always get an array - be it of 0, 1 or N items. Note: If the item is already an array putting #() has no effect - it will still be the very same array (i.e. there is no extra array wrapper).
# The pipeline doesn't enumerate hashtables.
$ht = #{"foo" = 1; "bar" = 2}
$ht | measure
# Workaround: call GetEnumerator
$ht.GetEnumerator() | measure
Here are my top 5 PowerShell gotchas
Here is something Ive stumble upon lately (PowerShell 2.0 CTP):
$items = "item0", "item1", "item2"
$part = ($items | select-string "item0")
$items = ($items | where {$part -notcontains $_})
what do you think that $items be at the end of the script?
I was expecting "item1", "item2" but instead the value of $items is: "item0", "item1", "item2".
Say you've got the following XML file:
<Root>
<Child />
<Child />
</Root>
Run this:
PS > $myDoc = [xml](Get-Content $pathToMyDoc)
PS > #($myDoc.SelectNodes("/Root/Child")).Count
2
PS > #($myDoc.Root.Child).Count
2
Now edit the XML file so it has no Child nodes, just the Root node, and run those statements again:
PS > $myDoc = [xml](Get-Content $pathToMyDoc)
PS > #($myDoc.SelectNodes("/Root/Child")).Count
0
PS > #($myDoc.Root.Child).Count
1
That 1 is annoying when you want to iterate over a collection of nodes using foreach if and only if there actually are any. This is how I learned that you cannot use the XML handler's property (dot) notation as a simple shortcut. I believe what's happening is that SelectNodes returns a collection of 0. When #'ed, it is transformed from an XPathNodeList to an Object[] (check GetType()), but the length is preserved. The dynamically generated $myDoc.Root.Child property (which essentially does not exist) returns $null. When $null is #'ed, it becomes an array of length 1.
On Functions...
The subtleties of processing pipeline input in a function with respect to using $_ or $input and with respect to the begin, process, and end blocks.
How to handle the six principal equivalence classes of input delivered to a function (no input, null, empty string, scalar, list, list with null and/or empty) -- for both direct input and pipeline input -- and get what you expect.
The correct calling syntax for sending multiple arguments to a function.
I discuss these points and more at length in my Simple-Talk.com article Down the Rabbit Hole- A Study in PowerShell Pipelines, Functions, and Parameters and also provide an accompanying wallchart--here is a glimpse showing the various calling syntax pitfalls for a function taking 3 arguments:
On Modules...
These points are expounded upon in my Simple-Talk.com article Further Down the Rabbit Hole: PowerShell Modules and Encapsulation.
Dot-sourcing a file inside a script using a relative path is relative to your current directory -- not the directory where the script resides!
To be relative to the script use this function to locate your script directory: [Update for PowerShell V3+: Just use the builtin $PSScriptRoot variable!]
function Get-ScriptDirectory
{ Split-Path $script:MyInvocation.MyCommand.Path }
Modules must be stored as ...Modules\name\name.psm1 or ...\Modules\any_subpath\name\name.psm1. That is, you cannot just use ...Modules\name.psm1 -- the name of the immediate parent of the module must match the base name of the module. This chart shows the various failure modes when this rule is violated:
2015.06.25 A Pitfall Reference Chart
Simple-Talk.com just published the last of my triumvirate of in-depth articles on PowerShell pitfalls. The first two parts are in the form of a quiz that helps you appreciate a select group of pitfalls; the last part is a wallchart (albeit it would need a rather high-ceilinged room) containing 36 of the most common pitfalls (some adapted from answers on this page), giving concrete examples and workarounds for most. Read more here.
There are some tricks to building command lines for utilities that were not built with Powershell in mind:
To run an executable who's name starts with a number, preface it with an Ampersand (&).
& 7zip.exe
To run an executable with a space anywhere in the path, preface it with an Ampersand (&) and wrap it in quotes, as you would any string. This means that strings in a variable can be executed as well.
# Executing a string with a space.
& 'c:\path with spaces\command with spaces.exe'
# Executing a string with a space, after first saving it in a variable.
$a = 'c:\path with spaces\command with spaces.exe'
& $a
Parameters and arguments are passed to legacy utilities positionally. So it is important to quote them the way the utility expects to see them. In general, one would quote when it contains spaces or does not start with a letter, number or dash (-).
C:\Path\utility.exe '/parameter1' 'Value #1' 1234567890
Variables can be used to pass string values containing spaces or special characters.
$b = 'string with spaces and special characters (-/&)'
utility.exe $b
Alternatively array expansion can be used to pass values as well.
$c = #('Value #1', $Value2)
utility.exe $c
If you want Powershell to wait for an application to complete, you have to consume the output, either by piping the output to something or using Start-Process.
# Saving output as a string to a variable.
$output = ping.exe example.com | Out-String
# Piping the output.
ping stackoverflow.com | where { $_ -match '^reply' }
# Using Start-Process affords the most control.
Start-Process -Wait SomeExecutable.com
Because of the way they display their output, some command line utilities will appear to hang when ran inside of Powershell_ISE.exe, particularly when awaiting input from the user. These utilities will usually work fine when ran within Powershell.exe console.
PowerShell Gotchas
There are a few pitfall that repeatedly reappear on StackOverflow. It is recommend to do some research if you are not familiar with these PowerShell gotchas before asking a new question. It might even be a good idea to investigate in these PowerShell gotchas before answering a PowerShell question to make sure that you teach the questioner the right thing.
TLDR: In PowerShell:
the comparison equality operator is: -eq
(Stackoverflow example: Powershell simple syntax if condition not working)
parentheses and commas are not used with arguments
(Stackoverflow example: How do I pass multiple parameters into a function in PowerShell?)
output properties are based on the first object in the pipeline
(Stackoverflow example: Not all properties displayed)
the pipeline unrolls
(Stackoverflow example: Pipe complete array-objects instead of array items one at a time?)
a. single item collections
(Stackoverflow example: Powershell ArrayList turns a single array item back into a string)
b. embedded arrays
(Stackoverflow example: Return Multidimensional Array From Function)
c. output collections
(Stackoverflow example: Why does PowerShell flatten arrays automatically?)
$Null should be on the left side of the equality comparison operator
(Stackoverflow example: Should $null be on the left side of the equality comparison)
parentheses and assignments choke the pipeline
(Stackoverflow example: Importing 16MB CSV Into Variable Creates >600MB's Memory Usage)
the increase assignment operator (+=) might become expensive
Stackoverflow example: Improve the efficiency of my PowerShell scrip
The Get-Content cmdlet returns separate lines
Stackoverflow example: Multiline regex to match config block
Examples and explanations
Some of the gotchas might really feel counter-intuitive but often can be explained by some very nice PowerShell features along with the pipeline, expression/argument mode and type casting.
1. The comparison equality operator is: -eq
Unlike the Microsoft scripting language VBScript and some other programming languages, the comparison equality operator differs from the assignment operator (=) and is: -eq.
Note: assigning a value to a variable might pass through the value if needed:
$a = $b = 3 # The value 3 is assigned to both variables $a and $b.
This implies that following statement might be unexpectedly truthy or falsy:
If ($a = $b) {
# (assigns $b to $a and) returns a truthy if $b is e.g. 3
} else {
# (assigns $b to $a and) returns a falsy if $b is e.g. 0
}
2. Parentheses and commas are not used with arguments
Unlike a lot of other programming languages and the way a primitive PowerShell function is defined, calling a function doesn't require parentheses or commas for their related arguments. Use spaces to separate the parameter arguments:
MyFunction($Param1, $Param2 $Param3) {
# ...
}
MyFunction 'one' 'two' 'three' # assigns 'one' to $Param1, 'two' to $Param2, 'three' to $Param3
Parentheses and commas are used for calling (.Net) methods.
Commas are used to define arrays. MyFunction 'one', 'two', 'three' (or MyFunction('one', 'two', 'three')) will load the array #('one', 'two', 'three') into the first parameter ($Param1).
Parentheses will intepret the containing contents as a single collection into memory (and choke the PowerShell pipeline) and should only be used as such, e.g. to call an embedded function, e.g.:
MyFunction (MyOtherFunction) # passes the results MyOtherFunction to the first positional parameter of MyFunction ($Param1)
MyFunction One $Two (getThree) # assigns 'One' to $Param1, $Two to $Param2, the results of getThree to $Param3
Note: that quoting text arguments (as the word one in the later example) is only required when it contains spaces or special characters.
3. Output properties are based on the first object in the pipeline
In a PowerShell pipeline each object is processed and passed on by a cmdlet (that is implemented for the middle of a pipeline) similar to how objects are processed and passed on by workstations in an assembly line. Meaning each cmdlet processes one item at the time while the prior cmdlet (workstation) simultaneously processes the upcoming one. This way, the objects aren't loaded into memory at once (less memory usage) and could already be processed before the next one is supplied (or even exists). The disadvantage of this feature is that there is no oversight of what (or how many) objects are expected to follow.
Therefore most PowerShell cmdlets assume that all the objects in the pipeline correspond to the first one and have the same properties which is usually the case, but not always...
$List =
[pscustomobject]#{ one = 'a1'; two = 'a2' },
[pscustomobject]#{ one = 'b1'; two = 'b2'; three = 'b3' }
$List |Select-Object *
one two
--- ---
a1 a2
b1 b2
As you see, the third column three is missing from the results as it didn't exists in the first object and the PowerShell was already outputting the results prior it was aware of the exists of the second object.
On way to workaround this behavior is to explicitly define the properties (of all the following objects) at forehand:
$List |Select-Object one, two, three
one two three
--- --- -----
a1 a2
b1 b2 b3
See also proposal: #13906 Add -UnifyProperties parameter to Select-Object
4. The pipeline unrolls
This feature might come in handy if it complies with the straightforward expectation:
$Array = 'one', 'two', 'three'
$Array.Length
3
a. single item collections
But it might get confusing:
$Selection = $Array |Select-Object -First 2
$Selection.Length
2
$Selection[0]
one
when the collection is down to a single item:
$Selection = $Array |Select-Object -First 1
$Selection.Length
3
$Selection[0]
o
Explanation
When the pipeline outputs a single item which is assigned to a variable, it is not assigned as a collection (with 1 item, like: #('one')) but as a scalar item (the item itself, like: 'one').
Which means that the property .Length (which is in fact an alias for the property .Count for an array) is no longer applied on the array but on the string: 'one'.length which equals 3. And in case of the index $Selection[0] , the first character of the string 'one'[0] (which equals the character o) is returned .
Workaround
To workaround this behavior, you might force the scalar item to an array using the Array subexpression operator #( ):
$Selection = $Array |Select-Object -First 1
#($Selection).Length
1
#($Selection)[0]
one
Knowing that in the case the $Selection is already an array, it will will not be further increased in depth (#(#('one', 'two')), see the next section 4b. Embedded collections are flattened).
b. embedded arrays
When an array (or a collection) includes embedded arrays, like:
$Array = #(#('a', 'b'), #('c', 'd'))
$Array.Count
2
All the embedded items will be processed in the pipeline and consequently returns a flat array when displayed or assigned to a new variable:
$Processed = $Array |ForEach-Object { $_ }
$Processed.Count
4
$Processed
a
b
c
d
To iterate the embedded arrays, you might use the foreach statement:
foreach ($Item in $Array) { $Item.Count }
2
2
Or a simply for loop:
for ($i = 0; $i -lt $Array.Count; $i++) { $Array[$i].Count }
2
2
c. output collections
Collections are usually unrolled when they are placed on the pipeline:
function GetList {
[Collections.Generic.List[String]]#('a', 'b')
}
(GetList).GetType().Name
Object[]
To output the collection as a single item, use the comma operator ,:
function GetList {
,[Collections.Generic.List[String]]#('a', 'b')
}
(GetList).GetType().Name
List`1
5. $Null should be on the left side of the equality comparison operator
This gotcha is related to this comparison operators feature:
When the input of an operator is a scalar value, the operator returns a Boolean value. When the input is a collection, the operator returns the elements of the collection that match the right-hand value of the expression. If there are no matches in the collection, comparison operators return an empty array.
This means for scalars:
'a' -eq 'a' # returns $True
'a' -eq 'b' # returns $False
'a' -eq $Null # returns $False
$Null -eq $Null # returns $True
and for collections, the matching elements are returned which evaluates to either a truthy or falsy condition:
'a', 'b', 'c' -eq 'a' # returns 'a' (truthy)
'a', 'b', 'c' -eq 'd' # returns an empty array (falsy)
'a', 'b', 'c' -eq $Null # returns an empty array (falsy)
'a', $Null, 'c' -eq $Null # returns $Null (falsy)
'a', $Null, $Null -eq $Null # returns #($Null, $Null) (truthy!!!)
$Null, $Null, $Null -eq $Null # returns #($Null, $Null, $Null) (truthy!!!)
In other words, to check whether a variable is $Null (and exclude a collection containing multiple $Nulls), put $Null at the LHS (left hand side) of the equality comparison operator:
if ($Null -eq $MyVariable) { ...
6. Parentheses and assignments choke the pipeline
The PowerShell Pipeline is not just a series of commands connected by pipeline operators (|) (ASCII 124). It is a concept to simultaneously stream individual objects through a sequence of cmdlets. If a cmdlet (or function) is written according to the Strongly Encouraged Development Guidelines and implemented for the middle of a pipeline, it takes each single object from the pipeline, processes it and passes the results to the next cmdlet just before it takes and processes the next object in the pipeline. Meaning that for a simple pipeline as:
Import-Csv .\Input.csv |Select-Object -Property Column1, Column2 |Export-Csv .\Output.csv
As the last cmdlet writes an object to the .\Output.csv file, the Select-Object cmdlet selects the properties of the next object and the Import-Csv reads the next object from the .\input.csv file (see also: Pipeline in Powershell). This will keep the memory usage low (especially where there are lots of object/records to process) and therefore might result in a faster throughput. To facilitate the pipeline, the PowerShell objects are quiet fat as each individual object contains all the property information (along with e.g. the property name).
Therefore it is not a good practice to choke the pipeline for no reason. There are two senarios that choke the pipeline:
Parentheses, e.g.:
(Import-Csv .\Input.csv) |Select-Object -Property Column1, Column2 |Export-Csv .\Output.csv
Where all the .\Input.csv records are loaded as an array of PowerShell objects into memory before passing it on to the Select-Object cmdlet.
Assignments, e.g.:
$Objects = Import-Csv .\Input.csv
$Objects |Select-Object -Property Column1, Column2 |Export-Csv .\Output.csv
Where all the .\Input.csv records are loaded as an array of PowerShell objects into $Objects (memory as well) before passing it on to the Select-Object cmdlet.
7. the increase assignment operator (+=) might become expensive
The increase assignment operator (+=) is syntactic sugar to increase and assign primitives as .e.g. $a += $b where $a is assigned $b + 1. The increase assignment operator can also be used for adding new items to a collection (or to String types and hash tables) but might get pretty expensive as the costs increases with each iteration (the size of the collection). The reason for this is that objects as array collections are immutable and the right variable in not just appended but *appended and reassigned to the left variable. For details see also: avoid using the increase assignment operator (+=) to create a collection
8. The Get-Content cmdlet returns separate lines
There are probably quite some more cmdlet gotchas, knowing that there exist a lot of (internal and external) cmdlets. In contrast to engine related gotchas, these gotchas are often easier to highlight (with e.g. a warning) as happend with ConvertTo-Json (see: Unexpected ConvertTo-Json results? Answer: it has a default -Depth of 2) or "fix". But there is very clasic gotcha in Get-Content which tight into the PowerShell general concept of streaming objects (in this case lines) rather than passing everything (the whole contents of the file) in once:
Get-Content .\Input.txt -Match '\r?\n.*Test.*\r?\n'
Will never work because, by default, Get-Contents returns a stream of objects where each object contains a single string (a line without any line breaks).
(Get-Content .\Input.txt).GetType().Name
Object[]
(Get-Content .\Input.txt)[0].GetType().Name
String
In fact:
Get-Content .\Input.txt -Match 'Test'
Returns all the lines with the word Test in it as Get-Contents puts every single line on the pipeline and when the input is a collection, the operator returns the elements of the collection that match the right-hand value of the expression.
Note: since PowerShell version 3, Get-Contents has a -Raw parameter that reads all the content of the concerned file at once, Meaning that this: Get-Content -Raw .\Input.txt -Match '\r?\n.*Test.*\r?\n' will work as it loads the whole file into memory.
alex2k8, I think this example of yours is good to talk about:
# -----------------------------------
function foo($a){
# I thought this is right.
#if($a -eq $null)
#{
# throw "You can't pass $null as argument."
#}
# But actually it should be:
if($null -eq $a)
{
throw "You can't pass $null as argument."
}
}
foo #($null, $null)
PowerShell can use some of the comparators against arrays like this:
$array -eq $value
## Returns all values in $array that equal $value
With that in mind, the original example returns two items (the two $null values in the array), which evalutates to $true because you end up with a collection of more than one item. Reversing the order of the arguments stops the array comparison.
This functionality is very handy in certain situations, but it is something you need to be aware of (just like array handling in PowerShell).
Functions 'foo' and 'bar' looks equivalent.
function foo() { $null }
function bar() { }
E.g.
(foo) -eq $null
# True
(bar) -eq $null
# True
But:
foo | %{ "foo" }
# Prints: foo
bar | %{ "bar" }
# PRINTS NOTHING
Returning $null and returning nothing is not equivalent dealing with pipes.
This one is inspired by Keith Hill example...
function bar() {}
$list = #(foo)
$list.length
# Prints: 0
# Now let's try the same but with a temporal variable.
$tmp = foo
$list = #($tmp)
$list.length
# Prints: 1
Another one:
$x = 2
$y = 3
$a,$b = $x,$y*5
because of operators precedence there is not 25 in $b; the command is the same as ($x,$y)*5
the correct version is
$a,$b = $x,($y*5)
The logical and bitwise operators don't follow standard precedence rules. The operator -and should have a higher priority than -or yet they're evaluated strictly left-to-right.
For example, compare logical operators between PowerShell and Python (or virtually any other modern language):
# PowerShell
PS> $true -or $false -and $false
False
# Python
>>> True or False and False
True
...and bitwise operators:
# PowerShell
PS> 1 -bor 0 -band 0
0
# Python
>>> 1 | 0 & 0
1
This works. But almost certainly not in the way you think it's working.
PS> $a = 42;
PS> [scriptblock]$b = { $a }
PS> & $b
42
This one has tripped me up before, using $o.SomeProperty where it should be $($o.SomeProperty).
# $x is not defined
[70]: $x -lt 0
True
[71]: [int]$x -eq 0
True
So, what's $x..?
Another one I ran into recently: [string] parameters that accept pipeline input are not strongly typed in practice. You can pipe anything at all and PS will coerce it via ToString().
function Foo
{
[CmdletBinding()]
param (
[parameter(Mandatory=$True, ValueFromPipeline=$True)]
[string] $param
)
process { $param }
}
get-process svchost | Foo
Unfortunately there is no way to turn this off. Best workaround I could think of:
function Bar
{
[CmdletBinding()]
param (
[parameter(Mandatory=$True, ValueFromPipeline=$True)]
[object] $param
)
process
{
if ($param -isnot [string]) {
throw "Pass a string you fool!"
}
# rest of function goes here
}
}
edit - a better workaround I've started using...
Add this to your custom type XML -
<?xml version="1.0" encoding="utf-8" ?>
<Types>
<Type>
<Name>System.String</Name>
<Members>
<ScriptProperty>
<Name>StringValue</Name>
<GetScriptBlock>
$this
</GetScriptBlock>
</ScriptProperty>
</Members>
</Type>
</Types>
Then write functions like this:
function Bar
{
[CmdletBinding()]
param (
[parameter(Mandatory=$True, ValueFromPipelineByPropertyName=$True)]
[Alias("StringValue")]
[string] $param
)
process
{
# rest of function goes here
}
}
Forgetting that $_ gets overwritten in blocks made me scratch my head in confusion a couple times, and similarly for multiple reg-ex matches and the $matches array. >.<
Remembering to explicitly type pscustom objects from imported data tables as numeric so they can be sorted correctly:
$CVAP_WA=foreach ($i in $C){[PSCustomObject]#{ `
County=$i.county; `
TotalVote=[INT]$i.TotalBallots; `
RegVoters=[INT]$i.regvoters; `
Turnout_PCT=($i.TotalBallots/$i.regvoters)*100; `
CVAP=[INT]($B | ? {$_.GeoName -match $i.county}).CVAP_EST }}
PS C:\Politics> $CVAP_WA | sort -desc TotalVote |ft -auto -wrap
County TotalVote RegVoters Turnout_PCT CVAP CVAP_TV_PCT CVAP_RV_PCT
------ --------- --------- ----------- ---- ----------- -----------
King 973088 1170638 83.189 1299290 74.893 90.099
Pierce 349377 442985 78.86 554975 62.959 79.837
Snohomish 334354 415504 80.461 478440 69.832 86.81
Spokane 227007 282442 80.346 342060 66.398 82.555
Clark 193102 243155 79.453 284190 67.911 85.52
Mine are both related to file copying...
Square Brackets in File Names
I once had to move a very large/complicated folder structure using Move-Item -Path C:\Source -Destination C:\Dest. At the end of the process there were still a number of files in source directory. I noticed that every remaining file had square brackets in the name.
The problem was that the -Path parameter treats square brackets as wildcards.
EG. If you wanted to copy Log001 to Log200, you could use square brackets as follows:
Move-Item -Path C:\Source\Log[001-200].log.
In my case, to avoid square brackets being interpreted as wildcards, I should have used the -LiteralPath parameter.
ErrorActionPreference
The $ErrorActionPreference variable is ignored when using Move-Item and Copy-Item with the -Verbose parameter.
Treating the ExitCode of a Process as a Boolean.
eg, with this code:
$p = Start-Process foo.exe -NoNewWindow -Wait -PassThru
if ($p.ExitCode) {
# handle error
}
things are good, unless say foo.exe doesn't exist or otherwise fails to launch.
in that case $p will be $null, and [bool]($null.ExitCode) is False.
a simple fix is to replace the logic with if ($p.ExitCode -ne 0) {},
however for clarity of code imo the following is better: if (($p -eq $null) -or ($p.ExitCode -ne 0)) {}