I would really like to use the function Split-Job to throttle or run the same script block in parallel, so it goes quicker. This is especially useful when I need to launch multiple times the same copy command on different input.
The code can be found here and always spits out the following error:
An empty pipe element is not allowed.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : EmptyPipeElement
I know the problem is within this function, but I can't seem to solve it:
function Init ($InputQueue){
# Create the shared thread-safe queue and fill it with the input objects
$Queue = [Collections.Queue]::Synchronized([Collections.Queue]#($InputQueue))
$QueueLength = $Queue.Count
# Do not create more runspaces than input objects
if ($MaxPipelines -gt $QueueLength) {$MaxPipelines = $QueueLength}
# Create the script to be run by each runspace
$Script = "Set-Location '$PWD'; "
$Script += {
$SplitJobQueue = $($Input)
& {
trap {continue}
while ($SplitJobQueue.Count) {$SplitJobQueue.Dequeue()}
} |
}.ToString() + $Scriptblock
# Create an array to keep track of the set of pipelines
$Pipelines = New-Object System.Collections.ArrayList
# Collect the functions and aliases to import
$ImportItems = ($Function -replace '^','Function:') +
($Alias -replace '^','Alias:') |
Get-Item | select PSPath, Definition
$stopwatch = New-Object System.Diagnostics.Stopwatch
$stopwatch.Start()
}
Thank you for your help. Because this little function might help me out a lot if it would just work.
In this section:
$Script += {
$SplitJobQueue = $($Input)
& {
trap {continue}
while ($SplitJobQueue.Count) {$SplitJobQueue.Dequeue()}
} |
}.ToString() + $Scriptblock
There doesn't appear to be any reason for that pipe to be there.
Related
$global:af_fp = "C:\Path\to\folder\"
Function function-name {
do things …
$global:af_fp = $global:af_fp + $variableFromDo_things + "_AF.csv"
}
function-name | ConvertTo-CSV -NoTypeInformation | Add-Content -Path $($af_fp)
Above is the generalized (and abbreviated) script contents for a powershell script.
Every time I run the script in this way, I get the following error:
Add-Content : Could not find a part of the path 'C:\Users\timeuser\Documents\'.
At C:\Users\timeuser\Documents\get_software.ps1:231 char:51
+ ... ware | ConvertTo-CSV -NoTypeInformation | Add-Content -Path $($af_fp)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\Users\timeuser\Documents\:String) [Add-Content], DirectoryNotFoundException
+ FullyQualifiedErrorId : GetContentWriterDirectoryNotFoundError,Microsoft.PowerShell.Commands.AddContentCommand
When I run
Get-Variable -Scope global
after running the script and seeing the error, the variable af_fp contains exactly the information I am seeking for the file name, however, the error shows the variable contents ending in ':String'.
To confuse me even more, if I comment out the lines containing '$global:...' and re-run the same script, IT ACTUALL RUNS AND SAVES THE FILE USING THE LINE
function-name | ConvertTo-CSV -NoTypeInformation | Add-Content -Path $($af_fp)
AS INTENDED. Of course, I had to run the script and watch it error first, then re-run the script with the global variable declaration and update commented out for it to actually work. I want to run the script ONCE and still get the same results.
FYI, I am a complete noob to powershell, but very familiar with the concept of variable scope.....but why is this global not working when initially created and updated, but then work the second time around, when, as far as I can tell, the CONTENT AND SCOPE of the global remains the same...…. any assistance to finding a solution to this small issue would be greatly appreciated; I have tried sooooo may different methods from inquiries through here and on Google...…..
EDIT: not sure why this will matter, because the script ran before as intended when I explicitly typed the parameter for -Path as 'C:\path\to\file'. The ONLY CHANGES MADE to the original, working script (below) were my inclusion of the global variable declaration, the update to the contents of the global variable (near the end of the function), and the attempt to use the global variable as the parameter to -Path, that is why I omitted the script:
'''
$global:af_fp = "C:\Users\timeuser\Documents\"
Function Get-Software {
[OutputType('System.Software.Inventory')]
[Cmdletbinding()]
Param(
[Parameter(ValueFromPipeline = $True, ValueFromPipelineByPropertyName = $True)]
[String[]]$Computername = $env:COMPUTERNAME
)
Begin {
}
Process {
ForEach ($Computer in $Computername) {
If (Test-Connection -ComputerName $Computer -Count 1 -Quiet) {
$Paths = #("SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall", "SOFTWARE\\Wow6432node\\Microsoft\\Windows\\CurrentVersion\\Uninstall")
ForEach ($Path in $Paths) {
Write-Verbose "Checking Path: $Path"
# Create an instance of the Registry Object and open the HKLM base key
Try {
$reg = [microsoft.win32.registrykey]::OpenRemoteBaseKey('LocalMachine', $Computer, 'Registry64')
}
Catch {
Write-Error $_
Continue
}
# Drill down into the Uninstall key using the OpenSubKey Method
Try {
$regkey = $reg.OpenSubKey($Path)
# Retrieve an array of string that contain all the subkey names
$subkeys = $regkey.GetSubKeyNames()
# Open each Subkey and use GetValue Method to return the required values for each
ForEach ($key in $subkeys) {
Write-Verbose "Key: $Key"
$thisKey = $Path + "\\" + $key
Try {
$thisSubKey = $reg.OpenSubKey($thisKey)
# Prevent Objects with empty DisplayName
$DisplayName = $thisSubKey.getValue("DisplayName")
If ($DisplayName -AND $DisplayName -notmatch '^Update for|rollup|^Security Update|^Service Pack|^HotFix') {
$Date = $thisSubKey.GetValue('InstallDate')
If ($Date) {
Try {
$Date = [datetime]::ParseExact($Date, 'yyyyMMdd', $Null)
}
Catch {
Write-Warning "$($Computer): $_ <$($Date)>"
$Date = $Null
}
}
# Create New Object with empty Properties
$Publisher = Try {
$thisSubKey.GetValue('Publisher').Trim()
}
Catch {
$thisSubKey.GetValue('Publisher')
}
$Version = Try {
#Some weirdness with trailing [char]0 on some strings
$thisSubKey.GetValue('DisplayVersion').TrimEnd(([char[]](32, 0)))
}
Catch {
$thisSubKey.GetValue('DisplayVersion')
}
$UninstallString = Try {
$thisSubKey.GetValue('UninstallString').Trim()
}
Catch {
$thisSubKey.GetValue('UninstallString')
}
$InstallLocation = Try {
$thisSubKey.GetValue('InstallLocation').Trim()
}
Catch {
$thisSubKey.GetValue('InstallLocation')
}
$InstallSource = Try {
$thisSubKey.GetValue('InstallSource').Trim()
}
Catch {
$thisSubKey.GetValue('InstallSource')
}
$HelpLink = Try {
$thisSubKey.GetValue('HelpLink').Trim()
}
Catch {
$thisSubKey.GetValue('HelpLink')
}
$Object = [pscustomobject]#{
#Potential Candidate for AssetID in the TIME system
AssetID = $Computer
#String that contains word or word combinations for the product field of CPE WFN; may also contain the valid values necessary for update, edition, language, sw_edition, target_hw/sw fields as well.
cpeprodinfo = $DisplayName
cpeversion = $Version
InstallDate = $Date
cpevendor = $Publisher
UninstallString = $UninstallString
InstallLocation = $InstallLocation
InstallSource = $InstallSource
HelpLink = $thisSubKey.GetValue('HelpLink')
EstimatedSizeMB = [decimal]([math]::Round(($thisSubKey.GetValue('EstimatedSize') * 1024) / 1MB, 2))
}
$Object.pstypenames.insert(0, 'System.Software.Inventory')
Write-Output $Object
}
}
Catch {
Write-Warning "$Key : $_"
}
}
}
Catch { }
$reg.Close()
}
}
Else {
Write-Error "$($Computer): unable to reach remote system!"
}
$global:af_fp = $global:af_fp + $Computer + "_AF.csv"
}
}
}
Get-Software | ConvertTo-CSV -NoTypeInformation | Add-Content -Path $($af_fp)
'''
IGNORE FORMATTING PLEASE- HAD TROUBLE MAKING INDENTS CORRECTLY FROM COPY-PASTE AND RESTRICTIONS ON SITE FOR CODE BLOCKS.....
NOTE: the ONLY changes I made, that I am asking about, are the global declaration, the global variable update in the function, and the attempt to use the global variable for the -Path parameter....script otherwise runs and will even run WITH THE LAST LINE AS IS if I ran it and errored the first time.....not sure how the addition script will help in any way, shape, or form!
With a little effort, Nasir's solution worked! HOWEVER, I ran across a sample file that had a way of adding to a parameter that inspired me to make a change to my ORIGINAL, that also worked: remove global variable from script entirely and add this code the very end:
$file_suffix = '_AF.csv'
Get-Software | ConvertTo-CSV -NoTypeInformation | Add-Content -Path $env:COMPUTERNAME$file_suffix
In this way, I was able to accomplish exactly what I was setting out to do! Thanks Nasir for your response as well! I was able to also make that work as intended!
Global variables are generally frowned upon, since they often lead to poor scripts, with hard to debug issues.
It seems like your function returns some stuff, which you need to write to a file, the name of which is also generated by the same function. You can try something like this:
function function-name {
param($PathPrefix)
#do things
[pscustomobject]#{"DoThings_data" = $somevariablefromDoThings; "Filename" = "$($PathPrefix)$($variableFromDo_Things)_AF.csv"}
}
function-name -PathPrefix "C:\Path\to\folder\" | Foreach-Object { $_.DoThings_data | Export-Csv -Path $_.Filename -NoTypeInformation }
Or just have your function write the CSV data out and then return the data if you need to further process it outside the function.
Edit: this is just me extrapolating from partial code you have provided. To Lee_Dailey's point, yes, please provide more details.
I have been doing a lot of reading on invoke-expression (also known as iex) and I'm having trouble getting it to work for me.
My understanding is, it will run any powershell code you give to it. However, when I run my tests on it, it does not run the code.
Example:
## testcode.ps1
$myvar = "i am here"
if ($myvar -ne $null) {
"($myvar) variable is Full"
} else {
"($myvar) variable is Empty"
}
Now, if I cat(gc) this file and I pass it to iex, it outputs a bunch of errors. Same thing happens when I save the code into a variable and then feed the variable to iex. Neither works.
Despite the fact that I've tried numerous examples, I feel there's something minor I'm doing wrong that I'm hoping someone can point out for me.
I'm new to Windows scripting, so please bear with me. These are the results of the tests I performed:
First Test:
PS C:\Users\J> gc C:\Users\J\testcode.ps1 | iex
Invoke-Expression : Cannot bind argument to parameter 'Command' because it is an empty string.
At line:1 char:31
+ cat C:\Users\J\testcode.ps1 | iex
+ ~~~
+ CategoryInfo : InvalidData: (:PSObject) [Invoke-Expression], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Microsoft.PowerShell.Commands.InvokeExpressionCommand
iex : At line:1 char:23
+ if ($myvar -ne $null) {
+ ~
Missing closing '}' in statement block or type definition.
At line:1 char:31
+ cat C:\Users\J\testcode.ps1 | iex
+ ~~~
+ CategoryInfo : ParserError: (:) [Invoke-Expression], ParseException
+ FullyQualifiedErrorId : MissingEndCurlyBrace,Microsoft.PowerShell.Commands.InvokeExpressionCommand
Second Test:
PS C:\Users\J> $scriptBlock = gc C:\Users\J\testcode.ps1
PS C:\Users\J>
PS C:\Users\J> iex -Command "$scriptBlock"
iex : At line:1 char:23
+ $myvar = "i am here" if ($myvar -ne $null) { "($myvar) variable ...
+ ~~
Unexpected token 'if' in expression or statement.
At line:1 char:1
+ iex -Command "$scriptBlock"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ParserError: (:) [Invoke-Expression], ParseException
+ FullyQualifiedErrorId : UnexpectedToken,Microsoft.PowerShell.Commands.InvokeExpressionCommand
PS C:\Users\J>
I'm aware that I can just run the file containing the code. However, I need help figuring out how iex works and what it is I'm doing wrong.
Please kindly advise.
First things first:
Invoke-Expression should generally be avoided and used only as a last resort, due to its security risks. In short: avoid it, if possible, given that superior alternatives are usually available. If there truly is no alternative, only ever use it on input you either provided yourself or fully trust - see this answer.
For the record: in the case at hand, the superior alternative is to directly invoke the script file:
# Prepend `& `, if the script file path is quoted or references a variable.
C:\Users\J\testcode.ps1
Invoke-Expression (iex) accepts multiple strings via the pipeline, and evaluates each individually, as a self-contained script.
Therefore, you must provide the contents of your script as a whole, as a single string, which is what Get-Content's (gc's) -Raw switch does[1]:
Get-Content -Raw C:\Users\J\testcode.ps1 | Invoke-Expression
Alternatively, pass the script-file contents as an argument:
Invoke-Expression (Get-Content -Raw C:\Users\J\testcode.ps1)
Note that passing the string to evaluate as an argument truly only accepts a single string, so the command would fail without -Raw.
[1] By default, the Get-Content cmdlet reads a file line by line, passing each line through the pipeline as it is being read.
$myvar = "I'm Here"
#Using Invoke-Expression - Accepts a STRING as Input
$SBCode = 'if ($Null -ne $myvar) {"($myvar) variable is Full"}' +
'else {"`$myvar variable is Empty"}'
Clear-Host
"Before Invoke-Expression `$myvar = $myvar"
$Result = Invoke-Expression $SBCode
"Invoke-Expression Returns: $Result"
#Using Invoke-Command - Accepts Script Block as Input
$SBCode = {
if ($myvar -ne $null) {
"($myvar) variable is Full"
}
else {
"`$myvar variable is Empty"
}
} #End $SBCode Script Block
"Before Invoke-Command `$myvar = $myvar"
$Result = Invoke-Command -ScriptBlock $SBCode
"Invoke-Command Returns: $Result"
Results:
Before Invoke-Expression $myvar = I'm Here
Invoke-Expression Returns: (I'm Here) variable is Full
Before Invoke-Command $myvar = I'm Here
Invoke-Command Returns: (I'm Here) variable is Full
# After changing $MyVar = $Null
Before Invoke-Expression $myvar =
Invoke-Expression Returns: $myvar variable is Empty
Before Invoke-Command $myvar =
Invoke-Command Returns: $myvar variable is Empty
HTH
You can use out-string to convert output into string.
cat C:\Users\J\testcode.ps1 | out-string | Invoke-Expression
My Powershell script exited with "[ref] cannot be applied to a variable that does not exist" after running a while (it actually worked for a while)
The code snippet is something like
function outputData(...) {
$data = $null
if ($outputQueue.TryTake([ref] $data, 1000) -eq $false) {
continue
}
Write-Host $data
}
The detail errors thrown at the end are as below:
[ref] cannot be applied to a variable that does not exist.
At C:\Program Files\mfile.ps1:1213 char:13
+ if ($outputQueue.TryTake([ref] $data, 1000) -eq $ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (data:VariablePath) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : NonExistingVariableReference
May i ask if any thoughts about the cause ?
Thanks !
While error messages aren't always helpful, this one is:
It tells you that the $data variable you're trying to use with [ref] must already exist, i.e., must have been created explicitly, which in PowerShell means:
creating it by assigning a value to it - even if that value is $null,
or using New-Variable to create it.
A simplified example:
$data = $null # create variable $data
# OK to use $data with [ref], now that it exists.
# $data receives [int] value 10 in the process.
[int]::TryParse('10', [ref] $data)
I try to figure how to determine if a command throw with Invoke-Expression fail.
Even the variable $?, $LASTEXITCODE or the -ErrorVariable don't help me.
For example :
PS C:\> $cmd="cat c:\xxx.txt"
Call $cmd with Invoke-Expression
PS C:\> Invoke-Expression $cmd -ErrorVariable err
Get-Content : Cannot find path 'C:\xxx.txt' because it does not exist.
At line:1 char:4
+ cat <<<< c:\xxx.txt
+ CategoryInfo : ObjectNotFound: (C:\xxx.txt:String) [Get-Content], ItemNotFoundExcep
tion
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand
The $? is True
PS C:\> $?
True
The $LASTEXITCODE is 0
PS C:\> $LASTEXITCODE
0
And the $err is empty
PS C:\> $err
PS C:\>
The only way I found is to redirect STD_ERR in a file and test if this file is empty
PS C:\> Invoke-Expression $cmd 2>err.txt
PS C:\> cat err.txt
Get-Content : Cannot find path 'C:\xxx.txt' because it does not exist.
At line:1 char:4
+ cat <<<< c:\xxx.txt
+ CategoryInfo : ObjectNotFound: (C:\xxx.txt:String) [Get-Content], ItemNotFoundExcep
tion
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand
Is it the only and best way to do this ?
I was going crazy trying to make capturing the STDERR stream to a variable work. I finally solved it. There is a quirk in the invoke-expression command that makes the whole 2&>1 redirect fail, but if you omit the 1 it does the right thing.
function runDOScmd($cmd, $cmdargs)
{
# record the current ErrorActionPreference
$ep_restore = $ErrorActionPreference
# set the ErrorActionPreference
$ErrorActionPreference="SilentlyContinue"
# initialize the output vars
$errout = $stdout = ""
# After hours of tweak and run I stumbled on this solution
$null = iex "& $cmd $cmdargs 2>''" -ErrorVariable errout -OutVariable stdout
<# these are two apostrophes after the >
From what I can tell, in order to catch the stderr stream you need to try to redirect it,
the -ErrorVariable param won't get anything unless you do. It seems that powershell
intercepts the redirected stream, but it must be redirected first.
#>
# restore the ErrorActionPreference
$ErrorActionPreference=$ep_restore
# I do this because I am only interested in the message portion
# $errout is actually a full ErrorRecord object
$errrpt = ""
if($errout)
{
$errrpt = $errout[0].Exception
}
# return a 3 member arraylist with the results.
$LASTEXITCODE, $stdout, $errrpt
}
It sounds like you're trying to capture the error output of a native in a variable without also capturing stdout. If capturing stdout was acceptable, you'd use 2>&1.
Redirecting to a file might be the simplest. Using Invoke-Expression for it's -ErrorVariable parameter almost seems like a good idea, but Invoke-Expression has many problems and I usually discourage it.
Another option will look a little cumbersome, but it can be factored into a function. The idea is to merge output streams using 2>&1, but then split them again based on the type of the object. It might look like this:
function Split-Streams
{
param([Parameter(ValueFromPipeline=$true)]$InputObject)
begin
{
$stdOut = #()
$stdErr = #()
}
process
{
if ($InputObject -is [System.Management.Automation.ErrorRecord])
{
# This works well with native commands but maybe not as well
# for other commands that might write non-strings
$stdErr += $InputObject.TargetObject
}
else
{
$stdOut += $InputObject
}
}
end
{
,$stdOut
,$stdErr
}
}
$o, $e = cat.exe c:\xxx.txt 2>&1 | Split-Streams
I am working on a script that outputs about 600 variables across several dozen worksheets and it takes up about 500 lines of code just to add the column headers to each of the worksheets. In an effort to reduce the immense amount of overall code I am trying to put the action of creating the column headers to a function but for some reason set-variable won't set the value. It only gives me an error when I try and set sub values in the item.
This returns the following error
Function PopulateWorkSheet($Worksheet,$Values) {
ForEach ($Value in $Values) {
$ColumnCount++
Set-Variable -Name $($Worksheet).Cells.Items(1,$ColumnCount) -value $Value
}
Remove-Variable -Name ColumnCount
}
Error
You cannot call a method on a null-valued expression.
At line:4 char:30
+ Set-Variable -Name $($Worksheet).Cells.Items(1,$ColumnCount) -value $Val ...
+ ~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
But this doesn't return an error.
Function PopulateWorkSheet($Worksheet,$Values) {
ForEach ($Value in $Values) {
$ColumnCount++
Set-Variable -Name $($Worksheet) -value $Value
}
Remove-Variable -Name ColumnCount
}
My Full Code
Function PopulateWorkSheet($Worksheet,$Values) {
ForEach ($Value in $Values) {
$ColumnCount++
Set-Variable -Name $($Worksheet).Cells.Items(1,$ColumnCount) -value $Value
}
Remove-Variable -Name ColumnCount
}
clear
#Create Excel Com Object
$Excel = New-Object -com Excel.Application
# Make the Excel Application Visible to the end user
$Excel.visible = $True
# Create a WorkBook inside the Excel application
# that we can start manipulating.
$Excel_Workbook = $Excel.Workbooks.Add()
#=======================================================
# Now that we have a workbook we need to create some
# additional worksheets (Tabs) beyond that initial 3
# that are created when the workbook is opened.
#=======================================================
$Temp_MakeWorkSheet = $Excel.Worksheets.Add()
#=======================================================
# Once all the sheets are created each of the worksheets
# need to be assigned to a variable so they can be
# manipulated.
#=======================================================
$Excel_Worksheet_SystemSummary = $Excel.Worksheets.Item(1)
$Excel_Worksheet_SystemSummary.Name = "System Summary"
#=======================================================
# = System Summary = Create Table Headers
#=======================================================
PopulateWorkSheet -Worksheet Excel_Worksheet_SystemSummary -value #("Computer Name","Serial Number","Operating System","Service Pack","OS Architecture","Part of Domain","Domain Role","System Uptime","IP Address","OS HD Smarts Check","# of Mice","# of Keyboards","# of Hard Drives","# of CD/DVD Drives","# of Network Adapters")
Why do you think you need to use Set-Variable at all? Set-Variable is for setting Powershell variables when you don't know the name in advance, or when you want to do special things like making them read-only. In this case you don't want to set a Powershell variable at all, you want to set the value of a cell in a worksheet.
So forget Set-Variable. Also forget Remove-Variable: returning from a function removes all the local variables anyway, you don't have to do it yourself.
$Worksheet, if I read your code correctly is simply a reference to the worksheet. So why did you write $($Worksheet)? That doesn't actually do anything that just writing $Worksheet wouldn't have done on its own.
Now consider $Worksheet.Cells.Items(1,$ColumnCount). The first two elements won't change inside your loop, so hoist them out of the loop. Now you should have something like:
Function PopulateWorkSheet($Worksheet,$Values) {
$cells = $Worksheet.Cells
$ColumnCount = 0
ForEach ($Value in $Values) {
$ColumnCount++
$cells.Item(1,$ColumnCount) = $Value
}
}
This also works:
Function PopulateWorkSheet($Worksheet,$Values) {
$Worksheet.Range(
$Worksheet.Cells.Item(1,1),
$Worksheet.Cells.Item(1,$Values.Count)).value() = $Values
}
and will assign the complete list in one go.
Instead of
Set-Variable -Name $($Worksheet).Cells.Items(1,$ColumnCount) -value $Value
Try
$var=Get-Variable -Name $($Worksheet)
$var.Cells.Items(1,$ColumnCount)=$Value
Note...I haven't tried this, but there's no reason it shouldn't work.
So I was way overthinking it. I did not need to try and load the variable in the function, but pass value to the "Worksheet" variable since the $Excel_Worksheet_SystemSummary is just a link to the cell contents, and it doesn't actually hold any real information.
Function PopulateWorkSheet($Worksheet,$Values) {
ForEach ($Value in $Values) {
$ColumnCount++
$worksheet.Cells.Item(1,$ColumnCount)=$Value
}
Remove-Variable -Name ColumnCount
}