Start-Sleep memory leak (or causing powershell memory leak)? - powershell

This question is a repost from https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/365ca396-400d-4401-bd7f-05d5d6c22cc8/powershells-startsleep-causes-memory-growth?forum=winserverpowershell
I did not get a working answer there so try if this forum can help. This is blocking my work.
Here is my two powershell scripts. The first one is to create an event source and the second one is to generate events using 4 threads. Start-Sleep is used in each thread to control the event generation rate. If I remove the Start-Sleep, then the powershell memory usage is constant, otherwise it grows fast until all system memory is used and the system becomes extremely slow.
Is this a known issue? Any workaround? Appreciate any clue.
# use this script to create channel and source if they does not exist.
$logName = "TestLog"
$sourceName = "TestSource"
New-EventLog -LogName $logName -Source $sourceName
# maximumSize's max is 4G.
Limit-EventLog -LogName $logName -OverflowAction OverWriteOlder -RetentionDays 30 -MaximumSize 3080000KB
Event generating script:
# use this script to generate events in TestLog channel.
Param(
[int]$sleepIntervalInMilliSeconds = 0
)
$eventGenScript = {
$logName = "TestLog"
$sourceName = "TestSource"
while($true) {
Write-EventLog -LogName $logName -Source $sourceName -Message "perfLog" -EventId 0 -EntryType information
Start-Sleep -ms $sleepIntervalInMilliSeconds
}
}
$threadCount = 4
for($i=0; $i -lt $threadCount; $i++)
{
Start-Job $eventGenScript
}
read-host "type a key to exit. You need to wait for some time for threads to exit."

It's not that Start-Sleep has a memory leak, you're calling it with an invalid Parameter (-ms) and the process memory of the jobs is filling up with error messages, because you keep calling the (invalid) statement in an infinite loop.
Demonstration:
PS C:\> Start-Sleep -ms 100
Start-Sleep : A parameter cannot be found that matches parameter name 'ms'.
At line:1 char:13
+ Start-Sleep -ms 100
+ ~~~
+ CategoryInfo : InvalidArgument: (:) [Start-Sleep], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.StartSleepCommand
Also, you're defining $sleepIntervalInMilliSeconds outside the scriptblock, but try to use it inside the scriptblock, which won't work, because the variable is undefined in the scope of the scriptblock. This is why your problem remained despite the correct advice you got on the Microsoft forums.
PS C:\> $ms = 100
PS C:\> $job = Start-Job -ScriptBlock { Start-Sleep -Milliseconds $ms }
PS C:\> $job | Wait-Job | Receive-Job
Cannot validate argument on parameter 'Milliseconds'. The argument is null,
empty, or an element of the argument collection contains a null value.
Supply a collection that does not contain any null values and then try the
command again.
+ CategoryInfo : InvalidData: (:) [Start-Sleep], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.StartSleepCommand
+ PSComputerName : localhost
You have three options to deal with this:
Define the variable inside the script block:
PS C:\> $job = Start-Job -ScriptBlock {
>> $ms = 100
>> $ms
>> Start-Sleep -Milliseconds $ms
>> }
>>
PS C:\> $job | Wait-Job | Receive-Job
100
Use the using scope modifier to get the local variable:
PS C:\> $ms = 100
PS C:\> $job = Start-Job -ScriptBlock {
>> $using:ms
>> Start-Sleep -Milliseconds $using:ms
>> }
>>
PS C:\> $job | Wait-Job | Receive-Job
100
Pass the variable into the scriptblock as an argument:
PS C:\> $ms = 100
PS C:\> $job = Start-Job -ScriptBlock {
>> param($ms)
>> $ms
>> Start-Sleep -Milliseconds $ms
>> } -ArgumentList $ms
>>
PS C:\> $job| Wait-Job | Receive-Job
100
Bottom line: replace
Start-Sleep -ms $sleepIntervalInMilliSeconds
with
Start-Sleep -Milliseconds $using:sleepIntervalInMilliSeconds
and the problem will disappear.

Related

Why does an argument passed into a script block fail to work?

Full disclosure: My problem may be based on a incomplete understanding of the Citrix PowerShell module for Xen Desktop.
I have the following script block. It's called in a loop, once for each VM in a list. I'm using PowerShell Jobs because I want to keep the UI thread free to update the UI while the jobs run.
Code "A"
$j = Start-Job -Name $jobName -ScriptBlock {
param($url, $uuid, $cred, $snapshotName)
$ErrorActionPreference = "Stop"
try
{
$error.clear()
$xSS = $cred | Connect-XenServer -url $url -NoWarnCertificates -SetDefaultSession -PassThru;
$vm = (Get-XenVM -SessionOpaqueRef $xss.opaque_ref -uuid $uuid)
#Create snapshot
Invoke-XenVM -Async -SessionOpaqueRef $xss.opaque_ref -VM $vm -XenAction Snapshot -NewName $snapshotName
return "OK"
}
catch
{
return ("ERROR: "+$error)
}
} -ArgumentList $global:configFileVmMetaData.poolUrl, $xenVm.key, $global:cred, $snapshotName
Code "A" works ok, but it takes longer then necessary because I'm executing the Connect-XenServer cmdlet each time I invoke the script block.
So, I tried calling Connect-XenServer once outside the loop and passing in the session variable as shown below in Code "B". The result was the error Could not find open sessions to any XenServers being thrown inside the script block. I'm assuming that the $xss session variable is getting whanged somehow when its passed into the script block.
Any ideas why $xss session variable is getting whanged?
Code "B"
$xSS = $cred | Connect-XenServer -url $global:configFileVmMetaData.poolUrl -NoWarnCertificates -SetDefaultSession -PassThru;
loop
{
$j = Start-Job -Name $jobName -ScriptBlock {
param($xss, $uuid, $snapshotName)
$ErrorActionPreference = "Stop"
try
{
$error.clear()
$vm = (Get-XenVM -SessionOpaqueRef $xss.opaque_ref -uuid $uuid)
#Create snapshot
Invoke-XenVM -Async -SessionOpaqueRef $xss.opaque_ref -VM $vm -XenAction Snapshot -NewName $snapshotName
return "OK"
}
catch
{
return ("ERROR: "+$error)
}
} -ArgumentList $xss, $xenVm.key, $snapshotName
}
Additional examples inspired by the Robert Cotterman answer
In all cases, I still get the Could not find open sessions to any
XenServers error
FYI - Using PowerShell 5.1
Example using $using. Variable contents are passed in and back out as expected
cls
$aLocal = "AAA"
$bLocal = "BBB"
$j = Start-Job -Name "TestJob" -ScriptBlock {
return ($using:aLocal + " *** " + $using:bLocal)
}
while ($true)
{
$g = get-job -name "TestJob"
write-host ("get-job " + $g.Name + " is " + $g.State)
if ($g.State -ne "Running")
{
break
}
start-sleep -Seconds 1
}
write-host ("receive-Job='" + (receive-Job -Name "TestJob") +"'")
$g = get-Job -Name "TestJob"
Write-Host ("get-Job "+$g.name + " " + $g.state + " " + $g.HasMoreData + " " + $g.id)
if($g)
{
Remove-Job -Name "TestJob"
}
Output
get-job TestJob is Running
get-job TestJob is Completed
receive-Job='AAA *** BBB'
get-Job TestJob Completed False 45
Remove-Job
Example using args. Variable contents are passed in and back out as expected
cls
$aLocal = "AAA"
$bLocal = "BBB"
$j = Start-Job -Name "TestJob" -ScriptBlock {
return ($args[0] + " *** " + $args[1])
} -ArgumentList ($aLocal, $bLocal)
while ($true)
{
$g = get-job -name "TestJob"
write-host ("get-job " + $g.Name + " is " + $g.State)
if ($g.State -ne "Running")
{
break
}
start-sleep -Seconds 1
}
write-host ("receive-Job='" + (receive-Job -Name "TestJob") +"'")
$g = get-Job -Name "TestJob"
Write-Host ("get-Job "+$g.name + " " + $g.state + " " + $g.HasMoreData + " " + $g.id)
if($g)
{
Remove-Job -Name "TestJob"
}
Output
get-job TestJob is Running
get-job TestJob is Completed
receive-Job='AAA *** BBB'
get-Job TestJob Completed False 49
Example using named args. Variable contents are passed in and back out as expected
cls
$aLocal = "AAA"
$bLocal = "BBB"
$j = Start-Job -Name "TestJob" -ScriptBlock {
param($a, $b)
return ($a + " *** " + $b)
} -ArgumentList ($aLocal, $bLocal)
while ($true)
{
$g = get-job -name "TestJob"
write-host ("get-job " + $g.Name + " is " + $g.State)
if ($g.State -ne "Running")
{
break
}
start-sleep -Seconds 1
}
write-host ("receive-Job='" + (receive-Job -Name "TestJob") +"'")
$g = get-Job -Name "TestJob"
Write-Host ("get-Job "+$g.name + " " + $g.state + " " + $g.HasMoreData + " " + $g.id)
if($g)
{
Remove-Job -Name "TestJob"
}
Output
get-job TestJob is Running
get-job TestJob is Completed
receive-Job='AAA *** BBB'
get-Job TestJob Completed False 55
I just don't think it works that way. The connection to the Xen server is created for your PowerShell session, with info about that connection collected in the $xss variable. Each job runs its own PowerShell session. So just passing $xss to the job isn't the same, it's just a description of the connection details.
Jobs and invoke-command require you to specify that you're using your variable. Simply change the variable from
$variable
To
$using:variable
Internal variables do not need this. But calling upon variables from the parent script do.
Alternatively, Since you're passing $xss as an argument, you wouldn't call it with $xss but rather
$args[0]
Since it's your first argument. And $args[1] for the second etc...
The reason is because the entire xss variable is being printed as an argument and is not named inside the job. It's given the name $args and has a place in the first slot (0).
I prefer $using:variable as it reduces confusion

Powershell stop-process not killing Logstash

I am writing a powershell script that allows Logstash to run for a certain amount of time and then stops the program by process id. Here is a basic version of the code:
$logstashFilepath = "C:\Users\emckenzie\Downloads\logstash-5.3.2\logstash-5.3.2\bin\logstash.bat"
$logstashArguments = "-f C:\users\emckenzie\Downloads\logstash-5.3.2\logstash-5.3.2\test_loader.conf"
$logstashprocess = Start-Process -FilePath $logstashFilepath -ArgumentList $logstashArguments -PassThru
$logstashid = $logstashprocess.Id
Write-Host $logstashid
Start-Sleep -s 60
Stop-Process $logstashid
In this test case Logstash only writes from stdin to stdout. When I run this program the output looks like this:
17120
Stop-Process : Cannot find a process with the process identifier 17120.
At C:\Users\emckenzie\Documents\Loaders\testLoader.ps1:13 char:13
+ Stop-Process <<<< $logstashid
+ CategoryInfo : ObjectNotFound: (17120:Int32) [Stop-Process], ProcessCommandException
+ FullyQualifiedErrorId : NoProcessFoundForGivenId,Microsoft.PowerShell.Commands.StopProcessCommand
Is this a problem in Logstash or Powershell?
Any help would be much appreciated!
I ended up using NSSM to start Logstash as a service instead, and I control NSSM through Powershell.
# You must configure testlogstash using the GUI NSSM provides
$logstashFilepath = "C:\Users\emckenzie\Downloads\nssm-2.24\nssm-
2.24\win64\nssm.exe"
$start = "start testlogstash"
$stop = "stop testlogstash"
$logstashprocess = Start-Process -FilePath $logstashFilepath -ArgumentList
$start -PassThru
Start-Sleep -s 60
Start-Process -FilePath $logstashFilepath -ArgumentList $stop
After reading the comments on the original post I believe there is a solution for you. If you know the program running that needs to stay running until the job is finished:
$logstashprocess = Start-Process -FilePath $logstashFilepath -ArgumentList $logstashArguments -PassThru
$logstashid = $logstashprocess.Id
$Global:IsRunning = $false
Do {
$process = Get-Process -processname "NameOfProcessHere"
If ($process){
$IsRunning = $true
}else{
$IsRunning = $false
}
} While ($IsRunning -eq $true)
Stop-Process -processname "$logstashid"
Let me know if this is helpful, or if i'm not understanding the question.

Function execution error: Value cannot be null [duplicate]

This question already has answers here:
How do I pass multiple parameters into a function in PowerShell?
(15 answers)
Closed 7 years ago.
I have created a PowerShell script that is triggered via Task Scheduler when a windows event occur.
The script is expected to receive eventID, eventRecordId and channel and gets event detail from Get-WinEvent and display the detail via a pop up.
Param( $eventID, $eventRecordId, $channel)
#$ErrorActionPreference="SilentlyContinue"
Stop-Transcript | out-null
$ErrorActionPreference = "Continue"
Start-Transcript -path C:\temp\output.txt -append
function PopUPEventInfo ( $eventID, $eventRecordId, $channel)
{
Add-Content -Value $eventID -Path C:\users\CFNLocalAdmin\data.txt
Add-Content -Value $eventRecordId -Path C:\users\CFNLocalAdmin\data.txt
Add-Content -Value $channel -Path C:\users\CFNLocalAdmin\data.txt
if($eventID -eq 7036)
{
$events = Get-WinEvent -LogName 'System' -FilterXPath "<QueryList><Query Path='System'><Select >*[System[(EventRecordID=8478)]]</Select></Query></QueryList>"
foreach($event in $events)
{
$eventParams=$event.ToXml()
$eventXml = [xml]$event.ToXml()
#Write-Host "Attempting parsing xml event"
$SysTime=$eventXml.Event.System.TimeCreated.SystemTime
#Write-Host $SysTime
$ProviderName=$eventXml.Event.System.Provider.Name
#Write-Host $ProviderName
$ServiceName=""
$ServiceStatus=""
$ServiceName= $eventXml.Event.EventData.Data[0].'#text'
$ServiceStatus=$eventXml.Event.EventData.Data[1].'#text'
$popipObj = New-Object -ComObject wscript.shell
$popipObj.popup("RecordID: "+$eventRecordId +", Channel :"+$channel+"Event Timestamp: "+$ServiceName +": "+$ServiceStatus)
}
}
}
PopUPEventInfo $eventID, $eventRecordId, $channel
The line
$events = Get-WinEvent -LogName 'System' -FilterXPath "<QueryList><Query Path='System'><Select >*[System[(EventRecordID=8478)]]</Select></Query></QueryList>"
works fine but when I replace constants with variables
$events = Get-WinEvent -LogName $channel -FilterXPath "<QueryList><Query Path='$channel'><Select >*[System[(EventRecordID=$eventRecordId)]]</Select></Query></QueryList>"
I get the following error.
TerminatingError(Get-WinEvent): "Value cannot be null.
Parameter name: collection"
Get-WinEvent : Value cannot be null.
Parameter name: collection
At C:\Users\CFNLocalAdmin\test.ps1:26 char:11
+ $events = Get-WinEvent -LogName $channel -FilterXPath "<QueryList><Query Path='S ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-WinEvent], ArgumentNullException
+ FullyQualifiedErrorId : System.ArgumentNullException,Microsoft.PowerShell.Commands.GetWinEventCommand
I am not sure what I am doing wrong. I can see the values of $eventID, $eventRecordId and $channel getting written to the data file but why the Get-WinEvent is giving null exception
I appreciate if anyone point me to the right direction
Most likely cause for the error is $channel being null. You are saying that you can see that it's not null because it's written to a file but I'm sceptical. Perhaps you are seeing values there from previous runs.
If you by any chance have a Cyrillic keyboard, or if you copied your script from a Cyrillic source, make sure that c, a and e in the $channel are not Cyrillic с, а and е. Powershell can stomach them happily, but $channel and $сhаnnеl are two different variables, even if you can't see that.
$channel = 'Hello'
$сhаnnеl = 'Again'
Write-Host $channel
Write-Host $сhаnnеl
Gives this output:
PS C:\WINDOWS\system32> $channel = 'Hello'
PS C:\WINDOWS\system32> $сhаnnеl = 'Again'
PS C:\WINDOWS\system32> Write-Host $channel
Hello
PS C:\WINDOWS\system32> Write-Host $сhаnnеl
Again
PS C:\WINDOWS\system32>
The issue that you are experiencing is that you are calling the function incorrectly. You have placed commas between your parameters when you call it, making them an array, so you are effectively passing all three variables to your first parameter.
PopUPEventInfo $eventID, $eventRecordId, $channel
Is effectively seen as this:
PopUPEventInfo -EventId #($eventID, $eventRecordId, $channel) -EventRecordId $null -Channel $null
If you simply remove the commas the command should work as expected.
PopUPEventInfo $eventID $eventRecordId $channel
Or fully:
PopUPEventInfo -EventId $eventID -EventRecordId $eventRecordId -Channel $channel

Powershell: passing parameters to functions stored in variables

I'm trying to get a simple working example of using functions inside of jobs. I've managed to pass my function into the scriptblock used for my job, but I can't seem to get parameters to the function.
# concurrency
$Logx =
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
# Execution starts here
cls
$colors = #("red","blue","green")
$colors | %{
$scriptBlock =
{
Invoke-Expression -Command $args[1]
Start-Sleep 3
}
Write-Host "Processing: " $_
Start-Job -scriptblock $scriptBlock -args $_, $Logx
}
Get-Job
while(Get-Job -State "Running")
{
write-host "Running..."
Start-Sleep 2
}
# Output
Get-Job | Receive-Job
# Cleanup jobs
Remove-Job *
Here's the output:
Processing: red
Id Name State HasMoreData Location Command
-- ---- ----- ----------- -------- -------
175 Job175 Running True localhost ...
Processing: blue
177 Job177 Running True localhost ...
Processing: green
179 Job179 Running True localhost ...
179 Job179 Running True localhost ...
177 Job177 Running True localhost ...
175 Job175 Running True localhost ...
Running...
Running...
OUT:
OUT:
OUT:
So as evidenced by the OUT: x3 in the output my function is getting called, but I haven't found any syntax that allows me to get the parameter to the function. Thoughts?
EDIT:
Note in Shawn's observation below and my response I tried using functions as variables because using a traditional function does not seem to work. If there is a way to get that working I'd be more than happy to not have to pass my functions around as variables.
The answer is to use the initializationscript parameter of Start-Job. If you define all your functions in a block and pass the block they become available.
Solution was found in this post:
How do I Start a job of a function i just defined?
Here is my example from before, now working:
# concurrency
$func = {
function Logx
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
}
# Execution starts here
cls
$colors = #("red","blue","green")
$colors | %{
$scriptBlock =
{
Logx $args[0]
Start-Sleep 9
}
Write-Host "Processing: " $_
Start-Job -InitializationScript $func -scriptblock $scriptBlock -args $_
}
Get-Job
while(Get-Job -State "Running")
{
write-host "Running..."
Start-Sleep 2
}
# Output
Get-Job | Receive-Job
# Cleanup jobs
Remove-Job *
If you do not prefix your function name with keyword function, PowerShell does not know to treat it as such. As you have written your script it is basically a variable with some special text in it. Which as your output shows it is only executing the commands it recognizes within that variable's content: Write-Host "OUT:".
Using the correct syntax will tell PowerShell it is a function and that you have variables to pass into it that you need executed:
function Logx
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
Then when you call it within your script you will just use Logx
Got this far. Have to run out, will try back later.
PS: What is getting passed at args[1], I am getting a lot of red,
CategoryInfo : InvalidData: (:) [Invoke-Expression], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.InvokeExpressionCommand
here is what I've managed so far.
# concurrency
$Logx =
{
param(
[parameter(ValueFromPipeline=$true)]
$msg
)
Write-Host ("OUT:"+$msg)
}
# Execution starts here
cls
$colors = #("red","blue","green")
$colors | %{
& $scriptBlock =
{ Invoke-Expression -Command $args[1]
Start-Sleep 3
}
Write-Host "Processing: " $_
Start-Job -scriptblock $scriptBlock -ArgumentList #($_, $Logx)
}
# Get-Job
while(Get-Job -State "Running")
{
write-host "Running..."
Start-Sleep 2
}
# Output
Get-Job | Receive-Job
# Cleanup jobs
Remove-Job *

Powershell start-job -scriptblock cannot recognize the function defined in the same file?

I have the following code.
function createZip
{
Param ([String]$source, [String]$zipfile)
Process { echo "zip: $source`n --> $zipfile" }
}
try {
Start-Job -ScriptBlock { createZip "abd" "acd" }
}
catch {
$_ | fl * -force
}
Get-Job | Wait-Job
Get-Job | receive-job
Get-Job | Remove-Job
However, the script returns the following error.
Id Name State HasMoreData Location Command
-- ---- ----- ----------- -------- -------
309 Job309 Running True localhost createZip "a...
309 Job309 Failed False localhost createZip "a...
Receive-Job : The term 'createZip' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:17 char:22
+ Get-Job | receive-job <<<<
+ CategoryInfo : ObjectNotFound: (function:createZip:String) [Receive-Job], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
It seems the function name cannot be recognized inside the script block of start-job. I tried function:createZip too.
Start-Job actually spins up another instance of PowerShell.exe which doesn't have your createZip function. You need to include it all in a script block:
$createZip = {
param ([String]$source, [String]$zipfile)
Process { echo "zip: $source`n --> $zipfile" }
}
Start-Job -ScriptBlock $createZip -ArgumentList "abd", "acd"
An example returning an error message from the background job:
$createZip = {
param ([String] $source, [String] $zipfile)
$output = & zip.exe $source $zipfile 2>&1
if ($LASTEXITCODE -ne 0) {
throw $output
}
}
$job = Start-Job -ScriptBlock $createZip -ArgumentList "abd", "acd"
$job | Wait-Job | Receive-Job
Also note that by using a throw the job object State will be "Failed" so you can get only the jobs which failed: Get-Job -State Failed.
If you are still new to using start-job and receive-job, and want to debug your function more easily, try this form:
$createZip = {
function createzipFunc {
param ([String]$source, [String]$zipfile)
Process { echo "zip: $source`n --> $zipfile" }
}
#other funcs and constants here if wanted...
}
# some secret sauce, this defines the function(s) for us as locals
invoke-expression $createzip
#now test it out without any job plumbing to confuse you
createzipFunc "abd" "acd"
# once debugged, unfortunately this makes calling the function from the job
# slightly harder, but here goes...
Start-Job -initializationScript $createZip -scriptblock {param($a,$b) `
createzipFunc $a $b } -ArgumentList "abc","def"
All not made simpler by the fact I did not define my function as a simple filter as you have, but which I did because I wanted to pass a number of functions into my Job in the end.
Sorry for digging this thread out, but it solved my problem too and so elegantly at that. And so I just had to add this little bit of sauce which I had written while debugging my powershell job.