I am attempting to mock up a data structure and logic to handle deferred log writing, and I am running into an odd situation. if ([collections.arrayList]$deferredLog = Get-PxDeferredLogItems) { errors when the returned arrayList is only one item, with
Cannot convert the "logged: 07/20/2019 10:56:29" value of type "System.String" to type "System.Collections.ArrayList".
Once there are two items it's fine. Now I know an array of a single item needs to be forced to be an array because PS is sometimes too clever for it's own good, but I thought the Add method of a [collections.arrayList] produced an arrayList even on the first Add. AM I totally misunderstanding, or is my logic wrong somewhere else?
I also tried casting the results of the function call like this
if ([collections.arrayList]$deferredLog = [collections.arrayList](Get-PxDeferredLogItems)) { but that is even worse, it errors at every call. I also tried initializing the variable to an empty arrayList with Set-PxDeferredLogItems (New-Object collections.arrayList) in Main, rather the n initializing it on the first call of Add-PxDeferredLogItem, but that shows the same behavior, errors until the deferred log has two items. I verified this by changing the Write-Host directly after the erroring line to Write-Host "Process deferred items $($deferredLog.count)", and I get errors till it shows a 2, then everything works as expected.
function Get-PxDeferredLogItems {
return $script:deferredLogItems
}
function Set-PxDeferredLogItems {
param (
[collections.arrayList]$deferredLog
)
[collections.arrayList]$script:deferredLogItems = $deferredLog
}
function Add-PxDeferredLogItem {
param (
[string]$item
)
if (-not $script:deferredLogItems) {
[collections.arrayList]$script:deferredLogItems = New-Object collections.arrayList
}
[void]$script:deferredLogItems.Add($item)
}
function Add-PxLogContent {
param (
[string]$string
)
function Assert-PxWrite {
if ((Get-Random -minimum:1 -maximum:4) -gt 1) {
return $true
} else {
return $false
}
}
if ([collections.arrayList]$deferredLog = Get-PxDeferredLogItems) {
Write-Host "Process deferred items"
:deferredItemsWrite do {
if (Assert-PxWrite) {
Write-Host "$($deferredLog[0]) ($(Get-Date))"
$deferredLog.RemoveAt(0)
} else {
break :deferredItemsWrite
}
} while ($deferredLog.count -gt 0)
if ($deferredLog.count -eq 0) {
$deferredLogPending = $false
} else {
$deferredLogPending = $true
}
Set-PxDeferredLogItems $deferredLog
} else {
$deferredLogPending = $false
}
if (-not $deferredLogPending) {
if (Assert-PxWrite) {
Write-Host $string
} else {
Add-PxDeferredLogItem $string
}
} else {
Add-PxDeferredLogItem $string
}
}
### MAIN
$startTime = Get-Date
$endTime = $startTime + (New-TimeSpan -minutes:2)
Clear-Host
do {
Add-PxLogContent "logged: $(Get-Date)"
Start-SLeep -s:5
} while ((Get-Date) -lt $endTime)
if ($deferredLog = Get-PxDeferredLogItems) {
foreach ($item in $deferredLog) {
Write-Host "!$item"
}
}
EDIT: It seems to be related to passing the arrayList, as this works as expected when there are no functions involved.
Clear-Host
$deferredLogItems = New-Object collections.arrayList
Write-Host "$($deferredLogItems.getType()) $($deferredLogItems.count) $deferredLogItems"
[void]$deferredLogItems.Add('One')
Write-Host "$($deferredLogItems.getType()) $($deferredLogItems.count) $deferredLogItems"
[void]$deferredLogItems.Add('Two')
Write-Host "$($deferredLogItems.getType()) $($deferredLogItems.count) $deferredLogItems"
[void]$deferredLogItems.Add('Three')
Write-Host "$($deferredLogItems.getType()) $($deferredLogItems.count) $deferredLogItems"
function Get-DeferredLogItems {
if (-not $script:deferredLogItems) {
[collections.arrayList]$script:deferredLogItems = New-Object collections.arrayList
}
[void]$script:deferredLogItems.Add("$($script:deferredLogItems.count + 1)")
return $script:deferredLogItems
}
$script:deferredLogItems = $localDeferredLogItems = $null
foreach ($i in 1..5) {
[collections.arrayList]$localDeferredLogItems = Get-DeferredLogItems
Write-Host "$($localDeferredLogItems.getType()) $($localDeferredLogItems.count) $localDeferredLogItems"
}
EDIT2: Replaced the line # reference with the actual offending line of code, because Confusion.
EDIT3: For what it's worth, forcing the results of Get-DeferredLogItems into an array DOES work. But I still wonder why that's even needed. Why does the Add method produce a correct arrayList at first add but this gets #%#$ed up by PowerShell when passed as a return value?
Related
I have a scenario where I am doing any number of things with a network resource, copying files or folders, deleting a ZIP file after it has been unzipped locally, checking an EXE to see if it was downloaded from the internet and needs unblocked, running a software install, etc. And all of these tasks are impacted by things like an installer that hasn't released a file lock on a log file to be deleted, or the unzip hasn't released the file lock on the zip file, or the network is for a fleeting moment "unavailable" so a network file is not found.
I have a technique that works well for handling this scenario, I do a loop and react to the specific exception, and when that exception occurs I just sleep for 6 seconds. The loop happens up to 10 times before I abandon. Here is the code for dealing with an Autodesk log that is still locked by the Autodesk installer.
$waitRetryCount = 10
:waitForAccess do {
try {
Remove-Item $odisLogPath -Recurse -errorAction:Stop
$retry = $False
} catch {
if (($PSItem.Exception -is [System.IO.IOException]) -or ($PSItem.Exception.InnerException -and ($PSItem.Exception.InnerException -is [System.IO.IOException]))) {
if ($waitRetryCount -eq 0) {
$invokeODISLogManagement.log.Add('E_Error deleting ODIS logs: retry limit exceeded')
break waitForFolderAccess
} else {
$retry = $True
$waitRetryCount --
Start-Sleep -s:6
}
} else {
$invokeODISLogManagement.log.Add('E_Error deleting ODIS logs')
$invokeODISLogManagement.log.Add("=_$($PSItem.Exception.GetType().FullName)")
$invokeODISLogManagement.log.Add("=_$($PSItem.Exception.Message)")
if ($PSItem.Exception.InnerException) {
$invokeODISLogManagement.log.Add("=_$($PSItem.Exception.InnerException.GetType().FullName)")
$invokeODISLogManagement.log.Add("=_$($PSItem.Exception.InnerException.Message)")
}
$retry = $False
}
}
} while ($retry)
The thing is, I would like to convert this to a function, since it needs to be handled in a lot of places. So I would need to pass to the function the specific exception I am looking for and the code to be run in the try block, and get back a log (as a generic.list) that I can then add to the actual log list. The first and last aspects I have, but I am unsure the best approach for the code to try. In the above example it is a single line, Remove-Item $odisLogPath -Recurse -errorAction:Stop, but it could be multiple lines I suspect.
To start playing with this I verified that this does seem to work, at least with a single line of code.
$code = {Get-Item '\\noServer\folder\file.txt' -errorAction:Stop}
try {
& $code
} catch {
Write-Host "$($_.Exception.GetType().FullName)"
}
But the error action is going to be duplicated a lot, so I thought to maybe address that within the function, however
$code = {Get-Item '\noServer\folder\file.txt'}
try {
& $code -errorAction:Stop
} catch {
Write-Host "$($_.Exception.GetType().FullName)"
}
does NOT work. I get the exception uncaught.
So, my questions are
1: Is this the right direction? I am pretty sure it is but perhaps someone has a gotcha that I am not seeing, yet. :)
2: Is there a mechanism to add the -errorAction:Stop in the try, so I don't need to do it/remember to do it, at every use of this new function.
3: I seem to remember reading about a programming concept of passing code to a function, and I can't remember what that is called, but I would like to know the generic term. Indeed, it probably would help if I could tag it for this post. I had thought it might be lama, but a quick search suggests that is not the case? Being self taught sucks sometimes.
EDIT:
I have now implemented a function, that starts to do what I want.
function Invoke-PxWaitForAccess {
param (
[System.Management.Automation.ScriptBlock]$code,
[String]$path
)
try {
(& $code -path $path)
} catch {
Return "$($_.Exception.GetType().FullName)!!"
}
}
$path = '\\noServer\folder\file.txt'
$code = {param ([String]$path) Write-Host "$path!"; Get-Item $path}
Invoke-PxWaitForAccess -code $code -path $path
I do wonder if the path couldn't somehow be encapsulated in the $code variable itself, since this implementation means it can ONLY be used where the code being run has a single variable called $path.
And, still wondering if this really is the best, or even a good, way to proceed? Or are there arguments for just implementing my loop 50 some odd times in all the situations where I need this behavior.
Also worth noting that this code does not yet implement the loop or address the fact that different exceptions apply in different situations.
EDIT #2:
And here is a more complete implementation, though it fails because it seems I am not actually passing a type, even though it looks like I am. So I get an error because what is to the right of -is must be an actual type.
function Invoke-PxWaitForAccess {
param (
[System.Management.Automation.ScriptBlock]$code,
[String]$path,
[Type]$exceptionType
)
$invokeWaitForAccess = #{
success = $Null
log = [System.Collections.Generic.List[String]]::new()
}
$waitRetryCount = 2
:waitForAccess do {
try {
Write-Host "$path ($waitRetryCount)"
& $code -path $path
$retry = $False
} catch {
Write-Host "!$($PSItem.Exception.GetType().FullName)"
if (($PSItem.Exception -is $exceptionType) -or ($PSItem.Exception.InnerException -and ($PSItem.Exception.InnerException -is $exceptionType))) {
Write-Host "($waitRetryCount)"
if ($waitRetryCount -eq 0) {
$invokeWaitForAccess.log.Add('E_retry limit exceeded')
break waitForFolderAccess
} else {
$retry = $True
$waitRetryCount --
Start-Sleep -s:6
}
} else {
$invokeWaitForAccess.log.Add("=_$($PSItem.Exception.GetType().FullName)")
$invokeWaitForAccess.log.Add("=_$($PSItem.Exception.Message)")
if ($PSItem.Exception.InnerException) {
$invokeWaitForAccess.log.Add("=_$($PSItem.Exception.InnerException.GetType().FullName)")
$invokeWaitForAccess.log.Add("=_$($PSItem.Exception.InnerException.Message)")
}
$retry = $False
}
}
} while ($retry)
if ($invokeWaitForAccess.log.Count -eq 0) {
$invokeWaitForAccess.success = $True
} else {
$invokeWaitForAccess.success = $False
}
return $invokeWaitForAccess
}
$path = '\\noServer\folder\file.txt'
$code = {param ([String]$path) Get-Item $path -errorAction:Stop}
if ($invoke = (Invoke-PxWaitForAccess -code $code -path $path -type ([System.Management.Automation.ItemNotFoundException])).success) {
Write-Host 'Good'
} else {
foreach ($line in $invoke.log) {
Write-Host "$line"
}
}
EDIT #3: This is what I have now, and it seems to work fine. But the code I am passing will sometimes be something like Remove-Object and the error is [System.IO.IOException], but at other times I actually need to return more than an error, like here where the code involves Get-Item. And that means defining the code block outside the function with a reference to the variable inside the function, which seems, fugly, to me. It may be that what I am trying to do is just more complicated than PowerShell is really designed to handle, but it seems MUCH more likely that there is a more elegant way to do what I am trying to do? Without being able to manipulate the script block from within the function I don't see any good options.
For what it is worth this last example shows a failure where the exception I am accepting for the repeat occurs and hits the limit, as well as an exception that just immediately fails because it is not the exception I am looping on, and an example where I return something. A fourth condition would be when I am trying to delete, and waiting on [System.IO.IOException] and a success would return nothing, no item, and no error log.
function Invoke-PxWaitForAccess {
param (
[System.Management.Automation.ScriptBlock]$code,
[String]$path,
[Type]$exceptionType
)
$invokeWaitForAccess = #{
item = $null
errorLog = [System.Collections.Generic.List[String]]::new()
}
$waitRetryCount = 2
:waitForSuccess do {
try {
& $code -path $path
$retry = $False
} catch {
if (($PSItem.Exception -is $exceptionType) -or ($PSItem.Exception.InnerException -and ($PSItem.Exception.InnerException -is $exceptionType))) {
if ($waitRetryCount -eq 0) {
$invokeWaitForAccess.errorLog.Add('E_Retry limit exceeded')
break waitForSuccess
} else {
$retry = $True
$waitRetryCount --
Start-Sleep -s:6
}
} else {
$invokeWaitForAccess.errorLog.Add("=_$($PSItem.Exception.GetType().FullName)")
$invokeWaitForAccess.errorLog.Add("=_$($PSItem.Exception.Message)")
if ($PSItem.Exception.InnerException) {
$invokeWaitForAccess.errorLog.Add("=_$($PSItem.Exception.InnerException.GetType().FullName)")
$invokeWaitForAccess.errorLog.Add("=_$($PSItem.Exception.InnerException.Message)")
}
$retry = $False
}
}
} while ($retry)
return $invokeWaitForAccess
}
CLS
$path = '\\noServer\folder\file.txt'
$code = {param ([String]$path) Get-Item $path -errorAction:Stop}
$invoke = (Invoke-PxWaitForAccess -code $code -path $path -exceptionType:([System.Management.Automation.ItemNotFoundException]))
if ($invoke.errorLog.count -eq 0) {
Write-Host "Good $path"
} else {
foreach ($line in $invoke.errorLog) {
Write-Host "$line"
}
}
Write-Host
$path = '\\noServer\folder\file.txt'
$code = {param ([String]$path) Get-Item $path -errorAction:Stop}
$invoke = (Invoke-PxWaitForAccess -code $code -path $path -exceptionType:([System.IO.IOException]))
if ($invoke.errorLog.count -eq 0) {
Write-Host "Good $path"
} else {
foreach ($line in $invoke.errorLog) {
Write-Host "$line"
}
}
Write-Host
$path = '\\Mac\iCloud Drive\Px Tools 3.#\# Dev 3.4.5\Definitions.xml'
$code = {param ([String]$path) $invokeWaitForAccess.item = Get-Item $path -errorAction:Stop}
$invoke = (Invoke-PxWaitForAccess -code $code -path $path -exceptionType:([System.Management.Automation.ItemNotFoundException]))
if ($invoke.errorLog.count -eq 0) {
Write-Host "Good $path !"
Write-Host "$($invoke.item)"
} else {
foreach ($line in $invoke.errorLog) {
Write-Host "$line"
}
}
Write-Host
I am working on a script logging solution for an install script that has many tasks and long processing times, and I am trying to address the issue of network dropout. I am also moving to a StreamWriter based approach vs my old Add-Content approach for performance reasons.
The problem I am having is that once the network drops out, the StreamWriter doesn't reconnect. So, first question is, CAN I reconnect, or is this a limitation of StreamWriter? The fact that the StreamWriter has a cache that can be Flushed makes me think I may be doing a bunch of work to recreate functionality that is already there.
And second, I am starting to think a simpler/better solution is simply to write the log to a local folder, so the log is always complete, then simply attempt to copy that to the network location for progress review. I had been thinking about implementing parallel logs, so a loss of the network would still leave the local copy complete. Curious if anyone else looks at this and says "Well, YEAH, dufus, obviously."
function Get-PxLogFile {
return $script:pxLogFile
}
function Set-PxLogFile {
param (
[string]$path
)
if (Test-Path $path) {
Remove-Item $path -force
}
[string]$script:pxLogFile= $path
}
function Get-PxLogWriter {
$logFile = Get-PxLogFile
if (-not $script:pxFileStream) {
$script:pxFileStream = New-Object io.fileStream $logFile, 'Append', 'Write', 'Read'
$script:pxlogWriter = New-Object io.streamWriter $script:pxFileStream
} elseif ($script:pxFileStream.name -ne $logFile) {
$script:pxFileStream = New-Object io.fileStream $logFile, 'Append', 'Write', 'Read'
$script:pxlogWriter = New-Object io.streamWriter $script:pxFileStream
}
return $script:pxlogWriter
}
function Dispose-PxLogFile {
$script:pxlogWriter.Dispose()
$script:pxFileStream.Dispose()
$script:pxFileStream = $null
$script:pxlogWriter = $null
}
# Shared
function Get-PxDeferredLog {
return ,$script:deferredLog # , keeps PS from unrolling a single item array into a string
}
function Set-PxDeferredLog {
param (
[collections.arrayList]$deferredLog
)
[collections.arrayList]$script:deferredLog = $deferredLog
}
function Get-PxDeferredLogTimes {
return ($script:deferredLogTimes -join ', ')
}
function Finalize-PxLogFile {
$logWriter = Get-PxLogWriter
if (($deferredLog = Get-PxDeferredLog).count -gt 0) {
$abandonTime = (Get-Date) + (New-TimeSpan -seconds:3)
$logWriter = Get-PxLogWriter
:lastChanceLogWindow do {
$deferredLog = Get-PxDeferredLog
:deferredItemsWrite do {
try {
$logWriter.WriteLine($deferredLog[0])
if ($deferredLog.count -gt 0) {
$deferredLog.RemoveAt(0)
} else {
break :lastChanceWindow
}
} catch {
break :deferredItemsWrite
}
} while ($deferredLog.count -gt 0)
Set-PxDeferredLog $deferredLog
if ($deferredLog.count -eq 0) {
break :lastChanceWindow
}
if ((Get-Date) -gt $abandonTime) {
break :lastChanceLogWindow
}
} while ((Get-Date) -lt $abandonTime)
if ($deferredLog) {
Write-Host "Failed to write all log entries"
}
}
$script:deferredLogItems = $script:deferredLogTimes = $null
}
function Add-PxLogFileContent {
param (
[string]$string
)
$logWriter = Get-PxLogWriter
# Nested Functions
function Add-PxDeferredLogItem {
param (
[string]$item
)
if (-not $script:deferredLog) {
[collections.arrayList]$script:deferredLog = New-Object collections.arrayList
}
if ($script:deferredLog.count -eq 0) {
Start-PxDeferredLogTime
}
[void]$script:deferredLog.Add($item)
}
function Start-PxDeferredLogTime {
if (-not $script:deferredLogTimes) {
[collections.arrayList]$script:deferredLogTimes = New-Object collections.arrayList
}
if ((-not $script:deferredLogTimes) -or (-not $script:deferredLogTimes[-1].EndsWith('-'))) {
[void]$script:deferredLogTimes.Add("$((Get-Date).ToString('T'))-")
}
}
function Stop-PxDeferredLogTime {
if ($script:deferredLogTimes[-1].EndsWith('-')) {
$script:deferredLogTimes[-1] = "$($script:deferredLogTimes[-1])$((Get-Date).ToString('T'))"
}
}
$deferredLogProcessed = $false
if ([collections.arrayList]$deferredLog = Get-PxDeferredLog) {
$deferredLogProcessed = $true
:deferredItemsWrite do {
try {
$logWriter.WriteLine($deferredLog[0])
$logWriter.Flush()
$deferredLog.RemoveAt(0)
} catch {
break :deferredItemsWrite
}
} while ($deferredLog.count -gt 0)
if ($deferredLog.count -eq 0) {
$deferredLogPending = $false
} else {
$deferredLogPending = $true
}
Set-PxDeferredLog $deferredLog
} else {
$deferredLogPending = $false
}
if (-not $deferredLogPending) {
try {
if ($logWriter.WriteLine($string)) {
$logWriter.Flush()
}
if ($deferredLogProcessed) {Stop-PxDeferredLogTime}
} catch {
Add-PxDeferredLogItem $string
Write-Host "Failed: $(Get-Date)`n$($_.Exception.Message)"
}
} else {
Add-PxDeferredLogItem $string
}
}
### MAIN
Clear-Host
$script:deferredLogItems = $script:deferredLogTimes = $null
$logPath = '\\px\Content'
Write-Host 'logTest1.txt'
$startTime = Get-Date
$endTime = $startTime + (New-TimeSpan -minutes:5)
#Set-PxLogFile "$([System.IO.Path]::GetFullPath($env:TEMP))\logTest1.txt"
Set-PxLogFile "$logPath\logTest1.txt"
do {
Add-PxLogFileContent "logged: $(Get-Date)"
Start-SLeep -s:10
} while ((Get-Date) -lt $endTime)
Finalize-PxLogFile
Write-Host 'logTest2.txt'
$startTime = Get-Date
$endTime = $startTime + (New-TimeSpan -minutes:5)
#Set-PxLogFile "$([System.IO.Path]::GetFullPath($env:TEMP))\logTest2.txt"
Set-PxLogFile "$logPath\logTest2.txt"
do {
Add-PxLogFileContent "logged: $(Get-Date)"
Start-SLeep -s:10
} while ((Get-Date) -lt $endTime)
if ([string]$deferredLogTimes = Get-PxDeferredLogTimes) {
Add-PxLogFileContent "Deferred logging time ranges: $deferredLogTimes"
}
Finalize-PxLogFile
Dispose-PxLogFile
I don't know much about PowerShell but have inherited a script from someone who is no longer available for assistance. This script imports AD Group Info and memberships related to Users and Computers. It works fine when run on a machine with PS 2.0 but it crashes if executed on PS 3.0 or newer.
I have not been able to figure out what needs to be modified but it seems the errors start occurring in the "Computer" membership import step and there are hundreds of errors that all say:
Command failed while processing computers: , Exception of type 'System.OutOfMemoryException' was thrown
Then at some point it looks like the script just stops and it never even gets to the 3rd step / function.
Any advice?
[Reflection.Assembly]::LoadWithPartialName("System.DirectoryServices") | Out-Null
$DBServer = "DBSERVER"
$DBName = "DBNAME"
$TableUsers = "[$DBName].[dbo].[AD_GroupToClient]"
$TableComps = "[$DBName].[dbo].[AD_GroupToDevice]"
$TableGroups = "[$DBName].[dbo].[AD_Group_Info]"
$sqldateformat = "yyyy/MM/dd HH:mm:ss:fff"
[system.Data.SqlClient.SqlConnection]$global:SqlConnection = $null
function Get-ScriptPath { $Invocation = (Get-Variable MyInvocation -Scope 1).Value; Split-Path $Invocation.MyCommand.Path }
$ScriptPath = Get-ScriptPath
$Logfile = "$ScriptPath\OutLog.log"
function Write-Logfile {
param($logtext)
[string](Get-Date -format $sqldateformat) + "`t$logtext" | Out-File $Logfile -Encoding ascii -Append
}
function Open-Database {
$global:SqlConnection = New-Object system.Data.SqlClient.SqlConnection
try {
$global:SqlConnection.ConnectionString = "Server=$DBServer;Database=$DBName;Integrated Security=True"
$global:SqlConnection.Open() | Out-Null
Write-Logfile "OK`tDatabase opened"
} catch {
Write-Host "Error Opening SQL Database`t$($_.Exception.Message)"
Write-Logfile "Error`tDatabase open failed, $($_.exception.message)"
exit
}
}
function Close-Database {
$global:SqlConnection.Close()
Write-Logfile "OK`tDatabase closed"
}
function Esc-Quote {
param($str)
if ($str) { $str.Replace("'","''") }
}
function Run-DBCommand {
param($SqlCommands, [switch]$getnumrows)
if ($SqlCommands.Count -ge 1) {
$SqlCommandText = [string]::Join(";", $SqlCommands)
try {
$SqlCmd = New-Object Data.SqlClient.SqlCommand($SqlCommandText, $SqlConnection)
$returnvalue = $SqlCmd.ExecuteNonQuery()
if ($getnumrows) { return $returnvalue }
} catch {
Write-Logfile "Error`tSQL Command failed, $($_.exception.message)"
}
}
}
function Run-GroupMemberExport {
param($exportmode)
switch ($exportmode) {
"users" {
$dom = [ADSI]"LDAP://OU=Clients123,DC=test1,DC=test2,DC=test3"
$query = "(&(objectClass=user)(objectCategory=person)(samaccountname=*))"
$table = $TableUsers
$namecolumn = "AD_Group_Member_Name"
$attribs = #("samaccountname")
}
"computers" {
$dom = [ADSI]"LDAP://DC=test1,DC=test2,DC=test3"
$query = "(&(objectClass=computer)(samaccountname=*))"
$table = $TableComps
$namecolumn = "AD_Group_Member_Device"
$attribs = #("samaccountname", "whencreated")
}
}
$starttime = (Get-Date).ToUniversalTime().ToString($sqldateformat)
$srch = New-Object DirectoryServices.DirectorySearcher($dom, $query, $attribs)
$srch.PageSize = 1000
$srch.Sort = New-Object DirectoryServices.SortOption("sAMAccountName", [DirectoryServices.SortDirection]::Ascending)
$results = $srch.FindAll()
$count = 0
$numaccounts = $results.Count
foreach ($res in $results) {
try {
$objAccount = $res.GetDirectoryEntry()
$samaccountname = $objAccount.properties["samaccountname"][0]
$whencreated = ""
if ($exportmode -eq "computers") { $whencreated = Get-Date ([datetime]$objAccount.properties["whencreated"][0]) -Format $sqldateformat }
$count++
Write-Progress "Querying accounts" $samaccountname -PercentComplete ($count * 100.0 / $numaccounts)
$objAccount.psbase.RefreshCache("tokenGroups")
$SIDs = $objAccount.psbase.Properties.Item("tokenGroups")
$groups = #()
ForEach ($Value In $SIDs) {
$SID = New-Object System.Security.Principal.SecurityIdentifier $Value, 0
try {
$Group = $SID.Translate([System.Security.Principal.NTAccount]).Value
} catch {
$Group = $SID.Translate([System.Security.Principal.SecurityIdentifier]).Value
}
if ($groups -notcontains $Group -and $Group.Split("\")[1] -ne $samaccountname) { $groups += $Group }
}
Run-DBCommand #("DELETE FROM $table WHERE [$namecolumn] = '$(Esc-Quote $samaccountname)'")
$sqlcommands = #()
$currenttime = (Get-Date).ToUniversalTime().ToString($sqldateformat)
if ($groups) {
$groups | sort | foreach {
if ($exportmode -eq "users") {
$sqlcommands += "INSERT INTO $table ([$namecolumn], [AD_Group_Name], [Last_Update]) VALUES ('$(Esc-Quote $samaccountname)', '$(Esc-Quote $_)', '$currenttime')"
} else {
$sqlcommands += "INSERT INTO $table ([$namecolumn], [AD_Group_Name], [Last_Update], [Record_Created]) VALUES ('$(Esc-Quote $samaccountname)', '$(Esc-Quote $_)', '$currenttime', '$whencreated')"
}
if ($sqlcommands.count -ge 50) { Run-DBCommand $sqlcommands; $sqlcommands = #() }
}
} else {
if ($exportmode -eq "users") {
$sqlcommands += "INSERT INTO $table ([$namecolumn], [AD_Group_Name], [Last_Update]) VALUES ('$(Esc-Quote $samaccountname)', 'ERROR: Unable to retrieve groups', '$currenttime')"
} else {
$sqlcommands += "INSERT INTO $table ([$namecolumn], [AD_Group_Name], [Last_Update], [Record_Created]) VALUES ('$(Esc-Quote $samaccountname)', 'ERROR: Unable to retrieve groups', '$currenttime', '$whencreated')"
}
}
Run-DBCommand $sqlcommands
} catch {
Write-Logfile "Error`tCommand failed while processing $exportmode`: $($objAccount.name), $($_.exception.message)"
}
}
Write-Progress " " " " -Completed
if ($count -eq $numaccounts) {
$numdeleted = Run-DBCommand #("DELETE FROM $table WHERE [Last_Update] < '$starttime' OR [Last_Update] IS NULL") -getnumrows
Write-Logfile "OK`tUpdates for $exportmode completed, $numdeleted old records deleted."
}
}
function Run-GroupDescriptionExport {
$dom = [ADSI]"LDAP://DC=test1,DC=test2,DC=test3"
$query = "(&(objectClass=group)(samaccountname=*))"
$table = $TableGroups
$attribs = #("samaccountname", "displayname", "description", "whencreated", "managedby", "grouptype","distinguishedname","whenchanged")
$srch = New-Object DirectoryServices.DirectorySearcher($dom, $query, $attribs)
$srch.PageSize = 1000
$srch.Sort = New-Object DirectoryServices.SortOption("sAMAccountName", [DirectoryServices.SortDirection]::Ascending)
$results = $srch.FindAll()
$count = 0
$numgroups = $results.Count
$sqlcommands = #()
$starttime = [datetime]::Now.ToUniversalTime().ToString($sqldateformat)
foreach ($res in $results) {
$count++
$samaccountname = $res.properties["samaccountname"][0]
Write-Progress "Querying accounts, $count/$numgroups" $samaccountname -PercentComplete ($count * 100.0 / $numgroups)
$displayName = ""; if ($res.properties.contains("displayname")) { $displayName = $res.properties["displayname"][0] }
$description = ""; if ($res.properties.contains("description")) { $description = $res.properties["description"][0] }
$managedby = ""; if ($res.properties.contains("managedby")) { $managedby = $res.properties["managedby"][0] }
$grouptype = ""; if ($res.properties.contains("grouptype")) { $grouptype = $res.properties["grouptype"][0] }
$distinguishedname = ""; if ($res.properties.contains("distinguishedname")) { $distinguishedname = $res.properties["distinguishedname"][0] }
$whencreated = ""; if ($res.properties.contains("whencreated")) { $whencreated = ([datetime]$res.properties["whencreated"][0]).ToString($sqldateformat) }
$whenchanged = ""; if ($res.properties.contains("whenchanged")) { $whenchanged = ([datetime]$res.properties["whenchanged"][0]).ToString($sqldateformat) }
$lastupdated = [datetime]::Now.ToUniversalTime().ToString($sqldateformat)
$sqlcommand = "DELETE FROM $table WHERE [AD_Group_Name] = '$(Esc-Quote $samaccountname)'; "
$sqlcommand += "INSERT INTO $table ([AD_Group_Name], [AD_Group_DisplayName], [AD_Group_Description], [Last_Update], [Managed_By],[Distinguished_Name],[Group_Category],[Created_On], AD_Last_Modified]) VALUES ('$(Esc-Quote $samaccountname)', '$(Esc-Quote $displayName)', '$(Esc-Quote $description)', '$lastupdated', '$(Esc-Quote $managedby)', '$(Esc-Quote $distinguishedname)', '$grouptype', '$whencreated','$whenchanged')"
$sqlcommands += $sqlcommand
if ($sqlcommands.count -ge 100) { Run-DBCommand $sqlcommands; $sqlcommands = #()
}
}
Run-DBCommand $sqlcommands
if ($numgroups -eq $count) {
Run-DBCommand #("DELETE FROM $table WHERE [Last_Update] <= '$starttime'")
}
Write-Progress " " " " -Completed
}
Open-Database
Run-GroupMemberExport "users"
Run-GroupMemberExport "computers"
Run-GroupDescriptionExport
Close-Database
This doesn't have anything to do with the PowerShell version. You're just plain running out of memory. You're pulling in a lot of data, so you need to be more conscious of getting rid of that data when you're done with it.
There are a couple things you can do to clean up memory:
First, the documentation for DirectorySearcher.FindAll() says:
Due to implementation restrictions, the SearchResultCollection class cannot release all of its unmanaged resources when it is garbage collected. To prevent a memory leak, you must call the Dispose method when the SearchResultCollection object is no longer needed.
So whenever you do:
$results = $srch.FindAll()
Make sure you call $results.Dispose() when you're done with it (at the end of the function).
Second, when you loop through the results in your Run-GroupMemberExport function, you're calling $res.GetDirectoryEntry(). Usually you can just let the garbage collector clean up DirectoryEntry objects, but when you're creating so many in a loop like that, the GC doesn't have time to run. This has happened to me when I've run a loop over thousands of accounts.
To solve this, you can call Dispose() on the DirectoryEntry objects yourself. Since you already have a try/catch block there, I would suggest adding a finally block to make sure it happens even if an error is thrown:
try {
...
} catch {
Write-Logfile "Error`tCommand failed while processing $exportmode`: $($objAccount.name), $($_.exception.message)"
} finally {
$objAccount.Dispose()
}
Actually, you could probably just not use GetDirectoryEntry() at all. Just ask the DirectorySearcher to return the other attributes you need. But if you want to still use it, then make sure you call RefreshCache for every attribute you need (you can put them all in one call to RefreshCache). If you access the Properties collection and ask for a value that it does not already have in cache, then it will ask AD for every attribute with a value - that's a lot of unnecessary data.
Last week I ran into a strange issue.
I modified code that should test if a file is locked and which I once found on the Internet (essentially similar to https://superuser.com/questions/876288/how-do-i-detect-and-skip-locked-files-in-a-powershell-script) to
1) deal with timeout and
2) to handle non existing files (and directories) correctly.
So here is my code:
function Test-IsFileLocked {
[CmdletBinding()]
Param (
[Parameter(Mandatory = $True, Position = 0)]
[ValidateNotNullOrEmpty()]
[System.IO.FileInfo] $File
, [ref] $MSecsPassed
)
try {
$Stream = $File.Open([System.IO.FileMode]"Open", [System.IO.FileAccess]"Read", [System.IO.FileShare]"None")
}
# The follwing catch block seems to cause the problem that the 'IO.FileNotFoundException' gets caught after some time (although it is still a simple 'IO.IOException.)
# Outcomment it by removing the '#' in front of '<# ...' to see how all works nicely...
#<#
catch [System.IO.DirectoryNotFoundException] {
Write-Host "System.IO.DirectoryNotFoundException: '$($_.Exception)'`nType = '$($_.Exception.GetType().FullName)'`nType = '$($_.GetType().FullName)'" -ForegroundColor:Yellow
return $False
}
#>
catch [System.IO.FileNotFoundException] {
Write-Host "Caught a 'FileNotFoundException' exception, although the exception still is of type of a simple 'System.IO.IOException'! MSecs passed: $($MSecsPassed.Value)" -ForegroundColor:Red
Write-Host "FileNotFoundException: '$($_.Exception)'`nType = '$($_.Exception.GetType().FullName)'`nType = '$($_.GetType().FullName)'" -ForegroundColor:Yellow
#if ($_.Exception.GetType().FullName -eq 'System.IO.IOException') {
# return $True
#} else {
return $False
#}
} catch [System.IO.IOException] {
#Write-Host "System.IO.IOException: '$($_.Exception)'`nType = '$($_.Exception.GetType().FullName)'`nType = '$($_.GetType().FullName)'"
if ($_.Exception.GetType().FullName -eq 'System.IO.IOException') {
return $True
} else {
return $False
}
} catch {
Write-Host "Any Exception: '$($_.Exception)'`nType = '$($_.Exception.GetType().FullName)'`nType = '$($_.GetType().FullName)'" -ForegroundColor:Yellow
if ($_.Exception.GetType().FullName -eq 'System.IO.IOException') {
return $True
} else {
return $False
}
} finally {
if ($Stream) {
$Stream.Dispose()
}
}
return $False
}
function Wait-UntilFileIsAccessible {
[CmdletBinding()]
Param (
[Parameter(Mandatory = $True, Position = 0)]
[ValidateNotNullOrEmpty()]
[System.IO.FileInfo] $File
, [Parameter(Position = 1)]
[uint32] $CheckIntervalInMSecs
, [Parameter(Position = 2)]
[uint32] $Timeout
)
if (!$CheckIntervalInMSecs) {
$CheckIntervalInMSecs = 500
}
[double] $SecondsPassed = 0.0
[double] $SecondsAdder = [double]$CheckIntervalInMSecs / 1000.0
[uint32] $MSecsPassed = 0
#Write-Verbose "Waiting for '$($File.Fullname)' to get unlocked!"
while (Test-IsFileLocked -File:$File -MSecsPassed:([ref]$MSecsPassed)) {
Start-Sleep -MilliSeconds:$CheckIntervalInMSecs
$MSecsPassed += $CheckIntervalInMSecs
if ($Timeout) {
$SecondsPassed += $SecondsAdder
if (([uint32]$SecondsPassed) -ge $Timeout) {
return $False
}
}
}
#Write-Vebose "'$($File.Fullname)' now isn't locked anymore"
return $True
}
If I call this code with
Wait-UntilFileIsAccessible -File:'C:\LockedByMSWord.txt' -Timeout:30
where 'C:\LockedByMSWord.txt' is an existing file actually locked (e.g. opened with MS Word) then after some time (16500 ms in most cases for me) the 'IO.FileNotFoundException' gets caught although the exception thrown seems to be of type 'IO.IOException'.
If I then repeat the call to 'Wait-UntilFileIsAccessible' then usually Powershell immediately catches this 'wrong' exception.
It drove me nuts and I tried different things until I removed the code block that catches the 'IO.DirectoryNotFoundException' (for testing purposes initially) just to find out that then everything works as expected.
Am I doing something wrong here (I mean with the code that catches this 'IO.DirectoryNotFoundException' exception), do I misunderstand something or could this be a Powershell bug?
This is with Powershell 4 and ErrorActionPreference set to 'Continue'.
Also some code obviously is just in there for testing purposes (I wouldn't need ref parameter MSecsPassed in Test-IsFileLocked and output to host normally)
P.S.: I know how to workaround it, but I would like feedback to my question if I am doing something wrong in my code or if this possibly could be a bug in Powershell.
Thanks,
Patrik
I have the following code that I'm trying to return the string Good as an out parameter from FooFunc.
How can I do that?
function FooFunc($a, [ref]$result){
if (4 -gt 1) {
$result = "Good"
return $true
} else {
$result = "Bad"
return $false
}
}
try
{
FooFunc "Bar" ([ref]$result)
Write-Host $result
}
catch
{
Write-Host $_.Exception.Message
}
EDIT I don't want to return the answer with return, I need it as ref or out parameters
You can try
Function Foo{
if(4 -gt 1)
{
return "Good"
}
else
{
return "Bad"
}
}
try{
$result = Foo
Write-Output $result
}
catch{
Write-Output "Some message"
}
[ref] is for passing variable into a function so it needs to exist in your current scope which is why you are getting the error. I think you are misunderstanding the usage. Try this:
function FooFunc([ref]$result){
if (4 -gt 1) {
$result = 'good' + $result.value
$result
} else {
$result = "Bad"
return $false
}
}
try
{
$result = 'badval'
FooFunc ([ref]$result)
}
catch
{
Write-Host $_.Exception.Message
}
If you pipe the function to get-member you will see it returns ref and the output is using the variable you are passing in.
Applying [ref] to a parameter will result in a PSReference object being passed to the function.
In the case of function FooFunc($a, [ref]$result), using $result = "Good" will change the local $result variable in the function from a PSReference to a string, but will have no effect on the variable the caller passed in. To modify the outer variable, you would want to set $result.Value = "Good".
And as Noah Sparks mentioned, on the caller side you also need to declare the variable before using it as argument.
function FooFunc( $a, [ref]$result ) {
if( 4 -gt 1 ) {
$result.Value = 'Good' # change the reference value, not the variable
return $true
}
else {
$result.Value = 'Bad'
return $false
}
}
try {
$result = $null # declare variable before using it for a reference
FooFunc 'Bar' ([ref]$result)
Write-Host $result
}
catch {
Write-Host $_.Exception.Message
}
Another option would be to output multiple values from your function:
function FooFunc( $a ) {
if( 4 -gt 1 ) {
# Output both result and success values
'Good'
return $true
}
else {
# This single return is the same as the output and return above
return 'Bad', $false
}
}
try {
$result,$success = FooFunc 'Bar' # Receive both values
$success # Implicitly output success value, as before
Write-Host $result
}
catch {
Write-Host $_.Exception.Message
}