Prevent modification of original array in a loop (foreach) - powershell - powershell

I need some assistance please, to understand behavior of arrays and powershell loops.
I have stumbled upon a strange array behavior, here is the issue and sample script:
# I create simple array
$item2 = New-Object Collections.ArrayList
# getting some sample data for array
$allitems = Get-ChildItem -Path "c:\" | select name,Attributes
#running loop
foreach ($item in $allitems){
# outputing iriginal array item
Write-Host $item
# addind to new array
$item2.Add($item)| Out-Null
# here is the issue I have - after I add member to my $item2 array
# it is replicated to also $allitems, I don't need this behavior
# to happen. Also I don't understand why it changes the original
# $item and $allitems. Am I creating array copy incorrectly ?
$item2[-1] | Add-Member -MemberType NoteProperty -name 'testproperty' -value 'testvalue'
write-host "$($item2[-1]) mod"
Write-Host $item
}
result of the script is this :
#{Name=rollback2.puc; Attributes=Archive} -> original entry
#{Name=rollback2.puc; Attributes=Archive; testproperty=testvalue} mod -> modified entry
#{Name=rollback2.puc; Attributes=Archive; testproperty=testvalue} -> original entry modified , why ?
Any help appreciated.
Thanks

Most types of objects in .NET are reference-type objects, meaning that what you have stored in $item is not the object itself, but a reference to it.
When you then call $item2.Add($item), the reference is copied to $item2, but both the array list and the variable still point back to the exact same object/memory address, and PowerShell will therefore "remember" the added property regardless of which copy of the object reference you use.
In this particular case, you can work around it and create 2 sets of objects by calling Get-ChildItem twice:
$item2 = New-Object Collections.ArrayList
$allitems = Get-ChildItem -Path "c:\" | select name,Attributes
foreach ($item in Get-ChildItem -Path "c:\" | select name,Attributes){
Write-Host $item
$item2.Add($item)| Out-Null
$item2[-1] | Add-Member -MemberType NoteProperty -name 'testproperty' -value 'testvalue'
# the objects stored in $allitems will have been unaffected by the call above
}
Or by calling Get-Item to create a new object to represent the file just before adding to the array list:
$item2 = New-Object Collections.ArrayList
$allitems = Get-ChildItem -Path "c:\" | select name,Attributes
#running loop
foreach ($item in $allitems){
Write-Host $item
# Create a new FileInfo object by calling `Get-Item`
$itemCopy = $item | Get-Item
$item2.Add($itemCopy) | Out-Null
$item2[-1] | Add-Member -MemberType NoteProperty -name 'testproperty' -value 'testvalue'
# $item2[-1] has been modified, but $item has not
}

Related

Create Table showing Foldername and Count of files

I'm trying to create a script that, based on a list of directories, produces a list or table showing 'Folder Name' and 'Count' of files contained within each folder.
I have the following so far:
$folders = #("c:\temp", "C:\temp\tosh")
foreach ($folder in $folders) {
$folderFiles = Get-ChildItem -Path $folder -File
$obj = New-Object psobject -Property #{
Folder = $folder
Count = $folderFiles.count
}
$obj | Format-Table
}
This works but produces effectively two tables, one for each folder. I need this as one table but when I try to add another row to the object I get an error that PSObject does not contain a method named 'op_Addition'. I'm sure that I've had a similar result before by adding the $obj object to an array at each iteration similar to this:
$results += $obj
But can't seem to get it working at present. Any advice appreciated, thanks.
You need to define $results first as type array and then you can add values to the array. If you don't do so, then $results will be System.Object BaseType and hence the error. Try this.
$folders = #("c:\temp", "C:\temp\tosh")
$results = #()
foreach ($folder in $folders) {
$folderFiles = Get-ChildItem -Path $folder -File
$obj = New-Object psobject -Property #{
Folder = $folder
Count = $folderFiles.count
}
$results += $obj
}
$results
Vivek's answer will get you what you're after.
You can simply this further, avoiding the need to initialise an empty array and to use a temporary object. This uses the pscsutomobject type accelerator.
$folders = #("c:\temp", "C:\temp\tosh")
foreach ($folder in $folders) {
[array]$results += [pscustomobject]#{
Folder = $folder
Count = (Get-ChildItem -Path $folder -File).count
}
}
$results
I see that you have$obj | Format-Table in your loop. You could have $results | Format-Table at the end instead.
ls \temp,\temp\tosh\ | group psparentpath | select count,name

Remove Custom Powershell Object from RAM after adding it to ArrayList?

I've been fiddling around with a script (written by Vidrine, sourced from http://jfrmilner.rdpress.com/) that catalogs the NTFS permissions of a file server. At the moment it looks through the entire server share and collects permissions only from folders and ignores files. It stores the data in an object and writes it to an array list that is later dumped into a CSV. For reference, the file server is fairly large and has about 7.4TB of data in it. Initially I called an array and wrote to it using the += operator, but the way .net handles arrays made running it on medium sized sub directories a full day event that was extremely memory heavy.
The issue I've run into is that the script is completely memory hungry, and I can't tell if this is simply because of how the script is written, or if it's holding onto objects after writing them to the arraylist. For example, when I run the script against a specific sub directory, rather than the entire server, the script will use roughly 1.4GB of RAM and produces a CSV that's 148MB (100,000 rows x 3 columns). The machine I'm running this on has 16GB of RAM and it still manages to cap out while running against the entire directory.
I've toyed with [void] trying to eliminate $objResults from memory after it's written to $arrResults, but I don't seem to understand how a void cast works. I also attempted to use the [System.GC]::Collect() function, but it just slows down the loop and the accumulation of memory, but doesn't help (this also seems to be a hated practice)
I've tried running it in the Powershell V5 (also V1, V2 and V4) ISE and straight from command line.
Is $objResults staying in memory after being added to $arrResults and is this fixable? Or is the arraylist actually getting up to 15GB in size?
I'm pretty new to Powershell FYI.
Import-module ActiveDirectory
$targetServer = 'S:\shared data\' #Enter hostname
$targetDirectory = '' #Enter directory name
$target = Join-Path -ChildPath $targetDirectory -Path $targetServer
$arrResults = New-ObjectSystem.Collections.Generic.List[System.Object]
$exportPath = 'C:\temp\NTFS-OUTPUT.csv' #Enter name of the CSV output file
#Query target directory for all 'folders' (excludes files via Where statement)
Get-ChildItem -Recurse -Path $target | Where { $_.PSIsContainer} |
forEach {
$objPath = $_.FullName
$coLACL = Get-Acl -Path $objPath
forEach ( $objACL in $colACL ) {
forEach ( $accessRight in $objACL.Access ) {
$objResults = New-Object –TypeName PSObject
$objResults | Add-Member –MemberType NoteProperty –Name DirectoryPath –Value $objPath
$objResults | Add-Member –MemberType NoteProperty –Name Identity –Value $accessRight.IdentityReference
$objResults | Add-Member –MemberType NoteProperty –Name SystemRights –Value $accessRight.FileSystemRights
#$objResults | Add-Member –MemberType NoteProperty –Name SystemRightsType –Value $accessRight.AccessControlType
#$objResults | Add-Member -MemberType NoteProperty -Name IsInherited -Value $accessRight.IsInherited
#$objResults | Add-Member -MemberType NoteProperty -Name InheritanceFlags -Value $accessRight.InheritanceFlags
#$objResults | Add-Member –MemberType NoteProperty –Name RulesProtected –Value $objACL.AreAccessRulesProtected
$arrResults.Add($objResults)
}
}
}
$arrResults | Export-CSV -NoTypeInformation -Path $exportPath
No need for the intermediate array: simply output the value and extend the pipeline to Export-Csv.
Add-Member is very slow, especially when called many times via pipeline.
In PowerShell 3.0 and newer you can use a [PSCustomObject] accelerator for a literal hashtable.
Get-ChildItem -Recurse -Path $target -Directory | ForEach {
foreach ($objACL in (Get-Acl -Path $_.FullName)) {
foreach ($accessRight in $objACL.Access ) {
[PSCustomObject]#{
DirectoryPath = $_.FullName
Identity = $accessRight.IdentityReference
SystemRights = $accessRight.FileSystemRights
}
}
}
} | Export-CSV -NoTypeInformation -Path $exportPath

Append Member/Column to PSObject

I'm looking for a way to add an extra member to an object in PowerShell without creating a new object and looping.
Typically when I run a script it will be again a list of servers and I want to append some extra information against the data I'm returning but I can't find a way to do it without creating a new PSObject and looping through the existing object and adding the extra member row by row (simplified example below)
$FileList = get-childitem | Select-Object Name, DirectoryName
$FileListOBJ = #()
foreach ($item in $FileList)
{
$Temp = New-Object PSObject
$Temp | Add-Member NoteProperty ServerName "XServerName"
$Temp | Add-Member NoteProperty FileName $item.Name
$Temp | Add-Member NoteProperty Directory $item.DirectoryName
$FileListOBJ += $Temp
}
$FileListOBJ
Is there a way to do it along these lines ...
get-childitem | Select-Object "ServerName", Name, DirectoryName
The above code creates the extra Member but I haven't been able to find a way to fill the additional member with the details I'm after.
you can also create a new object without the add-member calls:
$FileList = get-childitem | Select-Object Name, DirectoryName
$FileListOBJ = #()
foreach ($item in $FileList)
{
$FileListOBJ += [PSCustomObject]#{ServerName="ServerName";FileName=$item.Name;Directory=$item.DirectoryName}
}
$FileListOBJ
Sure, you can use a calculated property (more about them here):
get-childitem | Select-Object #{l="ServerName"; e={"XServerName"}}, Name, DirectoryName

Powershell. Need help importing hostnames from .csv. The rest of this works

I am trying to figure out how to correct this script I've wrote. I know it is something wrong with the way it is importing the list of hostnames. I don't know how to fix it.
Part 1: This is supposed to import a .csv with the hostnames and dig the registry for the application's uninstall information, put it into an array, and export into .csv's for later use. Also it creates .txt files in order to later compare the applications on the system to a baseline.
$path = "\\path"
$computers = Import-Csv -Path "\\Path\hostnames.csv"
$array = #()
foreach($pc in $computers)
{
$computername = $pc.computername
#$computername = "KNOWN_HOSTNAME" #test line for one system
$UninstallKey = "SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall"
$reg = [microsoft.win32.registrykey]::OpenRemoteBaseKey('LocalMachine',$computername)
$regkey = $reg.OpenSubKey($UninstallKey)
$subkeys = $regkey.GetSubKeyNames()
foreach($key in $subkeys)
{
$thisKey=$UninstallKey+"\\"+$key
$thisSubKey=$reg.OpenSubKey($thisKey)
$obj = New-Object PSObject
$obj | Add-Member -MemberType NoteProperty -Name "ComputerName" -Value $computername
$obj | Add-Member -MemberType NoteProperty -Name "DisplayName" -Value $($thisSubKey.GetValue("DisplayName"))
$obj | Add-Member -MemberType NoteProperty -Name "DisplayVersion" -Value $($thisSubKey.GetValue("DisplayVersion"))
$obj | Add-Member -MemberType NoteProperty -Name "Publisher" -Value $($thisSubKey.GetValue("Publisher"))
$obj | Add-Member -MemberType NoteProperty -Name "InstallDate" -Value $($thisSubKey.GetValue("InstallDate"))
$array += $obj
$ExportArray = $array | Where-Object { $_.DisplayName } |
select ComputerName , DisplayName, DisplayVersion, Publisher, InstallDate
$ExportArray |
Export-csv $path\$computername.csv -NoTypeInformation
$ExportArray2 = $array |
Where-Object { $_.DisplayName } |
select DisplayName, DisplayVersion, Publisher
$ExportArray2 |
Export-csv $path\$computername.txt -NoTypeInformation
}
}
Part 2: This portion compiles the .csv's into one excel document for reporting
$csvs = Get-ChildItem $path\* -Include *.csv
$outputfilename = "Network_" + (Get-Date -Format yyyyMMdd)
$excelapp = new-object -comobject Excel.Application
$excelapp.sheetsInNewWorkbook = $csvs.Count
$xlsx = $excelapp.Workbooks.Add()
$sheet=1
foreach ($csv in $csvs)
{
$row=1
$column=1
$worksheet = $xlsx.Worksheets.Item($sheet)
$worksheet.Name = $csv.Name
$file = (Get-Content $csv.PSPath | ForEach-Object {$_ -replace '"', ""})
foreach($line in $file)
{
$linecontents = $line -split ‘,(?!\s*\w+”)’
foreach($cell in $linecontents)
{
$worksheet.Cells.Item($row,$column) = $cell
$column++
}
$column = 1
$row++
}
$sheet++
}
$output = $path + “\” + $outputfilename + ".xlsx"
$xlsx.SaveAs($output)
$excelapp.quit()
Part 3: This portion loads up a baseline, and the .txt's created preciously, and checks for differences in the files. (also deletes blank ouput files)
$bline = Get-ChildItem $path\* -Include Baseline.txt
$txts = Get-ChildItem $path\* -Include *.txt -Exclude Baseline.txt
foreach ($txt in $txts)
{
Compare-Object -referenceobject $(Get-Content $bline) -differenceobject $(Get-Content $txt) |
ft inputobject, #{n = "file"; e = {if ($_.SideIndicator -eq '=>') {"System"} else {"Baseline"}}} |
Out-File $txt'_has_diff'.csv -Width 256
Get-ChildItem $path |
where {$_.Length -eq 0} |
Remove-Item
}
Thank you
Edit:
The Hostnames.csv files I've tried are:
HOSTNAME1
HOSTNAME2
and
"HOSTNAME1","HOSTNAME2"
It's a little unclear what the problem is, because you say there is "something wrong with the way it is importing the list of hostnames", but you haven't specified what kind of results you're getting and how they differ from the intended results.
However, based on your sample data I think I can infer what the problem is: You're trying to use Import-Csv on non-CSV data. Neither of your examples looks like a CSV file. They both look like lists. A list in which the items are separated by commas, such as
"HOSTNAME1","HOSTNAME2","HOSTNAME3","HOSTNAME4"
is not called a "CSV file". CSV files are a form of "flat file", in which the data represents the rows and columns of a single database table. An example of a CSV file would be something like this, where the first line is a list of field (column) names, and the other lines are records (rows) with the comma-separated values corresponding to the columns in the header row:
"Hostname","OS","OS Version","Primary Function","Location"
"BOSEXCH01","Windows","Server 2012","Microsoft Exchange","Boston"
"BOSDC01","Windows","Server 2008 R2","Active Directory domain controller","Boston"
"MYWEB","Linux","Ubuntu 13.04","Apache web server","Phoenix"
The cmdlet Import-Csv imports a CSV file into an array of objects in which the properties are the field names in the header row, and the values are the comma-separated items in each row corresponding to the property names derived from the header row. Export-Csv does the reverse—it creates a CSV file from an array of objects.
It looks like what you're trying to do is read a simple list of hostnames into an array of strings. If your data looks like the first example,
HOSTNAME1
HOSTNAME2
[etc...]
you can read it into an array by simply using Get-Content, as follows (note that I changed the extension to .txt to reflect the actual format of the data):
$computers = Get-Content "\\Path\hostnames.txt"
If your data looks like the second example,
"HOSTNAME1","HOSTNAME2",[etc...]
you can read it into array like this:
$computers = (Get-Content "\\Path\hostnames.txt") -split ','
On the other hand, it appears that you are using Export-Csv correctly: You're exporting a bunch of objects with the same properties into a flat file, which is the correct usage of the term "CSV".

powershell convert array of hastables into csv

Have done googling but have not found suitable answer
I have
$a = #(
#{"name"="a"; "value"="1"},
#{"name"="b"; "value"="2"}
)
Foreach($x in $a){
$obj = New-object psobject
Foreach($key in $x.keys) {
$obj ¦ add-member -membertype NoteProperty -name $key -value $x[$key]
}
$obj ¦ export-cvs test.csv
}
I get only last line in cab. I want all the lines
It's truth that Export-Csv won't append (< v3), but I do not think ConvertTo-Csv will solve the problem... In the end you will get file with headers repeated for every object in your array. I would suggest one of two things:
move Export-Csv outside foreach {} (need to put it in subexpression though, because it won't be possible to pipe it by default)
use Foreach-Object instead and get same results
I would also suggest to use New-Object PSObject -Property parameter instead of adding members later: because you use hashtable as a base you are half way there, see attached code samples:
# Using $( foreach ($x in $a) {} ) | Export-Csv
$(Foreach($x in $a){
New-object psobject -Property $x
}) | Export-Csv test.csv
# Using $a | Foreach-Object {} | Export-Csv
$a | ForEach-Object {
New-Object PSObject -Property $_
} | Export-Csv test.csv
You're overwriting the csv file each time. Try using convertTo-CSV each time (instead of export-csv) and appending that to a file.
2022 update
You always need to use the -append parameter if you are iterating through a collection
using powershell 7.2 and later you don`t need to parse the hashtable again, it matches the repeating properties and writes to csv.
so this will work automatically
$hashtable=#()
$hashtable += $pscustomobject
$hashtable | export-csv