This question already has answers here:
How can I "zip" two arrays in PowerShell?
(2 answers)
Perform math operations with two equal length arrays
(2 answers)
Powershell foreach regarding multiple collections
(1 answer)
Is there a PowerShell equivalent of `paste` (i.e., horizontal file concatenation)? [duplicate]
(1 answer)
CMD or Powershell command to combine (merge) corresponding lines from two files [duplicate]
(6 answers)
Closed 6 months ago.
still very inexperienced in Powershell, but as is well known, every journey begins with the first steps.
I define two arrays in the script:
$array1 = #("server1", "server2")
$array2 = #("SID1", "SID2")
The SID1 and server1 belong together.
I then want to combine the two arrays using a loop:
Example:
foreach ($i in $array1) {
Write-Host "Server = ${i}"
}
How can I combine the two arrays?
Ideal is:
...
Write-Host "Server=${i}" and SID=${?}"
...
Can the two arrays be built into the foreach so that both values are filled when executing on the write host?
Thanks for your ideas and help.
Many greetings
Carlos
An alternative approach is to use [System.Linq.Enumerable]::Zip which provides convenient API(like python zip function). ie, You could do
$array1 = #("server1", "server2")
$array2 = #("SID1", "SID2")
$doSomethingWithPair = {
param($a, $b)
Write-Output "I have $a and $b"
}
[System.Linq.Enumerable]::Zip(
$array1,
$array2,
[Func[Object, Object, Object]]$doSomethingWithPair
)
This will print
I have server1 and SID1
I have server2 and SID2
Use a for loop to generate a range of valid indices for each array:
for($i = 0; $i -lt $array1.Count; $i++){
Write-Host "Server named '$($array1[$i])' has SID '$($array2[$i])'"
}
But a better solution would be to create a single array of objects that have both pieces of information stored in named properties:
$array = #'
Name,ID
Server1,SID1
Server2,SID2
'# |ConvertFrom-Csv
Now you can use a foreach loop without having to worry about the index:
foreach($computer in $array){
Write-Host "Server named '$($computer.Name)' has SID '$($computer.ID)'"
}
You could use a hash table or ordered dictionary (if you need the keys to keep their order), might be a good approach for your need.
Hash Table
$hash = #{
server1 = "SID1"
server2 = "SID2"
}
Ordered Dictionary
$dict = [ordered]#{
server1 = "SID1"
server2 = "SID2"
}
Then you can iterate over the key / value pairs either using .GetEnumerator():
foreach($pair in $hash.GetEnumerator()) {
Write-Host ("Server={0} and SID={1}" -f $pair.Key, $pair.Value)
}
Or by it's .Keys property:
foreach($key in $hash.PSBase.Keys) {
Write-Host ("Server={0} and SID={1}" -f $key, $hash[$key])
}
$array1 = #("server1", "server2")
$array2 = #("SID1", "SID2")
for($i=0; $i -lt $array1.Count; $i++){
Write-Host $array1[$i] ":"$array2[$i]
}
Related
I have a TXT file with 1300 megabytes (huge thing). I want to build code that does two things:
Every line contains a unique ID at the beginning. I want to check for all lines with the same unique ID if the conditions is met for that "group" of IDs. (This answers me: For how many lines with the unique ID X have all conditions been met)
If the script is finished I want to remove all lines from the TXT where the condition was met (see 2). So I can rerun the script with another condition set to "narrow down" the whole document.
After few cycles I finally have a set of conditions that applies to all lines in the document.
It seems that my current approach is very slow.( one cycle needs hours). My final result is a set of conditions that apply to all lines of code.
If you find an easier way to do that, feel free to recommend.
Help is welcome :)
Code so far (does not fullfill everything from 1&2)
foreach ($item in $liste)
{
# Check Conditions
if ( ($item -like "*XXX*") -and ($item -like "*YYY*") -and ($item -notlike "*ZZZ*")) {
# Add a line to a document to see which lines match condition
Add-Content "C:\Desktop\it_seems_to_match.txt" "$item"
# Retrieve the unique ID from the line and feed array.
$array += $item.Split("/")[1]
# Remove the line from final document
$liste = $liste -replace $item, ""
}
}
# Pipe the "new cleaned" list somewhere
$liste | Set-Content -Path "C:\NewListToWorkWith.txt"
# Show me the counts
$array | group | % { $h = #{} } { $h[$_.Name] = $_.Count } { $h } | Out-File "C:\Desktop\count.txt"
Demo Lines:
images/STRINGA/2XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg images/STRINGA/3XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg images/STRINGB/4XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg images/STRINGB/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg images/STRINGC/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
performance considerations:
Add-Content "C:\Desktop\it_seems_to_match.txt" "$item"
try to avoid wrapping cmdlet pipelines
See also: Mastering the (steppable) pipeline
$array += $item.Split("/")[1]
Try to avoid using the increase assignment operator (+=) to create a collection
See also: Why should I avoid using the increase assignment operator (+=) to create a collection
$liste = $liste -replace $item, ""
This is a very expensive operation considering that you are reassigning (copying) a long list ($liste) with each iteration.
Besides it is a bad practice to change an array that you are currently iterating.
$array | group | ...
Group-Object is a rather slow cmdlet, you better collect (or count) the items on-the-fly (where you do $array += $item.Split("/")[1]) using a hashtable, something like:
$Name = $item.Split("/")[1]
if (!$HashTable.Contains($Name)) { $HashTable[$Name] = [Collections.Generic.List[String]]::new() }
$HashTable[$Name].Add($Item)
To minimize memory usage it may be better to read one line at a time and check if it already exists. Below code I used StringReader and you can replace with StreamReader for reading from a file. I'm checking if the entire string exists, but you may want to split the line. Notice I have duplicaes in the input but not in the dictionary. See code below :
$rows= #"
images/STRINGA/2XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGA/3XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGB/4XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGB/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGC/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGA/2XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGA/3XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGB/4XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGB/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGC/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
"#
$dict = [System.Collections.Generic.Dictionary[int, System.Collections.Generic.List[string]]]::new();
$reader = [System.IO.StringReader]::new($rows)
while(($row = $reader.ReadLine()) -ne $null)
{
$hash = $row.GetHashCode()
if($dict.ContainsKey($hash))
{
#check if list contains the string
if($dict[$hash].Contains($row))
{
#string is a duplicate
}
else
{
#add string to dictionary value if it is not in list
$list = $dict[$hash].Value
$list.Add($row)
}
}
else
{
#add new hash value to dictionary
$list = [System.Collections.Generic.List[string]]::new();
$list.Add($row)
$dict.Add($hash, $list)
}
}
$dict
This question already has answers here:
PowerShell output is crossing between functions
(1 answer)
How do I prevent Powershell from closing after completion of a script?
(2 answers)
weird delay of the output of an object when followed by start-sleep (or until script end)
(3 answers)
Closed 4 months ago.
I'm having a problem with unexpected order of things being run and returned in the below Powershell script.
The Write-ArrayToTable function is to output the data in the arrays in a pretty table-like fashion via custom object.
The problem is when I call my Write-ArrayToTable function, it then does not return the data until AFTER the Read-Host command returns.
Below is the output from a run of the script, and the code itself is below that. The table output should be displayed BEFORE the Read-Host call but is instead held until the end and then displayed.
What am I missing? Any help greatly appreciated!
Output:
Test: y
Label1 Label2
------ ------
Test1 Test3
Test2 Test4
y
Code:
Function Write-ArrayToTable{
param(
[String[]]$Names,
[Object[][]]$Data
)
for($i = 0;; ++$i){
$Props = [ordered]#{}
for($j = 0; $j -lt $Data.Length; ++$j){
if($i -lt $Data[$j].Length){
$Props.Add($Names[$j], $Data[$j][$i])
}
}
if(!$Props.get_Count()){
break
}
[PSCustomObject]$Props
}
}
$arr1 = #("Test1","Test2")
$arr2 = #("Test3","Test4")
Write-ArrayToTable "Label1","Label2" $arr1,$arr2
Read-Host "Test"
Instead of dropping you objects like this:
[PSCustomObject]$Props
You can be more explicit:
$Props | Out-String
If you want to print all objects in one table, collect them first, before printing:
Function Write-ArrayToTable{
param(
[String[]]$Names,
[Object[][]]$Data
)
$myProps = for($i = 0;; ++$i){
$Props = [ordered]#{}
for($j = 0; $j -lt $Data.Length; ++$j){
if($i -lt $Data[$j].Length){
$Props.Add($Names[$j], $Data[$j][$i])
}
}
if(!$Props.get_Count()){
break
}
[PSCustomObject]$Props
}
$myProps | Format-Table
}
This question already has answers here:
Add items into a collection array dynamically
(2 answers)
Closed 1 year ago.
I am creating a script to parse a CSV file, where I store the content of each indexed field in the CSV as a NoteProperty in a PSCustomObject.
As I parse the file line by line, I add the PSCustomObject to a list type. When I output my list, I want to be able to do something like:
$list | Format-Table
and have a nice view of each row in the csv file, separated into columns with the heading up top.
Problem
When I add a PSCustomObject to the list, it changes the type of the list to a PSCustomObject. In practice, this has the apparent effect of applying any updates made to that PSCustomObject to every element in the list retroactively.
Here is a sample:
$list = [System.Collections.Generic.List[object]]::new()
$PSCustomObject = [PSCustomObject]#{ count = 0}
Foreach ($i in 1..5) {
$PSCustomObject.count +=1
$list.Add($PSCustomObject)
}
Expected Output:
PS>$list
count
-----
1
2
3
4
5
Actual Output:
PS>$list
count
-----
5
5
5
5
5
Question
Is there any way to get the expected output?
Limitations / additional context if it helps
I'm trying to optimize performance, as I may parse very large CSV files. This is why I am stuck with a list. I understand the Add method in lists is faster than recreating an array with += for every row. I am also using a runspace pool to parse each field separately and update the object via $list.$field[$lineNumber] = <field value>, so this is why I need a way to dynamically update the PSCustomObject. A larger view of my code is:
$out = [hashtable]::Synchronized(#{})
$out.Add($key, #{'dataSets' = [List[object]]::new() } ) ### $key is the file name as I loop through each csv in a directory.
$rowTemplate = [PSCustomObject]#{rowNum = 0}
### Additional steps to prepare the $out dictionary and some other variables
...
...
try {
### Skip lines prior to the line with the headers
$fileParser = [System.IO.StreamReader]$path
Foreach ( $i in 1..$headerLineNumber ) {
[void]$fileParser.ReadLine()
}
### Load the file into a variable, and add empty PSCustomObjects for each line as a placeholder.
while ($null -ne ($line = $fileParser.ReadLine())) {
[void]$fileContents.Add($line)
$rowTemplate.RowNum += 1
[void]$out.$key.dataSets.Add($rowTemplate)
}
}
finally {$fileParser.close(); $fileParser.dispose()}
### Prepare the script block for each runspace
$runspaceScript = {
Param( $fileContents, $column, $columnIndex, $delimiter, $key, $out )
$columnValues = [System.Collections.ArrayList]::new()
$linecount = 0
Foreach ( $line in $fileContents) {
$entry = $line.split($delimiter)[$columnIndex]
$out.$key.dataSets[$linecount].$column = $entry
$linecount += 1
}
}
### Instantiate the runspace pool.
PS Version (5.1.19041)
You're (re-)adding the same object to the list, over and over.
You need to create a new object every time your loop runs, but you can still "template" the objects - just use a hashtable/dictionary instead of a custom object:
# this hashtable will be our object "template"
$scaffold = #{ Count = 0}
foreach($i in 1..5){
$scaffold.Count += 1
$newObject = [pscustomobject]$scaffold
$list.Add($newObject)
}
As mklement0 suggests, if you're templating objects with multiple properties you might want to consider using an ordered dictionary to retain the order of the properties:
# this hashtable will be our object "template"
$scaffold = [ordered]#{ ID = 0; Count = 0}
foreach($i in 1..5){
$scaffold['ID'] = Get-Random
$scaffold['Count'] = $i
$newObject = [pscustomobject]$scaffold
$list.Add($newObject)
}
I have two dictionaries like this:
$first = #{}
$first.Add('John', 'Doe')
$first.Add('Johnny', 'Doe')
$second = #{}
$second.Add('Jack', 'Test')
$second.Add('Jacky', 'Test')
And I have a general $all = #{} dictionary, that stands for all dictionaries combined.
Ex. when I want to see all keys that $all contains:
foreach($key in $all){
Write-Host $key
}
It will show this:
John
Johnny
Jack
Jacky
p.s. I have this one:
$all = #{}
$all_dict = #{}
$all_dict += $first
$all_dict += $second
foreach($dict in $all_dict){
foreach($key in $dict.Key){
$all.Add($key, $dict[$key])
}
}
But I was wondering if there is another way to do it without the need to add all dictionaries to an array and then iterate through them
I wouldn't do the += addition to hashtables, but instead use a ForEach-Object on the hashes .Keys. That way, the code can be shortened, but also it will leave you an easier choice whether you would want the possible duplicates from Hashtable 1 ($first) to be overwritten by the values from the second Hashtable ($second).
Something like this:
$first = #{}
$first.Add('John', 'Doe')
$first.Add('Johnny', 'Doe')
$second = #{}
$second.Add('Jack', 'Test')
$second.Add('Jacky', 'Test')
$second.Add('Johnny', 'Depp') # Duplicate key: same first name, different lastname
$all = #{}
# copy all keys and values from the $first Hashtable into $all
$first.Keys | ForEach-Object { $all[$_] = $first[$_] }
For the next part, you'll have to decide what to do with duplicate keys:
Method 1
# add the stuff from Hashtable $second to it:
# this will overwrite the value if the key already exists (i.e. $second value 'wins')
$second.Keys | ForEach-Object { $all[$_] = $second[$_] }
OR use Method 2
# make sure the value of the $first hashtable is NOT overwritten (i.e. $first value 'wins')
$second.Keys | ForEach-Object { if (!($all.ContainsKey($_))) { $all[$_] = $second[$_] }}
In case you choose to overwrite (method 1), the $all hash will contain
Name Value
---- -----
John Doe
Jacky Test
Johnny Depp
Jack Test
If you choose NOT to overwrite (method 2), $all will be
Name Value
---- -----
John Doe
Jacky Test
Johnny Doe
Jack Test
Edit
There is another approach where you rely on the fact that an exception is thrown if you try to add an entry that already exists. In that case, use the .Add(key, value) method and wrap it inside a try{..} catch{..} block.
Without that catch, the error prevents the $all Hashtable to be filled, as it stops at the first duplicate key you try to add.
$second.Keys | ForEach-Object {
try {
$all.Add($_, $second[$_])
}
catch {
# catch the exception in order to carry on adding items
# the effect will be that the values from $first will not be overwritten
# just like with method 2
Write-Warning $_.Exception.Message
}
}
I think your $all_dict already contains what you want (i.e. a hashtable with all 4 entries), but your foreach( $dict in $all_dict ) isn't enumerating the hashtable entries like you expect it to.
The quick answer is to iterate over the Keys collection instead:
foreach( $key in $all.Keys )
{
write-host $key
}
The longer answer is that in your example PowerShell is doing some "helper" things for you with enumeration - foreach($key in $all) is only enumerating over a single object ($all), but write-host $all is evaluating an array of all of the entries in $all and serializing them into a single string:
Compare the behaviour of these two lines and you can see the difference:
PS> foreach($item in #{ "aaa"="bbb"; "ccc"="ddd" }) { write-host $item }
System.Collections.DictionaryEntry System.Collections.DictionaryEntry
PS> foreach($item in #{ "aaa"="bbb"; "ccc"="ddd" }.Keys) { write-host $item }
ccc
aaa
By the way, watch out for if your keys collide - if you try #{ "aaa"="bbb"; "ccc"="ddd" } + #{ "aaa"="eee" } for example, you'll get an error Item has already been added. Key in dictionary: 'aaa' Key being added: 'aaa'. so you might want to find a better way to merge your hashtables rather than just using +.
Am I not understanding the question? You can add them.
$first = #{John = 'Doe'; Johnny = 'Doe'} # hashtables
$second = #{Jacky = 'Test'; Jack = 'Test'}
$all = $first + $second # merge two hashtables
foreach ($i in $all.getenumerator()) { $i } # loops 4 times
This question already has answers here:
Powershell create array of arrays
(3 answers)
Closed 5 years ago.
This is building $ret into a long 1 dimensional array rather than an array of arrays. I need it to be an array that is populated with $subret objects. Thanks.
$ret = #()
foreach ($item in $items){
$subret = #()
$subRet = $item.Name , $item.Value
$ret += $subret
}
there might be other ways but arraylist normally works for me, in this case I would do:
$ret = New-Object System.Collections.ArrayList
and then
$ret.add($subret)
The suspected preexisting duplicate question is indeed a duplicate:
Given that + with an array as the LHS concatenates arrays, you must nest the RHS with the unary form of , (the array-construction operator) if it is an array that should be added as a single element:
# Sample input
$items = [pscustomobject] #{ Name = 'n1'; Value = 'v1'},
[pscustomobject] #{ Name = 'n2'; Value = 'v2'}
$ret = #() # create an empty *array*
foreach ($item in $items) {
$subret = $item.Name, $item.Value # use of "," implicitly creates an array
$ret += , $subret # unary "," creates a 1-item array
}
# Show result
$ret.Count; '---'; $ret[0]; '---'; $ret[1]
This yields:
2
---
n1
v1
---
n2
v2
The reason the use of [System.Collections.ArrayList] with its .Add() method worked too - a method that is generally preferable when building large arrays - is that .Add() only accepts a single object as the item to add, irrespective of whether that object is a scalar or an array:
# Sample input
$items = [pscustomobject] #{ Name = 'n1'; Value = 'v1'},
[pscustomobject] #{ Name = 'n2'; Value = 'v2'}
$ret = New-Object System.Collections.ArrayList # create an *array list*
foreach ($item in $items) {
$subret = $item.Name, $item.Value
# .Add() appends whatever object you pass it - even an array - as a *single* element.
# Note the need for $null = to suppress output of .Add()'s return value.
$null = $ret.Add($subret)
}
# Produce sample output
$ret.Count; '---'; $ret[0]; '---'; $ret[1]
The output is the same as above.
Edit
It is more convoluted to create an array of tuples than fill an array with PsObjects containing Name Value as the two properties.
Select the properties you want from $item then add them to the array
$item = $item | select Name, Value
$arr = #()
$arr += $item
You can reference the values in this array by doing this
foreach($obj in $arr)
{
$name = $obj.Name
$value = $obj.Value
# Do actions with the values
}