ForEach Nested Loop in PowerShell - powershell

I have this piece of code where I am extracting table names from the adventureworks.bim file using a for each loop. However, I am missing something here because I required a Table name per object and not the final table in the loop. The details are as below
cls
$BIM = "C:\Users\Desktop\adventureworks.bim"
$origmodel = (Get-Content $BIM -Raw) | Out-String | ConvertFrom-Json
ForEach($table in $origmodel.Model.tables.name)
{
$ColumnProperty = $origmodel.Model.tables.columns | ForEach-Object {
[pscustomobject] #{
'Table Name' = $table
'Object Name' = $_.name
'DataType' = $_.dataType
}
}
}
$ColumnProperty | ConvertTo-Csv -NoTypeInformation
Result I Get
"Table Name", "Object Name", "Datatype"
"Date","RowNumber-2662979B-1795-4F74-8F37-6A1BA8059B61","int64"
"Date","CurrencyKey","int64"
"Date","Currency Code","string"
"Date","CurrencyName","string"
"Date","RowNumber-2662979B-1795-4F74-8F37-6A1BA8059B61","int64"
"Date","CustomerKey","int64"
"Date","GeographyKey","int64"
"Date","Customer Id","string"
"Date","Title","string"
"Date","First Name","string"
"Date","Middle Name","string"
"Date","Last Name","string"
"Date","Name Style","boolean",
"Date","Birth Date","dateTime",
"Date","Marital Status","string"
"Date","Suffix","string"
"Date","Gender","string"
"Date","RowNumber-2662979B-1795-4F74-8F37-6A1BA8059B61","int64"
"Date","DateKey","int64"
"Date","Date","dateTime",
"Date","Day Number Of Week","int64"
"Date","Day Name Of Week","string"
"Date","Day Of Year","int64"
"Date","Week Of Year","int64"
"Date","Month Name","string"
Result I Need
"Table Name", "Object Name", "DataType"
"Currency","RowNumber-2662979B-1795-4F74-8F37-6A1BA8059B61","int64"
"Currency","CurrencyKey","int64"
"Currency","Currency Code","string"
"Currency","CurrencyName","string"
"Customer","RowNumber-2662979B-1795-4F74-8F37-6A1BA8059B61","int64"
"Customer","CustomerKey","int64"
"Customer","GeographyKey","int64"
"Customer","Customer Id","string"
"Customer","Title","string"
"Customer","First Name","string"
"Customer","Middle Name","string"
"Customer","Last Name","string"
"Customer","Name Style","boolean",
"Customer","Birth Date","dateTime",
"Customer","Marital Status","string"
"Customer","Suffix","string"
"Customer","Gender","string"
"Date","RowNumber-2662979B-1795-4F74-8F37-6A1BA8059B61","int64"
"Date","DateKey","int64"
"Date","Date","dateTime",
"Date","Day Number Of Week","int64"
"Date","Day Name Of Week","string"
"Date","Day Of Year","int64"
"Date","Week Of Year","int64"
"Date","Month Name","string"

Try this on for size:
$BIM = "C:\Users\Desktop\adventureworks.bim"
$origmodel = Get-Content $BIM -Raw | ConvertFrom-Json
ForEach ($table in $origmodel.Model.tables) {
$ColumnProperty += $table.columns | ForEach-Object {
[pscustomobject] #{
'Table Name' = $table.name
'Object Name' = $_.name
'DataType' = $_.dataType
}
}
}
$ColumnProperty | ConvertTo-Csv -NoTypeInformation
Corrections
No need to use Out-String
ColumnProperty needed a += as = was overwriting every entry
Just a few mix-ups around the foreach loops.

You need to loop over .model.tables first then an inner loop for each columns:
$req = Invoke-RestMethod https://raw.githubusercontent.com/TabularEditor/TabularEditor/master/TabularEditorTest/AdventureWorks.bim
$req.model.tables | ForEach-Object {
foreach($column in $_.columns) {
[pscustomobject]#{
Table = $_.name
ObjectName = $column.name
DataType = $column.dataType
}
}
} | Export-Csv path\to\export.csv -NoTypeInformation
Output to the console would look like this for the first few objects:
Table ObjectName DataType
----- ---------- --------
Currency RowNumber-2662979B-1795-4F74-8F37-6A1BA8059B61 int64
Currency CurrencyKey int64
Currency Currency Code string
Currency CurrencyName string
Customer RowNumber-2662979B-1795-4F74-8F37-6A1BA8059B61 int64
Customer CustomerKey int64
Customer GeographyKey int64
Customer Customer Id string
Customer Title string
...
...

Related

search csv for multiple strings in column a in powershell

I have a sample.csv file
#Period,Account,Entity,Year,Version,Currency,HSP_Rates,Scenario,Data
Apr,1,9,FY22,F,L,H,And,2
Apr,1,9,FY22,F,L,H,And,2
Apr,1,9,FY22,F,L,H,OR,2
here i want to get output csv file where scenario only equals to AND
#Period,Account,Entity,Year,Version,Currency,HSP_Rates,Scenario,Data
Apr,1,9,FY22,F,L,H,And,2
Apr,1,9,FY22,F,L,H,And,2
i have written code which is not giving required output
$csvRaw = Get-Content -Path 'D:\sample.csv' -Raw
$Csv = $CsvRaw.TrimStart('#') | ConvertFrom-Csv
$NewCsv = $csv | ForEach-Object {
[PsCustomObject]#{
Period = $_.Period
Account = $_.Account
Entity = $_.Entity
Year = $_.Year
Version = $_.Version
Currency = $_.Currency
HSP_Rates = $_.HSP_Rates
Scenario = $_.Scenario | where {$_.Scenario -match "And" }
Data = $_.Data
} }
$OutCsv = ($NewCsv | ConvertTo-Csv -NoTypeInformation).TrimStart('"#')
# Converting back to CSV surround everything with double quotes. # We need to insert back the hash sign for the #period header$OutCsv[0] = """#" + $OutCsv[0]
$OutCsv | Out-File 'D:\sample_1.csv'
It looks like a syntax issue in the construction of the [PSCustomObject].
Try replacing:
$NewCsv = $csv | ForEach-Object {
[PsCustomObject]#{
Period = $_.Period
Account = $_.Account
Entity = $_.Entity
Year = $_.Year
Version = $_.Version
Currency = $_.Currency
HSP_Rates = $_.HSP_Rates
Scenario = $_.Scenario | where {$_.Scenario -match "And" }
Data = $_.Data
} }
With:
$NewCsv = $csv | ForEach-Object {
[PsCustomObject]#{
Period = $_.Period
Account = $_.Account
Entity = $_.Entity
Year = $_.Year
Version = $_.Version
Currency = $_.Currency
HSP_Rates = $_.HSP_Rates
Scenario = $_.Scenario
Data = $_.Data
} } | Where { $_.Scenario -match "And" }
Which - as Theo commented - is basically the same as:
$NewCsv = $csv | Where { $_.Scenario -match "And" }
Also, if you're using Powershell 6+, you may take advantage of the -UseQuotes parameter from the ConvertTo-CSV cmdlet to remove the quotes from the output.

How can I add string and create new column in my csv file using PowerShell

In my existing CSV file I have a column called "SharePoint ID" and it look like this
1.ylkbq
2.KlMNO
3.
4.MSTeam
6.
7.MSTEAM
8.LMNO83
and I'm just wondering how can I create a new Column in my CSV call "SharePoint Email" and then add "#gmail.com" to only the actual Id like "ylkbq", "KLMNO" and "LMNO83" instead of applying to all even in the blank space. And Maybe not add/transfer "MSTEAM" to the new Column since it's not an Id.
$file = "C:\AuditLogSearch\New folder\OriginalFile.csv"
$file2 = "C:\AuditLogSearch\New folder\newFile23.csv"
$add = "#GMAIL.COM"
$properties = #{
Name = 'Sharepoint Email'
Expression = {
switch -Regex ($_.'SharePoint ID') {
#Not sure what to do here
}
}
}, '*'
Import-Csv -Path $file |
Select-Object $properties |
Export-Csv $file2 -NoTypeInformation
Using calculated properties with Select-Object this is how it could look:
$add = "#GMAIL.COM"
$expression = {
switch($_.'SharePoint ID')
{
{[string]::IsNullOrWhiteSpace($_) -or $_ -match 'MSTeam'}
{
# Null value or mathces MSTeam, leave this Null
break
}
Default # We can assume these are IDs, append $add
{
$_.Trim() + $add
}
}
}
Import-Csv $file | Select-Object *, #{
Name = 'SharePoint Email'
Expression = $expression
} | Export-Csv $file2 -NoTypeInformation
Sample Output
Index SharePoint ID SharePoint Email
----- ------------- ----------------
1 ylkbq ylkbq#GMAIL.COM
2 KlMNO KlMNO#GMAIL.COM
3
4 MSTeam
5
6 MSTEAM
7 LMNO83 LMNO83#GMAIL.COM
A more concise expression, since I misread the point, it can be reduced to just one if statement:
$expression = {
if(-not [string]::IsNullOrWhiteSpace($_.'SharePoint ID') -and $_ -notmatch 'MSTeam')
{
$_.'SharePoint ID'.Trim() + $add
}
}

Format-Table not taking effect (Exchange - powershell)

first of all sorry if my english is not the best. but ill try to explain my issue with as much detail as i can
Im having an issue where i cant get Format-Table to effect the output i give it.
below is the part im having issues with atm.
cls
$TotalSize = $($mailboxes. #{name = ”TotalItemSize (GB)”; expression = { [math]::Round((($_.TotalItemSize.Value.ToString()).Split(“(“)[1].Split(” “)[0].Replace(“,”, ””) / 1GB), 2) } });
$UserN = $($mailboxes.DisplayName)
$itemCount = $($mailboxes.ItemCount)
$LastLogonTime = $($mailboxes.ItemCount)
$allMailboxinfo = #(
#lager dataen som skal inn i et objekt
#{Username= $UserN; ItemCount = $itemCount; LastLogonTime = $($mailboxes.ItemCount); Size = $TotalSize}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$Table = $allMailboxinfo | Format-Table | Out-String
$Table
the output of this gives me what almost looks like json syntax below each title of the table.
Username LastLogonTime ItemCount Size
-------- ------------- --------- ----
{username1, username2,username3,userna...} {$null, $null, $null, $null...} {$null, $null, $null, $null...} {$null, $null, $null, $null...}
running the commands by themselves seem to work tho. like $mailboxes.DisplayName gives the exact data i want for displayname. even in table-format.
the reason im making the table this way instead of just using select-object, is because im going to merge a few tables later. using the logic from the script below.
cls
$someData = #(
#{Name = "Bill"; email = "email#domain.com"; phone = "12345678"; id = "043546" }) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$moreData = #(
#{Name = "Bill"; company = "company 04"}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$Merge = #(
#plots the data into a new table
#{Name = $($someData.Name); e_mail = $($someData.email); phone = $($someData.phone); id = $($someData.id); merged = $($moreData.company) }) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
#formatting table
$Table = $Merge | Format-Table | Out-String
#print table
$Table
if you are wondering what im doing with this.
My goal, all in all. is a table with using the info from Exchange;
DisplayName, TotalItemSize(GB), ItemCount, LastLogonTime, E-mail adress, archive + Maxquoata, Quoata for mailbox.
You're creating a single object where each property holds an array of property values from the original array of mailbox objects.
Instead, create 1 new object per mailbox:
# construct output objects with Select-Object
$allMailBoxInfo = $mailboxes |Select #{Name='Username';Expression='DisplayName'},ItemCount,#{Name='LastLogonTime';Expression='ItemCount'},#{Name='Size';Expression={[math]::Round((($_.TotalItemSize.Value.ToString()).Split("(")[1].Split(" ")[0].Replace(",", "") / 1GB), 2) }}
# format table
$Table = $allMailBoxInfo | Format-Table | Out-String
# print table
$Table

How to join and group 2 tables with a comma list in Powershell?

I have two array which have the following objects:
$dataset1 = #(
#{
MachineName = "AAA"
ID = "111"
},
#{
MachineName = "BBB"
ID = "222"
},
#{
MachineName = "CCC"
ID = "111"
},
#{
MachineName = "DDD"
ID = "333"
},
#{
MachineName = "EEE"
ID = "111"
}
)
$dataset2 = #(
#{
ID = "111"
TagName = "ALPHA2"
},
#{
ID = "222"
TagName = "ALPHA0"
},
#{
ID = "333"
TagName = "ALPHA8"
},
#{
ID = "444"
TagName = "ALPHA29"
},
)
Now I want to create an array which have an object of TagName and for each object of TagName it should contain a list of MachineNames separated by comma so something like this:
TagName | MachineName
ALPHA2 AAA,CCC,EEE
ALPHA0 BBB
ALPHA8 DDD
This is the code that I have tried:
$Joined= Foreach ($row in $dataset1)
{
[pscustomobject]#{
ID = $row.ID
MachineName = $row.MachineName -join ','
TagName = $dataset2 | Where-Object {$_.ID -eq $row.ID} | Select-Object -ExpandProperty TagName
}
}
But it is not generating a comma list of Machine names instead it is printing individual rows for each machine name.
I would iterate $dataset2 instead of $dataset1.
$joined = foreach($row in $dataset2){
[PSCustomObject]#{
TagName = $row.tagname
MachineName = ($dataset1 | Where-Object id -eq $row.id).MachineName -join ','
}
}
If you're trying to exclude those entries in $dataset1 that don't have a corresponding entry in $dataset2, change to this.
$joined = foreach($row in $dataset2 | Where-Object id -in $dataset1.id){
[PSCustomObject]#{
TagName = $row.tagname
MachineName = ($dataset1 | Where-Object id -eq $row.id).MachineName -join ','
}
}
it is not generating a comma list of Machine names instead it is printing individual rows for each machine name.
That's actually a great starting point!
Just pipe the resulting objects to Group-Object and extract the machine names from each resulting group:
$joined |Group-Object TagName |Select Name,#{Name='Machines';Expression = {$_.Group.MachineName -join ','}}
Which should give you something like:
Name Machines
---- --------
ALPHA2 AAA,CCC,EEE
ALPHA0 BBB
ALPHA8 DDD

Find out Text data in CSV File Numeric Columns in Powershell

I am very new in powershell.
I am trying to validate my CSV file by finding out if there is any text value in my numeric fields. I can define with columns are numeric.
This is my source data like this
ColA ColB ColC ColD
23 23 ff 100
2.30E+01 34 2.40E+01 23
df 33 ss df
34 35 36 37
I need output something like this (only text values if found in any column)
ColA ColC ColD
2.30E+01 ff df
df 2.40E+01
ss
I have tried some code but not getting any results, get only some output like as under
System.Object[]
---------------
xxx fff' ddd 3.54E+03
...
This is what I was trying
#
cls
function Is-Numeric ($Value) {
return $Value -match "^[\d\.]+$"
}
$arrResult = #()
$arraycol = #()
$FileCol = #("ColA","ColB","ColC","ColD")
$dif_file_path = "C:\Users\$env:username\desktop\f2.csv"
#Importing CSVs
$dif_file = Import-Csv -Path $dif_file_path -Delimiter ","
############## Test Datatype (Is-Numeric)##########
foreach($col in $FileCol)
{
foreach ($line in $dif_file) {
$val = $line.$col
$isnum = Is-Numeric($val)
if ($isnum -eq $false) {
$arrResult += $line.$col
$arraycol += $col
}
}
}
[pscustomobject]#{$arraycol = "$arrResult"}| out-file "C:\Users\$env:username\Desktop\Errors1.csv"
####################
can someone guide me right direction?
Thanks
You can try something like this,
function Is-Numeric ($Value) {
return $Value -match "^[\d\.]+$"
}
$dif_file_path = "C:\Users\$env:username\desktop\f2.csv"
#Importing CSVs
$dif_file = Import-Csv -Path $dif_file_path -Delimiter ","
#$columns = $dif_file | Get-member -MemberType 'NoteProperty' | Select-Object -ExpandProperty 'Name'
# Use this to specify certain columns
$columns = "ColB", "ColC", "ColD"
foreach($row in $dif_file) {
foreach ($col in $columns) {
if ($col -in $columns) {
if (!(Is-Numeric $row.$col)) {
$row.$col = ""
}
}
}
}
$dif_file | Export-Csv C:\temp\formatted.txt
Look up name of columns as you go
Look up values of each col in each row and if it is not numeric, change to ""
Exported updated file.
I think not displaying columns that have no data creates the challenge here. You can do the following:
$csv = Import-Csv "C:\Users\$env:username\desktop\f2.csv"
$finalprops = [collections.generic.list[string]]#()
$out = foreach ($line in $csv) {
$props = $line.psobject.properties | Where {$_.Value -notmatch '^[\d\.]+$'} |
Select-Object -Expand Name
$props | Where {$_ -notin $finalprops} | Foreach-Object { $finalprops.add($_) }
if ($props) {
$line | Select $props
}
$out | Select-Object ($finalprops | Sort)
Given the nature of Format-Table or tabular output, you only see the properties of the first object in the collection. So if object1 has ColA only, but object2 has ColA and ColB, you only see ColA.
The output order you want is quite different than the input CSV; you're tracking bad text data not by first occurrence, but by column order, which requires some extra steps.
test.csv file contents:
ColA,ColB,ColC,ColD
23,23,ff,100
2.30E+01,34,2.40E+01,23
df,33,ss,df
34,35,36,37
Sample code tested to meet your description:
$csvIn = Import-Csv "$PSScriptRoot\test.csv";
# create working data set with headers in same order as input file
$data = [ordered]#{};
$csvIn[0].PSObject.Properties | foreach {
$data.Add($_.Name, (New-Object System.Collections.ArrayList));
};
# add fields with text data
$csvIn | foreach {
$_.PSObject.Properties | foreach {
if ($_.Value -notmatch '^-?[\d\.]+$') {
$null = $data[$_.Name].Add($_.Value);
}
}
}
$removes = #(); # remove `good` columns with numeric data
$rowCount = 0; # column with most bad values
$data.GetEnumerator() | foreach {
$badCount = $_.Value.Count;
if ($badCount -eq 0) { $removes += $_.Key; }
if ($badCount -gt $rowCount) { $rowCount = $badCount; }
}
$removes | foreach { $data.Remove($_); }
0..($rowCount - 1) | foreach {
$h = [ordered]#{};
foreach ($key in $data.Keys) {
$h.Add($key, $data[$key][$_]);
}
[PSCustomObject]$h;
} |
Export-Csv -NoTypeInformation -Path "$PSScriptRoot\text-data.csv";
output file contents:
"ColA","ColC","ColD"
"2.30E+01","ff","df"
"df","2.40E+01",
,"ss",
#Jawad, Finally I have tried
function Is-Numeric ($Value) {
return $Value -match "^[\d\.]+$"
}
$arrResult = #()
$columns = "ColA","ColB","ColC","ColD"
$dif_file_path = "C:\Users\$env:username\desktop\f1.csv"
$dif_file = Import-Csv -Path $dif_file_path -Delimiter "," |select $columns
$columns = $dif_file | Get-member -MemberType 'NoteProperty' | Select-Object -ExpandProperty 'Name'
foreach($row in $dif_file) {
foreach ($col in $columns) {
$val = $row.$col
$isnum = Is-Numeric($val)
if ($isnum -eq $false) {
$arrResult += $col+ " " +$row.$col
}}}
$arrResult | out-file "C:\Users\$env:username\desktop\Errordata.csv"
I get correct result in my out file, order is very ambiguous like
ColA ss
ColB 5.74E+03
ColA ss
ColC rrr
ColB 3.54E+03
ColD ss
ColB 8.31E+03
ColD cc
any idea to get proper format? thanks
Note: with your suggested code, I get complete source file with all data , not the specific error data.