I have two arrays $list_CloudUsers and $list_Active. Both have a column called Alias.
I want to filter $list_CloudUsers so that the list does not have any of the ALIASes that are contained in $list_Active.
I was able to do it with:
$list_arc = Import-CSV $Arc_LastAccess
$list_Active = $list_arc | Where { [int]$_.InactivityDays -le 30}
$NewList = #()
ForEach ($User_Cloud in $list_CloudUsers)
{
$Alias = $User_Cloud.Alias
if ($list_Active -match $alias) {continue}
$NewList += $User_Cloud
}
But it does not work with this. Any ideas how I can get the WHERE to work correctly.
$list_arc = Import-CSV $Arc_LastAccess
$list_Active = $list_arc | Where { [int]$_.InactivityDays -le 30}
$NewList1 = $list_CloudUsers | Where {$list_Active -NotMatch $_.alias}
Try
$NewList1 = $list_CloudUsers | Where-Object {$list_Active.Alias -notcontains $_.Alias}
Related
I need some help for executing a little script and filtering results ...
I checkmy VM diskspace with the following script
Get-VM | ForEach-Object {
$VM = $_
$_.Guest.Disks | ForEach-Object {
$Report = "" | Select-Object -Property VM,Path,Capacity,FreeSpace,PercentageFreeSpace
$Report.VM = $VM.Name
$Report.Path = $_.Path
$Report.Capacity = $_.Capacity
$Report.FreeSpace = $_.FreeSpace
if ($_.Capacity) {$Report.PercentageFreeSpace = [math]::Round(100*($_.FreeSpace/$_.Capacity))}
$report
}
}
But I'd like to add a filter that my report only show me PercentageFreeSPace lesser than 20.
I try to add a where-object condition to my report with no success...
Can somebody help me please ? I m a beginner in PS...
Thanks in advance,
Best regards
For future questions you should format your script to be more readable.
I believe I have done it quite well below:
[EDIT] Seems that something gone wrong with displaying your question on my laptop. Just after my answer, the formatting is almost the same as provided below.
Get-VM | ForEach-Object {
$VM = $_
$_.Guest.Disks | ForEach-Object {
$Report = "" | Select-Object -Property VM,Path,Capacity,FreeSpace,PercentageFreeSpace $report
$Report.VM = $VM.Name
$Report.Path = $_.Path
$Report.Capacity = $_.Capacity
$Report.FreeSpace = $_.FreeSpace
if ($_.Capacity) {
$Report.PercentageFreeSpace = [math]::Round(100*($_.FreeSpace/$_.Capacity))
}
$report
}
}
To be honest I do not understand why you are using pipe at line 4.
Regarding your question, you should puy Where-Object clause before you will go to second for each loop.
First off, try and avoid multiple nested pipes to foreach-object, use foreach instead. That is, don't do
Get-VM | ForEach-Object {
$VM = $_
$_.Guest.Disks | ForEach-Object {
but
foreach($vm in Get-VM) {
foreach($disk in $vm.Guest.disks) {
This makes it easy to see what objects are handled later, and there isn't need to save current object on the pipeline into a temp variable (which you do by $VM = $_).
Also, objects can be initialized via a hash table instead of using select-object. Like so,
$pctLimit = 20 # Variable sets percentage limit instead of a magic number
foreach($vm in Get-VM) {
foreach($disk in $vm.Guest.Disks) {
# Report object is created via hash table
$Report = new-object PSObject -Property #{
VM = $VM.Name
Path = $disk.Path
Capacity = $disk.Capacity
FreeSpace = $disk.FreeSpace
ZeroDisk = ($disk.Capacity -gt 0) # NB: calculated property
}
if (-not $Report.ZeroDisk) { # Process only non-zero disks
$Report.PercentageFreeSpace = [math]::Round(100*($Report.FreeSpace/$Report.Capacity))
if($Report.PercentageFreeSpace -lt $pctLimit) {
$report
} else {
# Disk had more than $pctLimit free space. Now what?
}
} else {
# Disk had zero capacity. Now what?
}
}
}
FIrst of all thanks for your help
I manage to filter with the following
Get-VM | ForEach-Object {
$VM = $_
$_.Guest.Disks | ForEach-Object {
$Report = "" | Select-Object -Property VM,Path,Capacity,FreeSpace,PercentageFreeSpace
$Report.VM = $VM.Name
$Report.Path = $_.Path
$Report.Capacity = $_.Capacity
$Report.FreeSpace = $_.FreeSpace
if ($_.Capacity) {$Report.PercentageFreeSpace = [math]::Round(100*($_.FreeSpace/$_.Capacity))}
if( $_.Capacity -and $Report.PercentageFreeSpace -lt 30 ) {
$Report
}
}
}
`
Thanks,
Best regards
I am very new in powershell.
I am trying to validate my CSV file by finding out if there is any text value in my numeric fields. I can define with columns are numeric.
This is my source data like this
ColA ColB ColC ColD
23 23 ff 100
2.30E+01 34 2.40E+01 23
df 33 ss df
34 35 36 37
I need output something like this (only text values if found in any column)
ColA ColC ColD
2.30E+01 ff df
df 2.40E+01
ss
I have tried some code but not getting any results, get only some output like as under
System.Object[]
---------------
xxx fff' ddd 3.54E+03
...
This is what I was trying
#
cls
function Is-Numeric ($Value) {
return $Value -match "^[\d\.]+$"
}
$arrResult = #()
$arraycol = #()
$FileCol = #("ColA","ColB","ColC","ColD")
$dif_file_path = "C:\Users\$env:username\desktop\f2.csv"
#Importing CSVs
$dif_file = Import-Csv -Path $dif_file_path -Delimiter ","
############## Test Datatype (Is-Numeric)##########
foreach($col in $FileCol)
{
foreach ($line in $dif_file) {
$val = $line.$col
$isnum = Is-Numeric($val)
if ($isnum -eq $false) {
$arrResult += $line.$col
$arraycol += $col
}
}
}
[pscustomobject]#{$arraycol = "$arrResult"}| out-file "C:\Users\$env:username\Desktop\Errors1.csv"
####################
can someone guide me right direction?
Thanks
You can try something like this,
function Is-Numeric ($Value) {
return $Value -match "^[\d\.]+$"
}
$dif_file_path = "C:\Users\$env:username\desktop\f2.csv"
#Importing CSVs
$dif_file = Import-Csv -Path $dif_file_path -Delimiter ","
#$columns = $dif_file | Get-member -MemberType 'NoteProperty' | Select-Object -ExpandProperty 'Name'
# Use this to specify certain columns
$columns = "ColB", "ColC", "ColD"
foreach($row in $dif_file) {
foreach ($col in $columns) {
if ($col -in $columns) {
if (!(Is-Numeric $row.$col)) {
$row.$col = ""
}
}
}
}
$dif_file | Export-Csv C:\temp\formatted.txt
Look up name of columns as you go
Look up values of each col in each row and if it is not numeric, change to ""
Exported updated file.
I think not displaying columns that have no data creates the challenge here. You can do the following:
$csv = Import-Csv "C:\Users\$env:username\desktop\f2.csv"
$finalprops = [collections.generic.list[string]]#()
$out = foreach ($line in $csv) {
$props = $line.psobject.properties | Where {$_.Value -notmatch '^[\d\.]+$'} |
Select-Object -Expand Name
$props | Where {$_ -notin $finalprops} | Foreach-Object { $finalprops.add($_) }
if ($props) {
$line | Select $props
}
$out | Select-Object ($finalprops | Sort)
Given the nature of Format-Table or tabular output, you only see the properties of the first object in the collection. So if object1 has ColA only, but object2 has ColA and ColB, you only see ColA.
The output order you want is quite different than the input CSV; you're tracking bad text data not by first occurrence, but by column order, which requires some extra steps.
test.csv file contents:
ColA,ColB,ColC,ColD
23,23,ff,100
2.30E+01,34,2.40E+01,23
df,33,ss,df
34,35,36,37
Sample code tested to meet your description:
$csvIn = Import-Csv "$PSScriptRoot\test.csv";
# create working data set with headers in same order as input file
$data = [ordered]#{};
$csvIn[0].PSObject.Properties | foreach {
$data.Add($_.Name, (New-Object System.Collections.ArrayList));
};
# add fields with text data
$csvIn | foreach {
$_.PSObject.Properties | foreach {
if ($_.Value -notmatch '^-?[\d\.]+$') {
$null = $data[$_.Name].Add($_.Value);
}
}
}
$removes = #(); # remove `good` columns with numeric data
$rowCount = 0; # column with most bad values
$data.GetEnumerator() | foreach {
$badCount = $_.Value.Count;
if ($badCount -eq 0) { $removes += $_.Key; }
if ($badCount -gt $rowCount) { $rowCount = $badCount; }
}
$removes | foreach { $data.Remove($_); }
0..($rowCount - 1) | foreach {
$h = [ordered]#{};
foreach ($key in $data.Keys) {
$h.Add($key, $data[$key][$_]);
}
[PSCustomObject]$h;
} |
Export-Csv -NoTypeInformation -Path "$PSScriptRoot\text-data.csv";
output file contents:
"ColA","ColC","ColD"
"2.30E+01","ff","df"
"df","2.40E+01",
,"ss",
#Jawad, Finally I have tried
function Is-Numeric ($Value) {
return $Value -match "^[\d\.]+$"
}
$arrResult = #()
$columns = "ColA","ColB","ColC","ColD"
$dif_file_path = "C:\Users\$env:username\desktop\f1.csv"
$dif_file = Import-Csv -Path $dif_file_path -Delimiter "," |select $columns
$columns = $dif_file | Get-member -MemberType 'NoteProperty' | Select-Object -ExpandProperty 'Name'
foreach($row in $dif_file) {
foreach ($col in $columns) {
$val = $row.$col
$isnum = Is-Numeric($val)
if ($isnum -eq $false) {
$arrResult += $col+ " " +$row.$col
}}}
$arrResult | out-file "C:\Users\$env:username\desktop\Errordata.csv"
I get correct result in my out file, order is very ambiguous like
ColA ss
ColB 5.74E+03
ColA ss
ColC rrr
ColB 3.54E+03
ColD ss
ColB 8.31E+03
ColD cc
any idea to get proper format? thanks
Note: with your suggested code, I get complete source file with all data , not the specific error data.
I am attempting to analyze a group of text files (MSFTP logs) and do counts of IP addresses that have submitted bad credentials. I think I have it worked out except I don't think that the array is passing to/from the function correctly. As a result, I get duplicate entries if the same IP appears in multiple log files. What am I doing wrong?
Function LogBadAttempt($FTPLog,$BadPassesArray)
{
$BadPassEx="PASS - 530"
Foreach($Line in $FTPLog)
{
if ($Line -match $BadPassEx)
{
$IP=($Line.Split(' '))[1]
if($BadPassesArray.IP -contains $IP)
{
$CurrentIP=$BadPassesArray | Where-Object {$_.IP -like $IP}
[int]$CurrentCount=$CurrentIP.Count
$CurrentCount++
$CurrentIP.Count=$CurrentCount
}else{
$info=#{"IP"=$IP;"Count"='1'}
$BadPass=New-Object -TypeName PSObject -Property $info
$BadPassesArray += $BadPass
}
}
}
return $BadPassesArray
}
$BadPassesArray=#()
$FTPLogs = Get-Childitem \\ftpserver\MSFTPSVC1\test
$Result = ForEach ($LogFile in $FTPLogs)
{
$FTPLog=Get-Content ($LogFile.fullname)
LogBadAttempt $FTPLog
}
$Result | Export-csv C:\Temp\test.csv -NoTypeInformation
The result looks like...
Count IP
7 209.59.17.20
20 209.240.83.135
18441 209.59.17.20
13059 200.29.3.98
and would like it to combine the entries for 209.59.17.20
You're making this way too complicated. Process the files in a pipeline and use a hashtable to count the occurrences of each IP address:
$BadPasswords = #{}
Get-ChildItem '\\ftpserver\MSFTPSVC1\test' | Get-Content | ? {
$_ -like '*PASS - 530*'
} | % {
$ip = ($_ -split ' ')[1]
$BadPasswords[$ip]++
}
$BadPasswords.GetEnumerator() |
select #{n='IP';e={$_.Name}}, #{n='Count';e={$_.Value}} |
Export-Csv 'C:\Temp\test.csv' -NoType
I have scraped two files from a website in order to list the companies in my city.
The first lists : name, city, phone number, email
The second lists : name, city, phone number
And I will have duplicate lines if I merge them, as an example, i will have the following :
> "Firm1";"Los Angeles";"000000";"info#firm1.lol"
> "Firm1";"Los Angeles";"000000";""
> "Firm2";"Los Angeles";"111111";""
> "Firm3";"Los Angeles";"000000";"contact#firm3.lol"
> "Firm3";"Los Angeles";"000000";""
> ...
Is there a way to merge the two files and keep the max info like this :
> "Firm1";"Los Angeles";"000000";"info#firm1.lol"
> "Firm2";"Los Angeles";"111111";""
> "Firm3";"Los Angeles";"000000";"contact#firm3.lol"
> ...
According to the fact you've got a file like this called 'firm.csv'
"Firm1";"Los Angeles";"000000";"info#firm1.lol"
"Firm1";"Los Angeles";"000000";""
"Firm2";"Los Angeles";"111111";""
"Firm3";"Los Angeles";"000000";"contact#firm3.lol"
"Firm3";"Los Angeles";"000000";""
You can load it using :
$firms = import-csv C:\temp\firm.csv -Header 'Firm','Town','Tel','Mail' -Delimiter ';'
Then
$firms | Sort-Object -Unique -Property 'Firm'
According to Joey's comment I improved the solution :
$firms | Group-Object -Property 'firm' | % {$_.group | Sort-Object -Property mail -Descending | Select-Object -first 1}
EDIT: just realized the two files don't contain the same headers. Here is an update.
$main = Import-Csv firm1.csv -Header 'Firm','Town','Tel','Mail' -Delimiter ";"
$alt = Import-Csv firm2.csv -Header 'Firm','Town','Tel' -Delimiter ";"
foreach ($f in $alt)
{
$found = $false
foreach($g in $main)
{
if ($g.Firm -eq $f.Firm -and $g.city -eq $f.city)
{
$found = $true
if ($g.Tel -eq "")
{
$g.Tel = $f.Tel
}
}
}
if ($found -eq $false)
{
$main += $f
}
}
# Everything is merged into the $main array
$main
There must be better approach but this is one costy way to do this.
$firms = import-csv C:\firm.csv -Header 'Firm','Town','Tel','Mail' -Delimiter ';'
$Result = #()
ForEach($i in $firms){
$found = 0;
ForEach($m in $Result){
if($m.Firm -eq $i.Firm){
$found = 1
if( $i.Mail.length -ne 0 )
{
$m.Mail = $i.Mail
}
break;
}
}
if($found -eq 0){
$Result += [pscustomobject] #{Firm=$i.Firm; Town=$i.Town; Tel=$i.Tel; Mail=$i.Mail}
}
}
$Result | export-csv C:\out.csv
I have two csv files, i want to check the users in username.csv matches with userdata.csv copy
to output.csv. If it does not match return the name alone in the output.csv
For Ex: User Data contains 3 columns
UserName,column1,column2
Hari,abc,123
Raj,bca,789
Max,ghi,123
Arul,987,thr
Prasad,bxa,324
username.csv contains usernames
Hari
Rajesh
Output.csv should contain
Hari,abc,123
Rajesh,NA,NA
How to achieve this. Thanks
Sorry for that.
$Path = "C:\PowerShell"
$UserList = Import-Csv -Path "$($path)\UserName.csv"
$UserData = Import-Csv -Path "$($path)\UserData.csv"
foreach ($User in $UserList)
{
ForEach ($Data in $UserData)
{
If($User.Username -eq $Data.UserName)
{
# Process the data
$Data
}
}
}
This returns only matching values. I also need to add the non-matching values in output
file. Thanks.
something like this will work:
$Path = "C:\PowerShell"
$UserList = Import-Csv -Path "$($path)\UserName.csv"
$UserData = Import-Csv -Path "$($path)\UserData.csv"
$UserOutput = #()
ForEach ($name in $UserList)
{
$userMatch = $UserData | where {$_.UserName -eq $name.usernames}
If($userMatch)
{
# Process the data
$UserOutput += New-Object PsObject -Property #{UserName =$name.usernames;column1 =$userMatch.column1;column2 =$userMatch.column2}
}
else
{
$UserOutput += New-Object PsObject -Property #{UserName =$name.usernames;column1 ="NA";column2 ="NA"}
}
}
$UserOutput | ft
It loops through each name in the user list. Line 9 does a search of the userdata CSV for a matching user name if it finds it it adds the user data for that user to the output if no match is found it adds the user name to the output with NA in both columns.
had to change your userList csv:
usernames
Hari
Rajesh
expected output:
UserName column1 column2
-------- ------- -------
Hari abc 123
Rajesh NA NA
I had a similar situation, where I needed a "changed record collection" holding the entire record when the current record was either new or had any changes when compared to the previous record. This was my code:
# get current and previous CSV
$current = Import-Csv -Path $current_file
$previous = Import-Csv -Path $previous_file
# collection with new or changed records
$deltaCollection = New-Object Collections.Generic.List[System.Object]
:forEachCurrent foreach ($row in $current) {
$previousRecord = $previous.Where( { $_.Id -eq $row.Id } )
$hasPreviousRecord = ($null -ne $previousRecord -and $previousRecord.Count -eq 1)
if ($hasPreviousRecord -eq $false) {
$deltaCollection.Add($current)
continue forEachCurrent
}
# check if value of any property is changed when compared to the previous
:forEachCurrentProperty foreach ($property in $current.PSObject.Properties) {
$columnName = $property.Name
$currentValue = if ($null -eq $property.Value) { "" } else { $property.Value }
$previousValue = if ($hasPreviousRecord) { $previousRecord[0]."$columnName" } else { "" }
if ($currentValue -ne $previousValue -or $hasPreviousRecord -eq $false) {
$deltaCollection.Add($currentCenter)
continue forEachCurrentProperty
}
}
}