Using powershell, lets say I have a csv file that contains
fname,lname,id,etc..
Is there a way to use where-object to look through all of the columns instead of just one.
For example, instead of doing:
Import-csv location |where-Object {$_.fname -eq "hi"}
next line:
Import-csv location |where-Object {$_.lname -eq "hi"} and so on.
It would be something like:
Import-csv location |where-Object {ANY -eq "hi"}
Yes, you could iterate over $_.psobject.Properties to inspect every column and return $true if one of them matches:
Import-Csv document.csv |Where-Object {
foreach($p in ($_.psobject.Properties |? {$_.MemberType -eq 'NoteProperty'})) {
if($p.Value -match "hi"){ $true }
}
}
Since the column headings / property names are irrelevant and all you care about is the values, you can handle it as text:
Get-Content c:\somedir\somefile.csv |
Where {$_ -like '*"hi"*'}
If you're just after a count, you can just count the matches. If you want objects, you can run the matched lines through convertfrom-csv.
Related
I'm again stuck on something that should be so simple. I have a CSV file in which I need to do a few string modifications and export it back out. The data looks like this:
FullName
--------
\\server\project\AOI
\\server\project\AOI\Folder1
\\server\project\AOI\Folder2
\\server\project\AOI\Folder3\User
I need to do the following:
Remove the "\\server\project" from each line but leave the rest of the line
Delete all rows which do not have a Folder (e.g., in the example above, the first row would be deleted but the other three would remain)
Delete any row with the word "User" in the path
Add a column called T/F with a value of "FALSE" for each record
Here is my initial attempt at this:
Get-Content C:\Folders.csv |
% {$_.replace('\\server\project\','')} |
Where-Object {$_ -match '\\'} |
#Removes User Folders rows from CSV
Where-Object {$_ -notmatch 'User'} |
Out-File C:\Folders-mod.csv
This works to a certain extent, except it deletes my header row and I have not found a way to add a column using Get-Content. For that, I have to use Import-Csv, which is fine, but it seems inefficient to be constantly reloading the same file. So I tried rewriting the above using Import-Csv instead of Get-Content:
$Folders = Import-Csv C:\Folders.csv
foreach ($Folder in $Folders) {
$Folder.FullName = $Folder.FullName.Replace('\\server\AOI\', '') |
Where-Object {$_ -match '\\'} |
Where-Object {$_ -notmatch 'User Files'}
}
$Folders | Export-Csv C:\Folders-mod.csv -NoTypeInformation
I haven't added the coding for adding the new column yet, but this keeps the header. However, I end up with a bunch of empty rows where the Where-Object deletes the line, and the only way I can find to get rid of them is to run the output file through a Get-Content command. This all seems overly complicated for something that should be simple.
So, what am I missing?
Thanks to TheMadTechnician for pointing out what I was doing wrong. Here is my final script (with additional column added):
$Folders= Import-CSV C:\Folders.csv
ForEach ($Folder in $Folders)
{
$Folder.FullName = $Folder.FullName.replace('\\server\project\','')
}
$Folders | Where-Object {$_ -match '\\' -and $_ -notmatch 'User'} |
Select-Object *,#{Name='T/F';Expression={'FALSE'}} |
Export-CSV C:\Folders.csv -NoTypeInformation
I would do this with a Table Array and pscustomobject.
#Create an empty Array
$Table = #()
#Manipulate the data
$Fullname = Get-Content C:\Folders.csv |
ForEach-Object {$_.replace('\\server\project\', '')} |
Where-Object {$_ -match '\\'} |
#Removes User Folders rows from CSV
Where-Object {$_ -notmatch 'User'}
#Define custom objects
Foreach ($name in $Fullname) {
$Table += [pscustomobject]#{'Fullname' = $name; 'T/F' = 'FALSE'}
}
#Export results to new csv
$Table | Export-CSV C:\Folders-mod.csv -NoTypeInformation
here's yet another way to do it ... [grin]
$FileList = #'
FullName
\\server\project\AOI
\\server\project\AOI\Folder1
\\server\project\AOI\Folder2
\\server\project\AOI\Folder3\User
'# | ConvertFrom-Csv
$ThingToRemove = '\\server\project'
$FileList |
Where-Object {
# toss out any blank lines
$_ -and
# toss out any lines with "user" in them
$_ -notmatch 'User'
} |
ForEach-Object {
[PSCustomObject]#{
FullName = $_.FullName -replace [regex]::Escape($ThingToRemove)
'T/F' = $False
}
}
output ...
FullName T/F
-------- ---
\AOI False
\AOI\Folder1 False
\AOI\Folder2 False
notes ...
putting a slash in the property name is ... icky [grin]
that requires wrapping the property name in quotes every time you need to access it. try another name - perhaps "Correct".
you can test for blank array items [lines] with $_ all on its own
the [regex]::Escape() stuff is really quite handy
I have a list of objects and want to filter all elements whose name is like one of the strings in a list.
My current approach is to use where-object, but this results in a long chain of calls to where-object.
Get-AppxPackage |
where-object {$_.name -notlike "*store*"} |
where-object {$_.name -notlike "*MSPaint*"} ...
I would like to reduce the boilerplate needed here as I expect the list of strings to get quite large.
I would further like to be able to use the filter on another output, which seems I am currently only able to do by copy pasting the where-object block.
You can use regex notmatch instead. This will be a lot faster to execute.
Something like
Get-AppxPackage | Where-Object {$_.name -notmatch 'store|MSPaint'}
Instead of typing in the literal names to not match, you can build the pattern from an array or by reading in a textfile.
Lets say you have a list of names in a textfile
store
MSPaint
...
You can then read in this file as array with
$list = Get-Content -Path "<PATH TO THE FILE>"
Next combine this list to build the pattern like
$pattern = (($list | ForEach-Object {[regex]::Escape($_)}) –join "|")
and do
Get-AppxPackage | Where-Object {$_.name -notmatch $pattern}
Hope this helps
this code shows how to filter using -and and -or in the where-object
class cCustomField
{
[string] $Id
[string] $Name
[string] $Description
[string] $Type
[bool] $On_projects
[bool] $On_people
}
$list = New-Object Collections.Generic.List[cCustomField]
... load your data ....
filter using -and -or
$my_object=$list | Where-Object {$_.name -EQ 'gender' -and $_.on_people -EQ $true }
I'm attempting to search a hash table for keys that have values that are -notlike or -notmatch 'Snagit'. I am able find an exact match with -notcontains.
Data Example:
Name Value
Host1 FireFox,Snagit 7,Chrome
Host2 Internet,Chrome
Host3 Snagit 5,Internet,Stuff
Code Example:
$global:Csv = Import-Csv -LiteralPath $global:ConvertedSNWReport
$global:hash = #{}
Import-Csv -LiteralPath $global:ConvertedSNWReport | ForEach-Object {
$global:hash[$_.'Computer name'] += #($_.Application)
}
$global:Results = $hash.GetEnumerator() | Where-Object {
$_.Value -notmatch '*Snagit*'
}
You can nest your Where-Object statements to accomplish this:
$results = $hash.GetEnumerator() |Where-Object {
-not($_.Value |Where-Object {$_ -like '*snagit*'})
}
If any of the items in the individual value array matches snagit, the expression will evaluate to $false and the hashtable entrt will be skipped. Conversely, if no items match snagit, it will evaluate to $true
As Ansgar just reminded me, the nested Where-Object statement isn't actually necessary when evaluating strings, since -like doubles as a filter operator on collections:
$results = $hash.GetEnumerator() |Where-Object {
-not($_.Value -like '*snagit*')
}
I have about 100 CSVs that I need to filter through, with the desired ouptut being all rows where the 'First Name' and 'Last Name' fields match a list that I have. Unfortunately, since the list I'm searching from was made separately and by hand there is no unique identifier or key beyond the combination of first and last name (I have street addresses but they are written in a different format than the CSVs).
What I've tried:
$myinput = Get-Content 'C:\folder\names.txt'
Import-Csv 'C:\folder\mycsv1.csv' | Where {
$myimput -match $_.'Last Name' -and
$myinput -match $_.'First_Name'
} | Out-File 'C:\folder\results.txt'
I get a blank .txt. When I tried again without the Out-File I got no output in PowerShell.
Also, If there is a way to automate this with all 100 CSVs, that would be VERY helpful. Any recommendations?
you have an error into imput, try Something like this for search into all csv (if all csv have same columns) :
$ListNames=get-content "c:\temp2\names.txt"
$dircsv="c:\temp2\"
$PathResultFile="c:\temp\result.csv"
gci $dircsv -file -filter "*.csv" | %{
import-csv $_.FullName | where {$csvrow=$_; ($ListNames | where {$_ -match $csvrow.'Last Name' -and $_ -match $csvrow.'First_Name'}).count -gt 0 }
} | export-csv $PathResultFile -NoType
I am converting code that currently parse a large csv file once for each agency. Below is what one line looks like:
Import-Csv $HostList | Where-Object {$_."IP Address" -Match "^192.168.532.*" -or $_Domain -eq "MYDOMAIN"`
-and (get-date $_.discovery_timestamp) -gt $PreviousDays} | select-object Hostname","Domain","IP Address","discovery_timestamp","some_version" | `
Export-Csv -NoTypeInformation -Path $out_data"\OBS_"$Days"_days.txt"
write-host "OBS_DONE"
I have about 30 of these.
I want to parse the csv file once, possibly using foreach and import.csv.
I thought I could do something like:
$HostFile = Import-csv .\HostList.csv
foreach ($line in $HostFile)
{
Where-Object {$_."IP Address" -Match "^172.31.52.*"}
write-host $line
#| Export-Csv -NoTypeInformation -Path H:\Case_Scripts\test.csv
I've tried many permutations of the above, and it never is matching on the "Where-Object" like it does on the example functioning script above.
Any guidance and learning opportunities are appreciated.
My good man, you need to be introduced to the Switch cmdlet. This will sort for all of your companies at once.
$CompanyA = #()
$CompanyB = #()
$CompanyD = #()
$Unknown = #()
Switch(Import-CSV .\HostList.csv){
{(($_."IP Address" -match "^192.168.532.*") -or ($_Domain -eq "CompanyA.com")) -and ((get-date $_.discovery_timestamp) -gt $PreviousDays)} {$CompanyA += $_; Continue}
{(($_."IP Address" -match "^192.26.19.*") -or ($_Domain -eq "CompanyB.net")) -and ((get-date $_.discovery_timestamp) -gt $PreviousDays)} {$CompanyB += $_; Continue}
{(($_."IP Address" -match "^94.8.222.*") -or ($_Domain -eq "CompanyC.org")) -and ((get-date $_.discovery_timestamp) -gt $PreviousDays)} {$CompanyC += $_; Continue}
default {$Unknown += $_}
}
$CompanyA | Export-Csv $out_data"\CompanyA"$Days"_days.txt" -NoType
$CompanyB | Export-Csv $out_data"\CompanyB"$Days"_days.txt" -NoType
$CompanyC | Export-Csv $out_data"\CompanyC"$Days"_days.txt" -NoType
If($Unknown.count -gt 0){Write-Host $Unknown.count + " Entries Did Not Match Any Company" -Fore Red
$Unknown}
That will import the CSV, and for each entry try to match it against the criteria for each of the three lines. If it matches the criteria in the first ScriptBlock, it will perform the action in the second ScriptBlock (add that entry to one of the three arrays I created first). Then it outputs each array to it's own text file in CSV format as you had done in your script. The ;Continue just makes it so it stops trying to match once it finds a valid match, and continue's to the next record. If it can't match any of them it will default to adding it to the Unknown array, and at the end it checks if there are any in there it warns the host and lists the unmatched entries.
Edit: Switch's -RegEx argument... Ok, so the purpose of that is if you want a simple regex match such as:
Switch -regex ($ArrayOfData){
".*#.+?\..{2,5}" {"It's an email address"}
"\d{3}(?:\)|\.|-)\d{3}(?:\.|-)\d{4}" {"It's a phone number"}
"\d{3}-\d{2}-\d{4}" {"Probably a social security number"}
}
What this doesn't allow for is -and, or -or statements. If you use a scriptblock to match against (like I did in my top example) you can use the -match operator, which performs a regex match by default, so you can still use regex without having to use the -regex argument for Switch.
Where-Object should have a collection of objects passed to it. In your second script block this isn't happening, so in the filter the $_ variable will be empty.
There are a couple of ways to fix this - the first one that comes to mind is to replace Where-Object with an 'if' statement. For example
if ($line."IP Address" -Match "^172.31.52.*") { write-output $line }