Powershell - search through CSV and return only rows with specific error - powershell

I'm trying to write a powershell script that returns only the rows that have a specific entry in one of the columns but struggling a bit.
The CSV has 2 relevant columns, the ID and the Error.
At the moment I have this:
$ImportFile = Import-csv X:\errorlist.csv
$Error = $userobject.Error
$ID = $userobject.ID
$ErrorMessage1 = "Error1"
foreach($test in $ImportFile)
{
$field1 = $test.Error
$field2 = $test.ID
Echo "$field1, $field2"
} Where-Object {$_.Error -like $ErrorMessage1}
However the echo still returns everything in the csv even with the Where-Object.
I'm trying to run a ForEach because ultimately where I have the echo I'm going to insert a SQL script that uses the ID information in the csv to fix specific errors. Therefore I think I need to keep the foreach to process each sql script in turn, unless anyone has a better idea?

Try this way, this will check for $ErrorMessage1 in the ID or Error columns:
Import-csv X:\errorlist.csv |
Where-Object {$_.Error -like "*$ErrorMessage1*" -or $_.ID -like "*$ErrorMessage1*"}

Filter the rows from the CSV before you run the SQL:
Import-Csv X:\errorlist.csv |
Where-Object { $_.Error -like $ErrorMessage } |
ForEach-Object {
# Don't use aliases in scripts, use the real cmdlet names. Makes maintenance easier.
Write-Host ('{0}, {1}' -f $_.Error,$_.ID)
# Run your SQL cmd here.
}

The foreach loop operates independent of the Where-object, so you all you're doing is dumping the contents of the CSV file to the console.
You can shorten this up a lot by just pumping the data through the pipeline. This will filter your CSV records down to just the ones that meet your criteria.
$ErrorMessage1 = "Error1"
$allerrors = Import-csv X:\errorlist.csv|where-object {$_.error -eq $ErrorMessage1}
$allerrors;

Related

How to speed up the process in PowerShell

I've a simple script that generate orphan OneDrive report. It's working but it's very slow so I'm just wondering how could I modify my script to speed up the process. Any help or suggestion would be really appreciated.
Basically this is what I'm doing:
Get the owner email from "Owner" column and check it using AzureAD to see if I get any error
If I get an error then check it in on-prem ADGroup
If that owner is existed in the on-prem ADGroup then it's an orphan
Export only that user to a new csv file
$ImportData = "E:\scripts\AllOneDriveUser.csv"
$Report = "E:\scripts\OrphanOneDrive.csv"
$CSVImport = Import-CSV $ImportData
ForEach ($CSVLine in $CSVImport) {
$CSVOwner = $CSVLine.Owner
try{
Get-AzureADUser -ObjectId $CSVOwner
}catch{
$StatusMessage = $_.Exception.Message
if($Null -eq $StatusMessage){
Write-Host "User Found, Ignore from orphan list."
}else{
#Owner not found in AzureAD
$group = 'TargetGroup'
$filter = '(memberof={0})' -f (Get-ADGroup $group).DistinguishedName
$filterName = Get-ADUser -LDAPFilter $filter
$ModifiedOwner = $CSVOwner -split"#"[0]
if( $ModifiedOwner[0] -in $filterName.Name ){
Write-host "Adding it into orphaned list"
$CSVLine | Export-Csv $Report -Append -notypeinformation -force
}else{
Write-Host "Not orphaned"
}
}
}
}
I have over 8000 record in my import csv file and over 5000 member in my on-prem AD group so it taking very long.
You can greatly improve your script by using a HashSet<T> in this case, but also the main issue of your code is that you're querying the same group over and over, it should be outside the loop!
There is also the use of Export-Csv -Append, appending to a file per loop iteration is very slow, better to streamline the process with the pipelines so Export-Csv receives the objects and exports only once instead of opening and closing the FileStream everytime.
Hope the inline comments explain the logic you can follow to improve it.
$ImportData = "E:\scripts\AllOneDriveUser.csv"
$Report = "E:\scripts\OrphanOneDrive.csv"
# we only need to query this once! outside the try \ catch
# a HashSet<T> enables for faster lookups,
# much faster than `-in` or `-contains`
$filter = '(memberof={0})' -f (Get-ADGroup 'GroupName').DistinguishedName
$members = [Collections.Generic.HashSet[string]]::new(
[string[]] (Get-ADUser -LDAPFilter $filter).UserPrincipalName,
[System.StringComparer]::OrdinalIgnoreCase
)
Import-CSV $ImportData | ForEach-Object {
# hitting multiple times a `catch` block is expensive,
# better use `-Filter` here and an `if` condition
$CSVOwner = $_.Owner
if(Get-AzureADUser -Filter "userprincipalname eq '$CSVOwner'") {
# we know this user exists in Azure, so go next user
return
}
# here is for user not found in Azure
# no need to split the UserPrincipalName, HashSet already has
# a unique list of UserPrincipalNames
if($hash.Contains($CSVOwner)) {
# here is if the UPN exists as member of AD Group
# so output this line
$_
}
} | Export-Csv $Report -NoTypeInformation

Script to scan computers from a list and identify which ones have a software title installed

I have narrowed down a little more what exactly my end game is.
I have pre-created the file that I want the results to write to.
Here is a rough script of what I want to do:
$computers = Get-content "C:\users\nicholas.j.nedrow\desktop\scripts\lists\ComputerList.txt"
# Ping all computers in ComputerList.txt.
# Need WinEst_Computers.csv file created with 3 Columns; Computer Name | Online (Y/N) | Does File Exist (Y/N)
$output = foreach ($comp in $computers) {
$TestConn = Test-connection -cn $comp -BufferSize 16 -Count 1 -ea 0 -quiet
if ($TestConn -match "False")
{
#Write "N" to "Online (Y/N)" Column in primary output .csv file
}
if ($TestConn -match "True")
{
#Write "Y" to "Online (Y/N)" Column in primary output .csv file
}
#For computers that return a "True" ping value:
#Search for WinEst.exe application on C:\
Get-ChildItem -Path "\\$comp\c$\program files (x86)\WinEst.exe" -ErrorAction SilentlyContinue |
if ("\\$comp\c$\program files (x86)\WinEst.exe" -match "False")
{
#Write "N" to "Does File Exist (Y/N)" Column in primary output .csv file
}
if ("\\$comp\c$\program files (x86)\WinEst.exe" -match "True")
{
#Write "Y" to "Does File Exist (Y/N)" Column in primary output .csv file
}
Select #{n='ComputerName';e={$comp}},Name
}
$output | Out-file "C:\users\nicholas.j.nedrow\desktop\scripts\results\CSV Files\WinEst_Computers.csv"
What I need help with is the following:
How to get each result to either write to the appropriate line (I.e. computername, online, file exist?) or would it be easier to do one column at a time;
--Write all PC's to Column A
--Ping each machine and record results in Column B
--Search each machine for the .exe and record results.
Any suggestions? Sorry I keep changing things. Just trying to figure out the best way to do this.
You are using the foreach command, which has a syntax foreach ($itemVariable in $collectionVariable) { }. If $computer is your collection, then your current item cannot also be $computer inside your foreach.
Get-Item does not return a property computerName. Therefore you cannot explicitly select it with Select-Object. However, you can use a calculated property to add a new property to the custom object that Select-Object outputs.
If your CSV file has a row of header(s), it is simpler to use Import-Csv to read the file. If it is just a list of computer names, then Get-Content works well.
If you are searching for a single file and you know the exact path, then just stick with -Path or -LiteralPath and forget -Include. -Include is not intuitive and isn't explained well in the online documentation.
If you are piping output to Export-Csv using a single pipeline, there's no need for -Append unless you already have an existing CSV with data you want to retain. However, if you choose to pipe to Export-Csv during each loop iteration, -Append would be necessary to retain the output.
Here is some updated code using the recommendations:
$computers = Get-content "C:\users\nicholas.j.nedrow\desktop\scripts\lists\ComputerList.txt"
$output = foreach ($comp in $computers) {
Get-Item -Path "\\$comp\c$\program files (x86)\WinEst.exe" -ErrorAction SilentlyContinue |
Select #{n='ComputerName';e={$comp}},Name
}
$output | Export-Csv -Path "C:\users\nicholas.j.nedrow\desktop\scripts\results\CSV Files\WinEst_Computers.csv" -NoType

powershell use CSV to create variables

I am a total novice when it comes to powershell. I would like to create a little script to save me a lot of time and after a little research, I am sure it can be achieved using Import-CSV command.
Basically I need to run a command on multiple PC's but the variable is different for each command. I wish to pull that variable from a comparision in a CSV file. So find current Hostname, then use that hostname to find the corresponding asset number in the CSV and then use that asset number as a variable in the final comamnd.
Looking at other examples on here, I have this so far:
$Asset = #()
$Host = #()
Import-Csv -Path "C:\hostnametoasset.csv" |`
ForEach-Object {
$Asset += $_.Asset
$Host += $_.Host
}
$Hostname = (Get-WmiObject -Class Win32_ComputerSystem -Property Name).Name
if ($Host -contains $Hostname)
{
C:\BiosConfigUtility64.exe /setvalue:"Asset Tracking Number","$Asset"
}
Section of the CSV:
Asset,Host
10756,PCD001
10324,PCD002
10620,PCD003
Any help would be greatly appreciated.
Couple of different points...
Importing a CSV results in an array of objects that you can filter on.
$Lines = Import-Csv -Path "C:\hostnametoasset.csv"
$Line = $Lines | ?{$_.host -match $ENV:COMPUTERNAME}
You can then use the filter results directly by accessing the member you need.
C:\BiosConfigUtility64.exe /setvalue:"Asset Tracking Number","$($Line.Asset)"
NOTE: I cannot test this directly right now so hopefully I got the syntax right.

Powershell Adjusting Column Values across multiple text files

New to Powershell and looking for some help. I have multiple xyz files that are currently displaying a z value with positive up and I need to make all the values negative (or z-positive down). So far my best attempt has been to try to cobble together what I know from other lines of code, but I'm still far from a solution.
$data = Get-ChildItem "C:\Users\dwilson\Desktop\test" -Recurse |
foreach ($item in $data) {$item.'Col 3' = 0 - $item.'Col 3'}
Any help would be greatly appreciated!
Somewhat of an aside, but you're using mixed syntax here.
foreach ($i in $set) { ... }
This is the Foreach statement. It does not accept input from the pipeline, nor will it automatically send output down the pipeline. See also Get-Help about_Foreach.
Get-ChildItem | ForEach-Object { ... }
This is the ForEach-Object command. It accepts input from the pipeline and sends output down the pipeline. Critically, this command also has a default alias of foreach. See also Get-Help ForEach-Object.
The help for the Foreach statement explains how PowerShell decides what foreach means (emphasis mine):
When Foreach appears in a command pipeline, PowerShell uses the foreach alias, which calls the ForEach-Object command. When you use the foreach alias in a command pipeline, you do not include the ($<item> in $<collection>) syntax as you do with the Foreach statement. This is because the prior command in the pipeline provides this information.
Using the looping over the files returned by Get-ChildItem you can import each with Import-CSV using the -Header parameter to assign the property name for each column. Then we can update the information for that property then export it. Using the ConvertTo-CSV cmdlet and then using Select-Object -Skip 1 we can drop the header off the CSV before exporting it.
$Files = Get-ChildItem "C:\Users\dwilson\Desktop\test" -Recurse
foreach ($File in $Files) {
$Data = Import-CSV $File -Header 'Col1','Col2','Col3'
$newData = ForEach ($Row in $Data) {
$Row.'Col3' = 0 - $Row.'Col3'
$Row
}
$newData |
Convertto-CSV -NoTypeInformation |
Select-Object -Skip 1 |
Out-File "$($File.Fullname).new)"
}

Import and modify multiple CSV files in Powershell

Thanks ahead of time. I'm trying to import multiple .csv files, add a column to them, then write to that column based on the value of another column
What I'm trying to do is follow this logic:
Loop through CSVs
For each CSV:
Add a new column
Look for some words.
If you see the words, write a different word in the new column
End loop
End loop
My code is here:
Function getTag ([string]$vuln){
if($vuln -like "adobe-apsb-")
{return "Adobe Flash"}
if($vuln -like "jre-")
{return "Java"}
else
{return "No vulnerability code available"}
}
$in = "D:\Splunk\Inbound"
$out = 'D:\Splunk\Outbound'
Get-ChildItem $in -Filter *.csv |
ForEach-Object{
Import-Csv $_ | Select-Object *,#{Name='Tag';Expression={getTag $_.'Vulnerability ID'}} | Export-Csv $_.Name
}
Right now, $_ is coming through as a string, not a CSV, and I believe that's my problem. Does anyone have a good way to access that csv file from inside the nested loop?
Your problem with using -like is a missing wildcard.
I guess the Outbound folder shall contain the modified csv?
These endless command lines aren't necessary, neither in a
script nor in the shell.
For me this is much better readable (and it works) :
Function getTag ([string]$vuln){
if ($vuln -like "adobe-apsb-*")
{return "Adobe Flash"}
if ($vuln -like "jre-*")
{return "Java"}
else
{return "No vulnerability code available"}
}
$in = "D:\Splunk\Inbound"
$out = "D:\Splunk\Outbound\"
Get-ChildItem $in -Filter *.csv |
ForEach-Object{
Import-Csv $_.FullName |
Select-Object *,#{
Name='Tag';
Expression={getTag $_.'Vulnerability ID'}
} | Export-Csv -path $($out+$_.Name) -notype
}
> tree . /f
│
├───Inbound
│ test.csv
│
└───Outbound
test.csv
> gc .\Inbound\test.csv
"A","Vulnerability ID","C"
0,"adobe-apsb-blah",2
5,"jre-trash",9
> gc .\Outbound\test.csv
"A","Vulnerability ID","C","Tag"
"0","adobe-apsb-blah","2","Adobe Flash"
"5","jre-trash","9","Java"