Powershell script to return search results from a list of keywords - powershell

I have a project name called 'SFO104' and I have a list of serial numbers i.e 5011849, 5011850 etc and I have to search a long list of 500+ serial numbers to see if they exist in any other documents not relating to the project name SFO104 or the PO number 114786.
I was thinking of outputting the search results to a csv for each serial number searched but the below isnt working.
$searchWords = gc C:\Users\david.craven\Documents\list.txt
$results = #()
Foreach ($sw in $searchWords)
{
$files = gci -path C:\Users\david.craven\Dropbox\ -filter "*$sw*" -recurse | select FullName
foreach ($file in $files)
{
$object = New-Object System.Object
$object | Add-Member -Type NoteProperty –Name SearchWord –Value $sw
$object | Add-Member -Type NoteProperty –Name FoundFile –Value $file
$results += $object
}
}
$results | Export-Csv C:\Users\david.craven\Documents\results.csv -NoTypeInformation
The image below shows my search of the serial number 5011849 and the results returned correspond to project SFO104 which is as expected.

Your code works, the file is getting populated. However, what you have specified does not have the headers defined as in your screen shot. Also, what does that list.txt look like. My searchlist.txt is a single column file:
Hello
client
Using your code as is, only changing the file path and name, and a slight modification to where the filename is accessed, gives these results...
$searchWords = gc 'D:\Scripts\searchlist.txt'
$results = #()
Foreach ($sw in $searchWords)
{
$files = gci -path d:\temp -filter "*$sw*" -recurse
foreach ($file in $files)
{
$object = New-Object System.Object
$object | Add-Member -Type NoteProperty –Name SearchWord –Value $sw
$object | Add-Member -Type NoteProperty –Name FoundFile –Value $file.FullName
$results += $object
}
}
$results | Export-Csv d:\temp\searchresults.csv -NoTypeInformation
# Results
# psEdit -filenames 'd:\temp\searchresults.csv'
SearchWord FoundFile
---------- ---------
Hello D:\temp\Duplicates\PowerShellOutput.txt
Hello D:\temp\Duplicates\BeforeRename1\PowerShellOutput.txt
Hello D:\temp\Duplicates\PoSH\PowerShellOutput.txt
Hello D:\temp\Duplicates\Text\PowerShellOutput.txt
client D:\temp\Client.txt
client D:\temp\Duplicates\CertLabClients_v1.ps1
client D:\temp\Duplicates\Check Logon Server for Client.ps1
client D:\temp\Duplicates\Create Wireless Hosted Networks in Windows Clients.ps1
...
Update for OP
Since you are using a comma separate list. You need to break that into separate items. I changed my file to this
Hello,client
You cannot match on that layout unless you are trying to match the whole consecutive string. So, if I break the above this way ...
$searchWords = (gc 'D:\Scripts\searchlist.txt') -split ','
… thus the results are as shown before.
Update for the OP
Example, test with this (a different rough approach)...
Foreach ($sw in $searchWords)
{
Get-Childitem -Path "d:\temp" -Recurse -include "*.txt","*.csv" |
Select-String -Pattern "$sw" |
Select Path,LineNumber,#{n='SearchWord';e={$sw}}
}
The LineNumber was sonly added so show where the string was located. Also, note, your code, and what I provide here, will only work for text, csv files.
If you plan to hit these, doc, docx, xls, xlsx, that means way more code as you have to use the default apps Word, Excel, to open and read these files.
This means using the COM Object model for each of those file types in your code. As discussed and shown here:
How do I make powershell search a Word document for wildcards and return the word it found?
You'd need to do a similar thing for Excel or PowerPoint, and if you have PDF, that requires and addon.
Update for OP
Like I said, I put this together quickly so it is a bit rough (no error handling, etc...) by I did test it using my input file and target folder tree and it does work.
# This is what my input looks like
Hello,client
595959, 464646
LIC
Running the code should have given you the results below, using only .txt,.csv files. Using any other file type will error by design as per my comment above regarding, you cannot use this approach for non text-based files without using the native app for the non text file type.
$searchWords = ((gc 'D:\Scripts\searchlist.txt') -split ',').Trim()
Foreach ($sw in $searchWords)
{
Get-Childitem -Path "d:\temp" -Recurse -include "*.txt","*.csv" |
Select-String -Pattern "$sw" |
Select Path,LineNumber,#{n='SearchWord';e={$sw}}
}
Path LineNumber SearchWord
---- ---------- ----------
D:\temp\Duplicates\BeforeRename1\PsGet.txt 157 Hello
...
D:\temp\Duplicates\PoSH\PsGet.txt 157 Hello
...
D:\temp\Duplicates\BeforeRename1\PoSH-Get-Mo... 108 client
D:\temp\Duplicates\BeforeRename1\Powershell ... 12 client
D:\temp\Duplicates\BeforeRename1\Powershell ... 15 client
D:\temp\Duplicates\BeforeRename1\PsGet.txt 454 client
...
D:\temp\newfile.txt 4 client
D:\temp\MyFile.txt 5 595959
D:\temp\ProcessNames.csv 4 595959
D:\temp\Duplicates\Text\JSON-CSS.txt 30 464646
D:\temp\Duplicates\JSON-CSS.txt 30 464646
D:\temp\MyFile.txt 5 464646
D:\temp\ProcessNames.csv 4 464646
D:\temp\Duplicates\BeforeRename1\GetSetScree... 7 LIC

Related

Why Powershell outputting this table?

I'm a powershell noob. How come the following code is also outputing the table at the end after the "File to Delete" loop?
$stopwatch = [System.Diagnostics.Stopwatch]::StartNew()
# use partial hashes for files larger than 100KB:
# see documentation at: https://powershell.one/tricks/filesystem/finding-duplicate-files#finding-duplicate-files-fast
$result = Find-PSOneDuplicateFileFast -Path '\\READYNAS\Pictures\2020\10' #-Debug -Verbose
$stopwatch.Stop()
# output duplicates
$allFilesToDelete = #(foreach($key in $result.Keys)
{
#filters out the LAST item in the array of duplicates, because a file name of xxxx (0) comes before one without the (0)
$filesToDelete = $result[$key][0..($result[$key].count - 2)]
#add each remaining duplicate file to table
foreach($file in $filesToDelete)
{
$file |
Add-Member -MemberType NoteProperty -Name Hash -Value $key -PassThru |
Select-Object Hash, Length, FullName
}
}
)
$allFilesToDelete | Format-Table -GroupBy Hash -Property FullName | Out-String | Write-Host
$allFilesToDelete | Sort-Object -Property FullName -OutVariable allFilesToDelete
$allFilesToDelete | Format-Table -Property FullName | Out-String | Write-Host
$confirmation = Read-Host "Are you Sure You Want To Delete $($allFilesToDelete.count) files? (y/n)"
if ($confirmation -eq 'y') {
$i = 0
foreach($fileToDelete in $allFilesToDelete)
{
$i++
Write-Host "$i File to Delete: $($fileToDelete.FullName)"
#Remove-Item $file.FullName -Force -Verbose 4>&1 | % { $x = $_; Write-Host "Deleted file ($i) $x" }
}
} else {
Write-Host "User chose NOT to delete files!"
}
$allFilesToDelete | Sort-Object -Property FullName -OutVariable allFilesToDelete produces output (the input objects in the requested sort order), and since you're not capturing or redirecting it, it prints to the host (display, terminal) by default.
It seems your intent is to sort the objects stored in $allFilesToDelete, which your command does, but it also produces output (the common -OutVariable parameter does not affect a cmdlet's output behavior, it simply also stores the output objects in the given variable); you could simply assign the output back to the original variable, which wouldn't produce any output:
$allFilesToDelete = $allFilesToDelete | Sort-Object -Property FullName
In cases where actively suppressing (discarding) output is needed, $null = ... is the simplest solution:
See this answer for details and alternatives.
Also see this blog post, which you found yourself.
Because the output resulted in implicitly Format-Table-formatted display representations (for custom objects that have no predefined formatting data), the subsequent Read-Host and Write-Host statements - surprisingly - printed first.
The reason is that this implicit use of Format-Table results in asynchronous behavior: output objects are collected for 300 msecs. in an effort to determine suitable column widths, and during that period output to other output streams may print.
The - suboptimal - workaround is to force pipeline output to print synchronously to the host (display), using Out-Host.
See this answer for details.

Power shell For Loop not Looping

So the output works fine but I'm having an issue with it only outputing the last line it runs. Is there anyway to check for loops to test in the future?
but i have a list of ip address and im trying to check if the firewall in windows is enabled or disabled.
They are on one LARGE (300+ workgroup). Any help in getting this to loop properly would be appreciated. Security and other things are not a concern cause i have other scripts that run fine. And i dont get any errors. just the single output.
ive already tried moving the array and that didn't help. im thinking it could be the PSCustomObject part as i'm just starting to learn these. Or could it be my input and output formats are different and that's causing issues??
clear
$ComputerList = get-content C:\Users\Administrator\Desktop\DavidsScripts\TurnOffFirewall\input.txt
$Status = #(
foreach ($Computer in $ComputerList) {
netsh -r $Computer advfirewall show currentprofile state})[3] -replace 'State' -replace '\s'
$Object = [PSCustomObject]#{
Computer = $Computer
Firewall = $Status
}
Write-Output $Object
$Object | Export-Csv -Path "C:\FirewallStatus.csv" -Append -NoTypeInformation
Your previous code was not escaping the loop and was only adding the last computer in the loop to the object.
The best way I have found, is to make a temp object and add it to an array list then export that. Much nicer.
$ComputerList = get-content C:\Users\Administrator\Desktop\DavidsScripts\TurnOffFirewall\input.txt
$collectionVariable = New-Object System.Collections.ArrayList
ForEach ($Computer in $ComputerList) {
# Create temp object
$temp = New-Object System.Object
# Add members to temp object
$temp | Add-Member -MemberType NoteProperty -Name "Computer" -Value $Computer
$temp | Add-Member -MemberType NoteProperty -Name "Firewall" -Value $((netsh -r $Computer advfirewall show currentprofile state)[3] -replace 'State' -replace '\s')
# Add the temp object to ArrayList
$collectionVariable.Add($temp)
}
Write-Output $collectionVariable
$collectionVariable | Export-Csv -Path "C:\FirewallStatus.csv" -Append -NoTypeInformation
Here's a streamlined, functional version of your code, using a single pipeline:
Get-Content C:\Users\Administrator\Desktop\DavidsScripts\TurnOffFirewall\input.txt |
ForEach-Object {
[pscustomobject] #{
Computer = $_
Firewall = (-split ((netsh -r $_ advfirewall show currentprofile state) -match '^State'))[-1]
}
} | Export-Csv -Path C:\FirewallStatus.csv -NoTypeInformation
Note:
No intermediate variables are needed; each computer name read from the input file is processed one by one, and each custom object constructed based on it is sent to the output CSV file.
The command for extracting the firewall status from netsh's output was made more robust in order to extract the state information based on the line content (regex ^State, i.e., a line starting with State) rather than a line index ([3]); the unary form of -split splits the line of interest into tokens by whitespace, and index [-1] extracts the last token, which is the state value.
As for what you tried:
Your foreach loop ended before $Object was constructed, so you ended up constructing just 1 object to send to the output file with Export-Csv.
If you had formatted your code properly, that fact would have been more obvious; try using Visual Studio Code with the PowerShell extension, which offers automatic formatting via the >Format Document (Shift+Alt+F) command.

Need to extract 8 & 9 digit file numbers from 40,000 emails using PowerShell

I am attempting to extract 8 & 9 digit file numbers from 40,000 emails that have been saved as .txt files. The File numbers can appear any where in the email(s)...(it's not a standard form), but should always be 8 or 9 digits in length. The file numbers can also be formatted several different ways Like: xxx xx xxxx, xxx-xx-xxxx, xxxxxxxxx, 8 digit #'s: YY YYY YYY, YY-YYY-YYY, YYYYYYYY. I created a PowerShell script that reads the text file extracts the file numbers matching the said pattern and creates & saves them to a .csv file.
Problems: If there is any text proceeding the file# on the line, the script fails to grab the file #. It also grabs additional text (on the same line after the File #). I need only exact matches to the set patterns.
Solution does not need to be in PowerShell, If there is a better solution in vbscript I'm also open to that.
current script is below:
$Num = #()
$Num += Select-String -Path "$PSSCRIPTROOT\text.txt" -Pattern '\d{8}$|^\d{2}\s\d{3}\s\d{3}$|^\d{2}-\d{3}-\d{3}$'
$Num += Select-String -Path "$PSSCRIPTROOT\text.txt" -Pattern '\d{9}$|^\d{3}\s\d{2}\s\d{4}$|^\d{3}-\d{2}-\d{4}$'
ForEach ($Matches in $Num){
$Found = $Matches.ToString().Split(":")
$o = new-object PSObject
$o | add-member NoteProperty "FoundOnLine" $Found[2]
$o | add-member NoteProperty "Number" $Found[3]
$o | export-csv "$PSscriptroot\FoundNumbers.csv" -notypeinformation -Append
Write-Output $o
PLEASE HELP!
This should do the trick actually ...
$File = "$PSSCRIPTROOT\text.txt"
$Pattern = '\d\d(\s|-)*\d(\s|-)*\d(\s|-)*\d{4,5}'
Select-String -Path $File -Pattern $Pattern -AllMatches |
Select-Object -ExpandProperty Matches |
Select-Object -ExpandProperty Value

Add header row to .csv

I realize this is a very commonly asked question, however none of the solutions that have worked in the questions I have found have worked for me. I have a powershell script that creates a .csv file (using new-item) and then populates it with four columns of information (using add-content). I would like to add to this script to add a header to the .csv so I can further parse it and separate the information by the contents of one of the columns, however nothing I have tried is working. I have tried using the -value parameter to make the first row of the csv the headers I want, which is working to put the header in the correct columns, however the first row of information that is being added through add-content is going into the header row.
Example:
Desired output:
Header | Header | Header | Header
Info | Info | Info | Info
Info | Info | Info | Info
Actual Output:
Header | Header | Header | Header | Info | Info | Info | Info |
Info | Info | Info | Info |
I have also tried using import-csv and adding a header that way, then piping the output to export-csv, however this has been returning a blank .csv each time. Code I have used for that:
Import-Csv $file -header "Header,Header,Header,Header" |export-csv $file -notypeinformation
Way I see it I need to skip the first row when doing add-content, however I'm quite new to powershell and don't know quite how to do that. I've been going in circles on this for a while now, so any input would be much appreciated.
** EDIT **
Answer by TheMadTechnician fixed the issue, pasting currently working code per second half of their answer. It is a relatively simple script to parse a csv, check if there is a tracking number, if there is check against ups website, if delivered deposit in csv, if delivery exception deposit in other csv:
$file= import-csv "**FILEPATH**\Intransit_report.csv"
$newfile= New-Item "**FILEPATH**\$(get-date -format dd-MM-yyyy)_delivered.csv" -type file -force -value "Tag Number,Final Dest,Tracking ID,Attribute Ref #`n"
$exceptionfile= New-Item "**FILEPATH**\$(get-date -format dd-MM-yyyy)_delivery_exceptions.csv" -type file -force -value "Tag Number,Final Dest,Tracking ID,Attribute Ref #`n"
foreach ($i in $file) {
$tagnum= $i | select-object -expandproperty "Tag Number"
$trackingid = $i | select-object -expandproperty "Tracking ID"
$refnum= $i | select-object -expandproperty "Attribute Ref #"
$findest= $i | select-object -expandproperty "Final Dest"
if ($trackingid.length -gt 1){
$uri= "**URI**$trackingid"
$html= Invoke-Webrequest -URI $uri
$fields= $HTML.ParsedHtml
$trackstate= $fields.getElementByID("ttc_tt_spStatus")
if($trackstate.innerText -like "*Business Days*"){
break}
elseif($trackstate.innerText -like "Delivered*"){
"$tagnum, $findest, $trackingid, $refnum" | Add-Content $newfile
}
elseif($trackstate.innerText -like "*Attempt Made*"){
"$tagnum, $findest, $trackingid, $refnum" | Add-Content $exceptionfile
}
}
}
I've got an easy answer, and a good answer. The easy answer is to add `n to your -value parameter.
New-Item C:\Path\To\File.csv -value "Header,Header,Header,Header`n"
That just appends the New Line character to the end of your header text, and anything added after that goes on the next line.
The better answer is that you're probably going about things the hard way. Rather than adding content to a file several times I would recommend collecting the data (unless there is a huge amount), and then outputting it all at once at the end. If you're looking at making a CSV you should make an array of objects, and then just export those objects at the end. Without more data on what you're doing it's hard to give a viable example that you can relate to. Let me know if you want more info on this, and update your question with some more code showing what you're doing.
Edit: Looking at what you have, I think I'd just add a status property to each item, and then export once at the end for each file filtering on the status. Something like this:
$file= import-csv "**FILEPATH**\Intransit_report.csv"
foreach ($i in ($file | Where{$_.'Tracking ID'})) {
$trackingid = $i | select-object -expandproperty "Tracking ID"
$uri= "**URI**$trackingid"
$html= Invoke-Webrequest -URI $uri
$fields= $HTML.ParsedHtml
$trackstate= $fields.getElementByID("ttc_tt_spStatus")
Switch -Regex ($trackstate.innerText){
"Business Days" {Add-Member -InputObject $i -NotePropertyName 'Status' -NotePropertyValue 'In Transit';Continue}
"Delivered" {Add-Member -InputObject $i -NotePropertyName 'Status' -NotePropertyValue 'Delivered';Continue}
"Attempt Made" {Add-Member -InputObject $i -NotePropertyName 'Status' -NotePropertyValue 'Attempt Made'}
}
}
$file | Where{$_.Status -eq 'Delivered'} | Select 'Tag Number','Final Dest','Tracking ID','Attribute Ref #' | Export-CSV "**FILEPATH**\$(get-date -format dd-MM-yyyy)_delivered.csv" -NoTypeInformation
$file | Where{$_.Status -eq 'Attempt Made'} | Select 'Tag Number','Final Dest','Tracking ID','Attribute Ref #' | Export-CSV "**FILEPATH**\$(get-date -format dd-MM-yyyy)_delivery_exceptions.csv" -NoTypeInformation
This way it's doing all of the processing at once, and then writting twice, instead of writing to the drive X number of times, where X is the number of items that are either already delivered or have had delivery attempts made.

Variable referencing. How to create arrays getting their names from elements of another array

This is as simplified version of what I'd like to achieve... I think it's called 'variable referencing'
I have created an array containing the content of the folder 'foo'
$myDirectory(folder1, folder2)
Using the following code:
$myDirectory= Get-ChildItem ".\foo" | ForEach-Object {$_.BaseName}
I'd like to create 2 arrays named as each folders, with the contained files.
folder1(file1, file2)
folder2(file1, file2, file3)
I tried the following code:
foreach ($myFolder in $myDirectory) {
${myFolder} = Get-ChildItem ".\$myFolders" | forEach-Object {$_.BaseName}
}
But obviously didn't work.
In bash it's possible create an array giving it a variable's name like this:
"${myForder[#]}"
I tried to search on Google but I couldn't find how to do this in Powershell
$myDirectory = "c:\temp"
Get-ChildItem $myDirectory | Where-Object{$_.PSIsContainer} | ForEach-Object{
Remove-Variable -Name $_.BaseName
New-Variable -Name $_.BaseName -Value (Get-ChildItem $_.FullName | Where-Object{!$_.PSIsContainer} | Select -ExpandProperty Name)
}
I think what you are looking for is New-Variable. Cycle through all the folders under C:\temp. For each folder make a new variable. It would throw errors if the variable already exists. What you could do for that is remove a pre-exising variable. Populate the variable with the current folders contents in the pipeline using Get-ChildItem. The following is a small explanation of how the -Value of the new variable is generated. Caveat Remove-Variable has the potiential to delete unintended variables depending on your folder names. Not sure of the implications of that.
Get-ChildItem $_.FullName | Where-Object{!$_.PSIsContainer} | Select -ExpandProperty Name
The value of each custom variable is every file ( not folder ). Use -ExpandProperty to just gets the names as strings as supposed to a object with Names.
Aside
What do you plan on using this data for? It might just be easier to pipe the output from the Get-ChildItem into another cmdlet. Or perhaps create a custom object with the data you desire.
Update from comments
$myDirectory = "c:\temp"
Get-ChildItem $myDirectory | Where-Object{$_.PSIsContainer} | ForEach-Object{
[PSCustomObject] #{
Hotel = $_.BaseName
Rooms = (Get-ChildItem $_.FullName | Where-Object{!$_.PSIsContainer} | Select -ExpandProperty Name)
}
}
You need to have at least PowerShell 3.0 for the above to work. Changing it for 2.0 is easy if need be. Create and object with hotel names and "rooms" which are the file names from inside the folder. If you dont want the extension just use BaseName instead of Name in the select.
This is how I did it at the end:
# Create an array containing all the folder names
$ToursArray = Get-ChildItem -Directory '.\.src\panos' | Foreach-Object {$_.Name}
# For each folder...
$ToursArray | ForEach-Object {
# Remove any variable named as the folder's name. Check if it exists first to avoid errors
if(Test-Path variable:$_.BaseName){ Remove-Variable -Name $_.BaseName }
$SceneName=Get-ChildItem ".\.src\panos\$_\*.jpg"
# Create an array using the main folder's name, containing the names of all the jpg inside
New-Variable -Name $_ -Value ($SceneName | Select -ExpandProperty BaseName)
}
And here it goes some code to check the content of all the arrays:
# Print Tours information
Write-Verbose "Virtual tours list: ($($ToursArray.count))"
$ToursArray | ForEach-Object {
Write-Verbose " Name: $_"
Write-Verbose " Scenes: $($(Get-Variable $_).Value)"
}
Output:
VERBOSE: Name: tour1
VERBOSE: Scenes: scene1 scene2
VERBOSE: Name: tour2
VERBOSE: Scenes: scene1