My script below searches for a specific part number (459279) recursively through a number of txt files.
set-location C:\Users\admin\Desktop\PartNumbers\
$exclude = #('PartNumbers.txt','*.ps1')
$Line = [Environment]::NewLine
Get-ChildItem -Recurse -Exclude $exclude | select-string 459279 | group count | Format-Table -Wrap -AutoSize -Property Count,
#{Expression={$_.Group | foreach { $_.filename} | Foreach {"$_$Line"} }; Name="Filename"},
#{Expression={$_.Group | foreach {$_.LineNumber} | Foreach {"$_$Line"} }; Name="LineNumbers"} | Out-File 459279Results.txt
My Results are:
Count Filename LineNumbers
----- -------- -----------
2 {Customer1.txt {2
, Customer2.txt , 3
} }
My ideal results would be if this is possible:
Part Number: 459279
Count: 2
Filename LineNumbers
-------- -----------
Customer1.txt 2
Customer2.txt 3
I have manually retrieved the part number '459279' from "PartNumbers.txt" and searched for it using the script.
I cannot seem to remove/replace the braces and commas to present a clean list.
What I hope to eventually do is to recursively search through "PartNumbers.txt" and produce a report with each part number appended to the next in the style mentioned above.
PartNumbers.txt is formatted:
895725
939058
163485
459279
498573
Customer*.txt are formatted:
163485
459279
498573
Something like this should work:
$exclude = 'PartNumbers.txt', '*.ps1'
$root = 'C:\Users\admin\Desktop\PartNumbers'
$outfile = Join-Path $root 'loopResults.txt'
Get-Content (Join-Path $root 'PartNumbers.txt') | % {
$partno = $_
$found = Get-ChildItem $root -Recurse -Exclude $exclude `
| ? { -not $_.PSIsContainer } `
| Select-String $partno `
| % {
$a = $_ -split ':'
New-Object -Type PSCustomObject -Property #{
'Filename' = Split-Path -Leaf $a[1];
'LineNumbers' = $a[2]
}
}
"Part Number: $partno"
'Count: ' + #($found).Count
$found | Format-Table Filename, LineNumbers
} | Out-File $outfile -Append
Related
Dear wise Stackoverflow Community
Is it possible with powershell, to get all filenames in a directory and search for them each in another directory?
Something like:
Path A (C:\Temp):
C:\Temp\Test2\Test.docx
C:\Temp\Test\test.txt
C:\Temp\Test\file.html`
and search for each file (Test.docx,test.txt,file.html) in a different path e.g. C:\Filesnew ?
and in the best case, the export to csv would be something like:
| Filename | Filepath | Found? | FoundIn |
| -------- | ------------------------ |--------| ----------------------------------------|
| Test.docx | C:\Temp\Test2\Test.docx | y | C:\Filesnew\TransferedFiles\Test.docx |
| test.txt | C:\Temp\Test\test.txt | n | |
| file.html | C:\Temp\Test\file.html | y | C:\Filesnew\webfiles\file.html |
I'm currently stuck at this code:
$files = gci -r * | select-object name, fullname
foreach ($file in $files)
{
gci -path C:\Filesnew -recurse | where {$_.Name -match $file.Name} | select-object $files.Name $files.fullname, $file.fullname | Export-CSV C:\test.csv
}
Can you help me out?
This requires a bit of logic, ideally the more efficient way to go about this is to have a hashtable with all File Names and their corresponding Absolute Paths excluding the target directory (C:\Filesnew) for fast lookups.
$targetDir = 'C:\Filesnew'
$queue = [Collections.Generic.Queue[IO.DirectoryInfo]]::new()
$queue.Enqueue((Get-Item C:\))
# This logic below builds a map having:
# - Keys as the FileName
# - Values as the corresponding Absolute Paths for each FileName
$map = #{}
while($queue.Count) {
try {
$target = $queue.Dequeue()
foreach($item in $target.EnumerateFileSystemInfos()) {
if($item.FullName.StartsWith($targetDir, $true, $null)) {
# exclude the `$targetDir` and any directory or file in it
continue
}
if($item -is [IO.DirectoryInfo]) {
$queue.Enqueue($item)
continue
}
if(-not $map.ContainsKey($item.Name)) {
$map[$item.Name] = [Collections.Generic.List[string]]::new()
}
$map[$item.Name].Add($item.FullName)
}
}
catch {
# you can add error handling here but most errors will be Access Denied
# can also leave blank here to ignore errors
}
}
# now that we have our map of FileNames and Absolute Paths
# query the target Directory
Get-ChildItem $targetDir -Recurse -File | ForEach-Object {
# construct the desired output
[pscustomobject]#{
FileName = $_.Name
FilePath = $_.Fullname
Found = ('n', 'y')[$map.ContainsKey($_.Name)]
FoundIn = $map[$_.Name] -join ', '
}
# Export the output to CSV
} | Export-Csv C:\test.csv -NoTypeInformation
I have this script I received to check folders and subfolders on a network drive. I wonder how it could be modified into checking only folders and subfolder and write in the CSV if there is any folder with more then 40.000 files in it and the number of files. The image show a sample output from the script as it is now and I do not need it to show any files as it currently do.
$dir = "D:\test"
$results = #()
gci $dir -Recurse -Depth 1 | % {
$temp = [ordered]#{
NAME = $_
SIZE = "{0:N2} MB" -f ((gci $_.Fullname -Recurse | measure -Property Length -Sum -ErrorAction SilentlyContinue).Sum / 1MB)
FILE_COUNT = (gci -File $_.FullName -Recurse | measure | select -ExpandProperty Count)
FOLDER_COUNT = (gci -Directory $_.FullName -Recurse | measure | select -ExpandProperty Count)
DIRECTORY_PATH = $_.Fullname
}
$results += New-Object PSObject -Property $temp
}
$results | export-csv -Path "C:\temp\output.csv" -NoTypeInformation
Instead of executing so many Get-ChildItem cmdlets, here's an approach that uses robocopy to do the heavy lifting of counting the number of files, folders and total sizes:
# set the rootfolder to search
$dir = 'D:\test'
# switches for robocopy
$roboSwitches = '/L','/E','/NJH','/BYTES','/NC','/NP','/NFL','/XJ','/R:0','/W:0','/MT:16'
# regex to parse the output from robocopy
$regEx = '\s*Total\s*Copied\s*Skipped\s*Mismatch\s*FAILED\s*Extras' +
'\s*Dirs\s*:\s*(?<DirCount>\d+)(?:\s+\d+){3}\s+(?<DirFailed>\d+)\s+\d+' +
'\s*Files\s*:\s*(?<FileCount>\d+)(?:\s+\d+){3}\s+(?<FileFailed>\d+)\s+\d+' +
'\s*Bytes\s*:\s*(?<ByteCount>\d+)(?:\s+\d+){3}\s+(?<BytesFailed>\d+)\s+\d+'
# loop through the directories directly under $dir
$result = Get-ChildItem -Path $dir -Directory | ForEach-Object {
$path = $_.FullName # or if you like $_.Name
$summary = (robocopy.exe $_.FullName NULL $roboSwitches | Select-Object -Last 8) -join [Environment]::NewLine
if ($summary -match $regEx) {
$numFiles = [int64] $Matches['FileCount']
if ($numFiles -gt 40000) {
[PsCustomObject]#{
PATH = $path
SIZE = [int64] $Matches['ByteCount']
FILE_COUNT = [int64] $Matches['FileCount']
FOLDER_COUNT = [int64] $Matches['DirCount']
}
}
}
else {
Write-Warning -Message "Path '$path' output from robocopy was in an unexpected format."
}
}
# output on screen
$result | Format-Table -AutoSize
# output to CSV file
$result | Export-Csv -Path "C:\temp\output.csv" -NoTypeInformation
So this script counts how many times words appear in the whole file.
Which works perfect.
Now I need it to show on which lines every word above 4 characters appears.
The problem is that I have almost no experience in scripting.
Credits to AdminOfThings for providing me with the current code!
Function AnalyseTo-Doc
{
param ([Parameter(Mandatory=$true)][string]$Pad )
New-Item C:\destination.txt -ItemType file
$destination = "C:\destination.txt"
$filecontents = Get-Content $Pad -Raw
$words = ($filecontents | Select-String -Pattern "\b[A-Za-z]{4,}\b" -AllMatches).Matches.Value
$words | Group-Object -NoElement | Foreach-Object {
("{0},{1}" -f $_.Count,$_.Name) | Add-Content -Path $destination
}
}
AnalyseTo-Doc
As AnsgarWiechers hinted, Select-String returns a structured object with Matches per Lines.
## Q:\Test\2019\06\11\SO_56543125.ps1
Function AnalyseTo-Doc{
param ([Parameter(Mandatory=$true)][string]$Pad )
$Lines = Select-String -Path $Pad -Pattern '\b[A-Za-z]{4,}\b' -AllMatches
$Words = ForEach($Line in $Lines){
ForEach($Match in $Line.Matches){
[PSCustomObject]#{
LineNumber = $Line.LineNumber
Word = $Match.Value
}
}
}
$Words | Group-Object Word | ForEach-Object {
[PSCustomObject]#{
Count= $_.Count
Word = $_.Name
Line = $_.Group.LineNumber -join ','
}
}
}
AnalyseTo-Doc Question_SO_56543125.txt
With your question text in file Question_SO_56543125.txt the script returns:
> Q:\Test\2019\06\11\SO_56543125.ps1
Count Word Line
----- ---- ----
1 this 1
1 script 1
1 counts 1
1 many 1
1 times 1
1 words 1
1 appear 1
1 whole 1
1 file 1
2 Which 2,3
1 works 2
...snip...
The output can easily be saved in a (csv) file.
The following modifications should achieve what you want.
Function AnalyseTo-Doc
{
param ([Parameter(Mandatory=$true)][string]$Pad )
New-Item C:\destination.txt -ItemType file
$destination = "C:\destination.txt"
$filecontents = Get-Content $Pad
$words = $filecontents | Select-String -Pattern "\b[A-Za-z]{4,}\b" -AllMatches
$group = $words.Matches.Value | Group-Object -NoElement
$output = foreach ($word in $group) {
[pscustomobject]#{Count = $Word.Count
Word = $word.Name
Linenumbers = $words.where{$_.Matches.Value -eq $word.Name}.linenumber
}
}
$output | Foreach-Object {
("{0},{1},{2}" -f $_.Count,$_.Word,($_.Linenumbers -Join " ")) | Add-Content -Path $Destination
}
}
The line numbers are joined by a space at the end of each line in the output file. You can change the join character by updating the -Join " " section.
Below I'll give you an example of how your problem can be solved:
$s = "aaa", "bbb", "ccc"
$findings = $s | select-string "bbb"
$valAndLinenumber = $findings | Select-Object #{ l="Value"; e={ $_.matches.value}}, linenumber, line
$valAndLinenumber
Output:
Value LineNumber Line
----- ---------- ----
bbb 2 bbb
Adaptions to make it work with your script are up to you.
You can find the example under this link.
Further reading:
Select-Object
Calculated property
I have this CSV file that I kind of do a lot to. My most recent task is to add a summary sheet.
With that said I have a CSV file I pull from a website and send through lot of checks. Code Below:
$Dups = import-csv 'C:\Working\cylrpt.csv' | Group-Object -Property 'Device Name'| Where-Object {$_.count -ge 2} | ForEach-Object {$_.Group} | Select #{Name="Device Name"; Expression={$_."Device Name"}},#{Name="MAC"; Expression={$_."Mac Addresses"}},Zones,#{Name="Agent"; Expression={$_."Agent Version"}},#{Name="Status"; Expression={$_."Is online"}}
$Dups | Export-CSV $working\temp\01-Duplicates.csv -NoTypeInformation
$csvtmp = Import-CSV $working\cylrpt.csv | Select #{N='Device';E={$_."Device Name"}},#{N='OS';E={$_."OS Version"}},Zones,#{N='Agent';E={$_."Agent Version"}},#{N='Active';E={$_."Is Online"}},#{N='Checkin';E={[DateTime]$_."Online Date"}},#{N='Checked';E={[DateTime]$_."Offline Date"}},Policy
$csvtmp | %{
if ($_.Zones -eq ""){$_.Zones = "Unzoned"}
}
$csvtmp | Export-Csv $working\cy.csv -NoTypeInformation
import-csv $working\cy.csv | Select Device,policy,OS,Zones,Agent,Active,Checkin,Checked | % {
$_ | Export-CSV -path $working\temp\$($_.Zones).csv -notypeinformation -Append
}
The first check is for duplicates, I used separate lines of code for this because I wanted to create a CSV for duplicates.
The second check backfills all blank cells under the Zones column with "UnZoned"
The third thing is does is goes through the entire CSV file and creates a CSV file for each Zone
So this is my base. I need to add another CSV file for a Summary of the Zone information. The Zones are in the format of XXX-WS or XXX-SRV, where XXX can be between 3 and 17 letters.
I would like the Summary sheet to look like this
ABC ###
ABC-WS ##
ABC-SRV ##
DEF ###
DEF-WS ##
DEF-SRV ##
My thoughts are to either do the count from the original CSV file or to count the number of lines in each CSV file and subtract 1, for the header row.
Now the Zones are dynamic so I can't just say I want ZONE XYZ, because that zone may not exist.
So what I need is to be able to either count the like zone type in the original file and either output that to an array or file, that would be my preferred method to give the number of items with the same zone name. I just don't know how to write it to look for and count matching variables. Here is the code I'm trying to use to get the count:
import-csv C:\Working\cylrpt.csv | Group-Object -Property 'Zones'| ForEach-Object {$_.Group} | Select #{N='Device';E={$_."Device Name"}},Zones | % {
$Znum = ($_.Zones).Count
If ($Znum -eq $null) {
$Znum = 1
} else {
$Znum++
}
}
$Count = ($_.Zones),$Znum | Out-file C:\Working\Temp\test2.csv -Append
Here is the full code minus the report key:
$cylURL = "https://protect.cylance.com/Reports/ThreatDataReportV1/devices/"
$working = "C:\Working"
Remove-item -literalpath "\\?\C:\Working\Cylance Report.xlsx"
Invoke-WebRequest -Uri $cylURL -outfile $working\cylrpt.csv
$Dups = import-csv 'C:\Working\cylrpt.csv' | Group-Object -Property 'Device Name'| Where-Object {$_.count -ge 2} | ForEach-Object {$_.Group} | Select #{Name="Device Name"; Expression={$_."Device Name"}},#{Name="MAC"; Expression={$_."Mac Addresses"}},Zones,#{Name="Agent"; Expression={$_."Agent Version"}},#{Name="Status"; Expression={$_."Is online"}}
$Dups | Export-CSV $working\temp\01-Duplicates.csv -NoTypeInformation
$csvtmp = Import-CSV $working\cylrpt.csv | Select #{N='Device';E={$_."Device Name"}},#{N='OS';E={$_."OS Version"}},Zones,#{N='Agent';E={$_."Agent Version"}},#{N='Active';E={$_."Is Online"}},#{N='Checkin';E={[DateTime]$_."Online Date"}},#{N='Checked';E={[DateTime]$_."Offline Date"}},Policy
$csvtmp | %{
if ($_.Zones -eq ""){$_.Zones = "Unzoned"}
}
$csvtmp | Export-Csv $working\cy.csv -NoTypeInformation
import-csv $working\cy.csv | Select Device,policy,OS,Zones,Agent,Active,Checkin,Checked | % {
$_ | Export-CSV -path $working\temp\$($_.Zones).csv -notypeinformation -Append
}
cd $working\temp;
Rename-Item "Unzoned.csv" -NewName "02-Unzoned.csv"
Rename-Item "Systems-Removal.csv" -NewName "03-Systems-Removal.csv"
$CSVFiles = Get-ChildItem -path $working\temp -filter *.csv
$Excel = "$working\Cylance Report.xlsx"
$Num = $CSVFiles.Count
Write-Host "Found the following Files: ($Num)"
ForEach ($csv in $CSVFiles) {
Write-host "Merging $CSVFiles.Name"
}
$EXc1 = New-Object -ComObject Excel.Application
$Exc1.SheetsInNewWorkBook = $CSVFiles.Count
$XLS = $EXc1.Workbooks.Add()
$Sht = 1
ForEach ($csv in $CSVFiles) {
$Row = 1
$Column = 1
$WorkSHT = $XLS.WorkSheets.Item($Sht)
$WorkSHT.Name = $csv.Name -Replace ".csv",""
$File = (Get-Content $csv)
ForEach ($line in $File) {
$LineContents = $line -split ',(?!\s*\w+")'
ForEach ($Cell in $LineContents) {
$WorkSHT.Cells.Item($Row,$Column) = $Cell -Replace '"',''
$Column++
}
$Column = 1
$Row++
}
$Sht++
}
$Output = $Excel
$XLS.SaveAs($Output)
$EXc1.Quit()
Remove-Item *.csv
cd ..\
Found the solution
$Zcount = import-csv C:\Working\cylrpt.csv | where Zones -ne "$null" | select #{N='Device';E={$_."Device Name"}},Zones | group Zones | Select Name,Count
$Zcount | Export-Csv -path C:\Working\Temp\01-Summary.csv -NoTypeInformation
I have two csv files, each that contain a PATH column. For example:
CSV1.csv
PATH,Data,NF
\\server1\folderA,1,1
\\server1\folderB,1,1
\\server2\folderA,1,1
\\server2\folderB,1,1
CSV2.csv
PATH,User,Access,Size
\\server1\folderA\file1,don,1
\\server1\folderA\file2,don,1
\\server1\folderA\file3,sue,1
\\server2\folderB\file1,don,1
What I'm attempting to do is create a script that will result in separate csv exports based on the paths in CSV1 such that the new files contain file values from CSV2 that match. For example, from the above, I'd end up with 2 results:
result1.csv
\\server1\folderA\file1,don,1
\\server1\folderA\file2,don,1
\\server1\folderA\file3,sue,1
result2.csv
\\server2\folderB\file1,don,1
Previously I've used a script lime this when the two values are exact:
$reportfile = import-csv $apireportoutputfile -delimiter ';' -encoding unicode
$masterlist = import-csv $pathlistfile
foreach ($record in $masterlist)
{
$path=$record.Path
$filename = $path -replace '\\','_'
$filename = '.\Working\sharefiles\' + $filename + '.csv'
$reportfile | where-object {$_.path -eq $path} | select FilePath,UserName,LastAccessDate,LogicalSize | export-csv -path $filename
write-host " Creating files list for $path" -foregroundcolor red -backgroundcolor white
}
however since the two path values are not the same, it returns nothing. I found a -like operator but am not sure how to use it in this code to get the results I want. where-object is a filter while -like ends up returning a true/false. Am I on the right track? Any ideas for a solution?
Something like this, maybe?
$ht = #{}
Import-Csv csv1.csv |
foreach { $ht[$_.path] = New-Object collections.arraylist }
Import-Csv csv2.csv |
foreach {
$path = $_.path | Split-Path -Parent
$ht[$path].Add($_) > $null
}
$i=1
$ht.Values |
foreach { if ($_.count)
{
$_ | Export-Csv "result$i.csv" -NoTypeInformation
$i++
}
}
My suggestion:
$1=ipcsv .\csv1.CSV
$2=ipcsv .\csv2.CSV
$equal = diff ($2|select #{n='PATH';e={Split-Path $_.PATH}}) $1 -Property PATH -IncludeEqual -ExcludeDifferent -PassThru
0..(-1 + $equal.Count) | %{%{$i = $_}{
$2 | ?{ (Split-Path $_.PATH) -eq $equal[$i].PATH } | epcsv ".\Result$i.CSV"
}}