Powershell, Loop through CSV files and search for a string in a row, then Export - powershell

I have a directory on a server called 'servername'. In that directory, I have subdirectories whose name is a date. In those date directories, I have about 150 .csv file audit logs.
I have a partially working script that starts from inside the date directory, enumerates and loops through the .csv's and searches for a string in a column. Im trying to get it to export the row for each match then go on to the next file.
$files = Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525'
ForEach ($file in $files) {
$Result = If (import-csv $file.FullName | Where {$_.'path/from' -like "*01May18.xlsx*"})
{
$result | Export-CSV -Path c:\temp\output.csv -Append}
}
What I am doing is searching the 'path\from' column for a string - like a file name. The column contains data that is always some form of \folder\folder\folder\filename.xls. I am searching for a specific filename and for all instances of that file name in that column in that file.
My issue is getting that row exported - export.csv is always empty. Id also like to start a directory 'up' and go through each date directory, parse, export, then go on to the next directory and files.
If I break it down to just one file and get it out of the IF it seems to give me a result so I think im getting something wrong in the IF or For-each but apparently thats above my paygrade - cant figure it out....
Thanks in advance for any assistance,
RichardX

The issue is your If block, when you say $Result = If () {$Result | ...} you are saying that the new $Result is equal what's returned from the if statement. Since $Result hasn't been defined yet, this is $Result = If () {$null | ...} which is why you are getting a blank line.
The If block isn't even needed. you filter your csv with Where-Object already, just keep passing those objects down the pipeline to the export.
Since it sounds like you are just running this against all the child folders of the parent, sounds like you could just use the -Recurse parameter of Get-ChildItem
Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\' -Recurse |
ForEach-Object {
Import-csv $_.FullName |
Where-Object {$_.'path/from' -like "*01May18.xlsx*"}
} | Export-CSV -Path c:\temp\output.csv
(I used a ForEach-Object loop rather than foreach just demonstrate objects being passed down the pipeline in another way)
Edit: Removed append per Bill_Stewart's suggestion. Will write out all entries for the the recursed folders in the run. Will overwrite on next run.

I don't see a need for appending the CSV file? How about:
Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525' | ForEach-Object {
Import-Csv $_.FullName | Where-Object { $_.'path/from' -like '*01May18.xlsx*' }
} | Export-Csv 'C:\Temp\Output.csv' -NoTypeInformation

Assuming your CSVs are in the same format and that your search text is not likely to be present in any other columns you could use a Select-String instead of Import-Csv. So instead of converting string to object and back to string again, you can just process as strings. You would need to add an additional line to fake the header row, something like this:
$files = Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525'
$result = #()
$result += Get-Content $files[0] -TotalCount 1
$result += ($files | Select-String -Pattern '01May18\.xlsx').Line
$result | Out-File 'c:\temp\output.csv'

Related

Powershell - Finding the output of get-contents and searching for all occurrences in another file using wild cards

I'm trying to get the output of two separate files although I'm stuck on the wild card or contains select-string search from file A (Names) in file B (name-rank).
The contents of file A is:
adam
george
william
assa
kate
mark
The contents of file B is:
12-march-2020,Mark-1
12-march-2020,Mark-2
12-march-2020,Mark-3
12-march-2020,william-4
12-march-2020,william-2
12-march-2020,william-7
12-march-2020,kate-54
12-march-2020,kate-12
12-march-2020,kate-44
And I need to match on every occurrence of the names after the '-' so my ordered output should look like this which is a combination of both files as the output:
mark
Mark-1
Mark-2
Mark-3
william
william-2
william-4
william-7
Kate
kate-12
kate-44
kate-54
So far I only have the following and I'd be grateful for any pointers or assistance please.
import-csv (c:\temp\names.csv) |
select-string -simplematch (import-csv c:\temp\names-rank.csv -header "Date", "RankedName" | select RankedName) |
set-content c:\temp\names-and-ranks.csv
I imagine the select-string isn't going to be enough and I need to write a loop instead.
The data you give in the example does not give you much to work with, and the desired output is not that intuitive, most of the time with Powershell you would like to combine the data in to a much richer output at the end.
But anyway, with what is given here and what you want, the code bellow will get what you need, I have left comments in the code for you
$pathDir='C:\Users\myUser\Downloads\trash'
$names="$pathDir\names.csv"
$namesRank="$pathDir\names-rank.csv"
$nameImport = Import-Csv -Path $names -Header names
$nameRankImport= Import-Csv -Path $namesRank -Header date,rankName
#create an empty array to collect the result
$list=#()
foreach($name in $nameImport){
#get all the match names
$match=$nameRankImport.RankName -like "$($name.names)*"
#add the name from the First list
$list+=($name.names)
#if there are any matches, add them too
if($match){
$list+=$match
}
}
#Because its a one column string, Export-CSV will now show us what we want
$list | Set-Content -Path "$pathDir\names-and-ranks.csv" -Force
For this I would use a combination of Group-Object and Where-Object to first group all "RankedName" items by the name before the dash, then filter on those names to be part of the names we got from the 'names.csv' file and output the properties you need.
# read the names from the file as string array
$names = Get-Content -Path 'c:\temp\names.csv' # just a list of names, so really not a CSV
# import the CSV file and loop through
Import-Csv -Path 'c:\temp\names-rank.csv' -Header "Date", "RankedName" |
Group-Object { ($_.RankedName -split '-')[0] } | # group on the name before the dash in the 'RankedName' property
Where-Object { $_.Name -in $names } | # use only the groups that have a name that can be found in the $names array
ForEach-Object {
$_.Name # output the group name (which is one of the $names)
$_.Group.RankedName -join [environment]::NewLine # output the group's 'RankedName' property joined with a newline
} |
Set-Content -Path 'c:\temp\names-and-ranks.csv'
Output:
Mark
Mark-1
Mark-2
Mark-3
william
william-4
william-2
william-7
kate
kate-54
kate-12
kate-44

Powershell - How to create array of filenames based on filename?

I'm looking to create an array of files (pdf's specifically) based on filenames in Powershell. All files are in the same directory. I've spent a couple of days looking and can't find anything that has examples of this or something that is close but could be changed. Here is my example of file names:
AR - HELLO.pdf
AF - HELLO.pdf
RT - HELLO.pdf
MH - HELLO.pdf
AR - WORLD.pdf
AF - WORLD.pdf
RT - WORLD.pdf
HT - WORLD.pdf
....
I would like to combine all files ending in 'HELLO' into an array and 'WORLD' into another array and so on.
I'm stuck pretty early on in the process as I'm brand new to creating scripts, but here is my sad start:
Get-ChildItem *.pdf
Where BaseName -match '(.*) - (\w+)'
Updated Info...
I do not know the name after the " - " so using regex is working.
My ultimate goal is to combine PDF's based on the matching text after the " - " in the filename and the most basic code for this is:
$file1 = "1 - HELLO.pdf"
$file2 = "2 - HELLO.PDF"
$mergedfile = "HELLO.PDF"
Merge-PDF -InputFile $file1, $file2 -OututFile $mergedfile
I have also gotten the Merge-PDF to work using this code which merges all PDF's in the directory:
$Files = Get-ChildItem *.pdf
$mergedfiles = "merged.pdf"
Merge-PDF -InputFile $Files -OutputFile $mergedfiles
Using this code from #Mathias the $suffix portion of the -OutputFile works but the -InputFile portion is returning an error "Exception calling "Close" with "0" argument(s)"
$groups = Get-ChildItem *.pdf |Group-Object {$_.BaseName -replace
'^.*\b(\w+)$','$1'} -AsHashTable
foreach($suffix in $groups.Keys) {Merge-PDF -InputFile $(#($groups[$suffix]))
-OutputFile "$suffix.pdf"}
For the -InputFile I've tried a lot of different varieties and I keep getting the "0" arguments error. The values in the Hashtable seem to be correct so I'm not sure why this isn't working.
Thanks
This should do the trick:
$HELLO = Get-ChildItem *HELLO.pdf |Select -Expand Name
$WORLD = Get-ChildItem *WORLD.pdf |Select -Expand Name
If you want to group file names by the last word in the base name and you don't know them up front, regex is indeed an option:
$groups = Get-ChildItem *.pdf |Group-Object {$_.BaseName -replace '^.*\b(\w+)$','$1'} -AsHashTable
And then you can do:
$groups['HELLO'].Name
for all the file names ending with the word HELLO, or, to iterate over all of them:
foreach($suffixGroup in $groups.GetEnumerator()){
Write-Host "There are $($suffixGroup.Value.Count) files ending in $($suffixGroup.Key)"
}
Another option is to get all items with Get-ChildItem and use Where-Object to filter.
$fileNames = Get-ChildItem | Select-Object -ExpandProperty FullName
#then filter
$fileNames | Where-Object {$_.EndsWith("HELLO.PDF")}
#or use the aliases if you want to do less typing:
$fileNames = gci | select -exp FullName
$fileNames | ? {$_.EndsWith("HELLO.PDF")}
Just wanted to show more options -especially the Where-Object cmdlet which comes in useful when you're calling cmdlets that don't have parameters to filter.
Side note:
You may be asking what -ExpandProperty does.
If you just call gci | select -exp FullName, you will get back an array of PSCustomObjects (each of them with one property called FullName).
This can be confusing for people who don't really see that the objects are typed as it is not visible just by looking at the PowerShell script.

powershell: Write specific rows from files to formatted csv

The following code gives me the correct output to console. But I would need it in a csv file:
$array = #{}
$files = Get-ChildItem "C:\Temp\Logs\*"
foreach($file in $files){
foreach($row in (Get-Content $file | select -Last 2)){
if($row -like "Total peak job memory used:*"){
$sp_memory = $row.Split(" ")[5]
$array.Add(($file.BaseName),([double]$sp_memory))
break
}
}
}
$array.GetEnumerator() | sort Value -Descending |Format-Table -AutoSize
current output (console):
required output (csv):
In order to increase performance I would like to avoid the array and write output directly to csv (no append).
Thanks in advance!
Change your last line to this -
$array.GetEnumerator() | sort Value -Descending | select #{l='FileName'; e={$_.Name}}, #{l='Memory (MB)'; e={$_.Value }} | Export-Csv -path $env:USERPROFILE\Desktop\Output.csv -NoTypeInformation
This will give you a csv file named Output.csv on your desktop.
I am using Calculated properties to change the column headers to FileName and Memory (MB) and piping the output of $array to Export-Csv cmdlet.
Just to let you know, your variable $array is of type Hashtable which won't store duplicate keys. If you need to store duplicate key/value pairs, you can use arrays. Just suggesting! :)

Build Row List dynamically using PowerShell

I have this requirement:
ListItem
--------
ABV
DEF
GED
ZAF
NOTE: ListItem is from an output of process one at a time.
foreach ($list in $listItem)
{ # Want to turn this into single row as each item get loaded
ABV,DEF,GED,ZAF
}
What I have done to solve this is to land this entries into file first before processing with the code below
$list = Get-Content -Path C:\Ftest
[string]::Join(',' , $list) | Set-Content -Path C:\FFtestFinal
What I cannot figure out is how to build the list without landing it in a file. I want to land it directory in C:\FFtestFinal with the values
ABV,DEF,GED,ZAF.
Try something like this:
$YourInput = 'ABV','DEF','GED','ZAF'
$YourInput -join ';'
Or (overall idea):
(Get-ChildItem -Filter '*.*' | Select-Object -ExpandProperty Name) -join ';'

Parse line of text and match with parse of CSV

As a continuation of a script I'm running, working on the following.
I have a CSV file that has formatted information, example as follows:
File named Import.csv:
Name,email,x,y,z
\I\RS\T\Name1\c\x,email#jksjks,d,f
\I\RS\T\Name2\d\f,email#jsshjs,d,f
...
This file is large.
I also have another file called Note.txt.
Name1
Name2
Name3
...
I'm trying to get the content of Import.csv and for each line in Note.txt if the line in Note.txt matches any line in Import.csv, then copy that line into a CSV with append. Continue adding every other line that is matched. Then this loops on each line of the CSV.
I need to find the best way to do it without having it import the CSV multiple times, since it is large.
What I got does the opposite though, I think:
$Dir = PathToFile
$import = Import-Csv $Dir\import.csv
$NoteFile = "$Dir\Note.txt"
$Note = GC $NoteFile
$Name = (($Import.Name).Split("\"))[4]
foreach ($j in $import) {
foreach ($i in $Note) {
$j | where {$Name -eq "$i"} | Export-Csv "$Dir\Result.csv" -NoTypeInfo -Append
}
}
This takes too long and I'm not getting the extraction I need.
This takes too long and I'm not getting the extraction I need.
That's because you only assign $name once, outside of the outer foreach loop, so you're basically performing the same X comparisons for each line in the CSV.
I would rewrite the nested loops as a single Where-Object filter, using the -contains operator:
$Import |Where-Object {$Note -contains $_.Name.Split('\')[4]} |Export-Csv "$Dir\Result.csv" -NoTypeInformation -Append
Group the imported data by your distinguishing feature, filter the groups by name, then expand the remaining groups and write the data to the output file:
Import-Csv "$Dir\import.csv" |
Group-Object { $_.Name.Split('\')[4] } |
Where-Object { $Note -contains $_.Name } |
Select-Object -Expand Group |
Export-Csv "$Dir\Result.csv" -NoType