I have a PowerShell script that almost does what I want.
Basically there are CSV file feeds that are written to a specific location and stored by year and month. I have to compare the number of rows between the two newest CSV files, as a large discrepancy indicates an issue.
Currently my script fetches the newest CSV file and returns the row count with no problems, but I can't work out how to get it to return the row count for the 2 newest files. It is likely due to the way I've structured the script:
$datemonth = (Get-Date).Month
$dateyear = (Get-Date).Year
## get latest csv files
$dir = "\\160.1.1.98\c$\Scheduled Task Software\ScheduledTask\Application Files\ScheduledTask_1_0_0_9\Files\$dateyear\$datemonth\SentFeedFiles"
$latest = Get-ChildItem -Path $dir |
Sort-Object LastAccessTime -Descending |
Select-Object -First 1
## get path to csv files, add headers and count number of rows.
$filepath = $dir + '\' + $latest
$CSVCOUNT = (Import-Csv $filepath -Header 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28).Count
If I change to -First 2 then I get the following error:
Import-Csv : Could not find file '\16.1.1.18\c$\Scheduled Task Software\ScheduledTask\Application Files\ScheduledTask_1_0_0_9\Files\2017\3\SentFeedFiles\lkrlkr200317.csv lkrlkr19017.csv'.
I know why I'm getting this error - its trying to join the two file names into one path. However, I'm at a loss of how to get around this. I'm thinking a loop may be required but I'm not sure where.
Chucked 3 CSV files in f:\tmp locally to test:
$dir = "F:\tmp"
$files = Get-ChildItem -Path $dir | Sort-Object LastAccessTime -Descending | Select-Object -First 2
($files | Get-Content).Count
Import-Csv only deals with a single file as far as I remember - so you can't pass two file paths to it.
If you want to use Import-CSV (for ignoring headers etc), you can foreach file, but you have to pass the full path into it:
($files.FullName | % { Import-Csv -Path $_ }).Count
To get two separate results, do the following:
Include headers:
($files[0] | Get-Content).count
($files[1] | Get-Content).count
Exclude headers:
(Import-Csv -Path $files[0].FullName).Count
(Import-Csv -Path $files[1].FullName).Count
Related
I check every day, whether a CSV-File has been exported to a specific folder (path). At the moment there are 14 different paths with 14 different files to check. The files are being stored in the folder and are not deleted. So i have to differ between a lot of files with "lastwritetime". I would like a code to display the results in table format. I would be happy with something like this:
Name LastWriteTime Length
ExportCSV1 21.09.2022 00:50 185
ExportCSV2 21.09.2022 00:51 155
My code looks like this:
$Paths = #('Path1', 'Path2', 'Path3', 'Path4', 'Path5', 'Path6', 'Path7', 'Path8', 'Path9', 'Path10', 'Path11', 'Path12', 'Path13', 'Path13')
foreach ($Path in $Paths){
Get-ChildItem $path | Where-Object {$_.LastWriteTime}|
select -last 1
Write-host $Path
}
pause
This way i want to make sure, that the files are being sent each day.
I get the results that i want, but it is not easy to look at the results individually.
I am new to powershell and would very much appreciate your help. Thank you in advance.
Continuing from my comments, here is how you could do this:
$Paths = #('Path1', 'Path2', 'Path3', 'Path4', 'Path5', 'Path6', 'Path7', 'Path8', 'Path9', 'Path10', 'Path11', 'Path12', 'Path13', 'Path13')
$Paths | ForEach-Object {
Get-ChildItem $_ | Where-Object {$_.LastWriteTime} | Select-Object -Last 1
} | Format-Table -Property Name, LastWriteTime, Length
If you want to keep using foreach() instead, you have to wrap it in a scriptblock {…} to be able to chain everything to Format-Table:
. {
foreach ($Path in $Paths){
Get-ChildItem $path | Where-Object {$_.LastWriteTime} | Select-Object -Last 1
}
} | Format-Table -Property Name, LastWriteTime, Length
Here the . operator is used to run the scriptblock immediately, without creating a new scope. If you want to create a new scope (e. g. to define temporary variables that exist only within the scriptblock), you could use the call operator & instead.
Scenario: Working PS script traverses multiple folders containing logs (folder name is the name of the originating server), extracting the last line from the last completed log - this works, but in case the last log line is part of a multiline message, I get useless data from this line.
Question: How can I implement in the working script a cleanup that puts multiline messages all in one line?
Working script (original post)
# List all folders on the Initial Path
$folders = Get-ChildItem D:\Logs | Select-Object -ExpandProperty FullName
$export = [system.collections.generic.list[pscustomobject]]::new()
foreach($folder in $folders)
{
# Get the newest file in each folder
$file = Get-ChildItem $folder -File | Sort-Object LastWriteTime -Descending | Select -First 1
# Read the content of the file and get the last line
#$content = (Get-Content $file)[-1]
$content = (Get-Content $file.FullName)[-1]
# Here you can create a new object that will be used to export to Csv
# As an example, i'll create an object with the File Name,
# Last Write Time and Last Line of the Content (This will be the CSV columns)
$export.Add(
[PSCustomObject]#{
DirName=$file.FullName
# FileName = $file.Name
LastWriteTime = $file.LastWriteTime
LastLine = $content
})
}
# After the loop finishes you can export the results to a Csv
$export | Export-Csv -NoTypeInformation D:\export\MyFilelist_LastLogLines.csv
I have a directory on a server called 'servername'. In that directory, I have subdirectories whose name is a date. In those date directories, I have about 150 .csv file audit logs.
I have a partially working script that starts from inside the date directory, enumerates and loops through the .csv's and searches for a string in a column. Im trying to get it to export the row for each match then go on to the next file.
$files = Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525'
ForEach ($file in $files) {
$Result = If (import-csv $file.FullName | Where {$_.'path/from' -like "*01May18.xlsx*"})
{
$result | Export-CSV -Path c:\temp\output.csv -Append}
}
What I am doing is searching the 'path\from' column for a string - like a file name. The column contains data that is always some form of \folder\folder\folder\filename.xls. I am searching for a specific filename and for all instances of that file name in that column in that file.
My issue is getting that row exported - export.csv is always empty. Id also like to start a directory 'up' and go through each date directory, parse, export, then go on to the next directory and files.
If I break it down to just one file and get it out of the IF it seems to give me a result so I think im getting something wrong in the IF or For-each but apparently thats above my paygrade - cant figure it out....
Thanks in advance for any assistance,
RichardX
The issue is your If block, when you say $Result = If () {$Result | ...} you are saying that the new $Result is equal what's returned from the if statement. Since $Result hasn't been defined yet, this is $Result = If () {$null | ...} which is why you are getting a blank line.
The If block isn't even needed. you filter your csv with Where-Object already, just keep passing those objects down the pipeline to the export.
Since it sounds like you are just running this against all the child folders of the parent, sounds like you could just use the -Recurse parameter of Get-ChildItem
Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\' -Recurse |
ForEach-Object {
Import-csv $_.FullName |
Where-Object {$_.'path/from' -like "*01May18.xlsx*"}
} | Export-CSV -Path c:\temp\output.csv
(I used a ForEach-Object loop rather than foreach just demonstrate objects being passed down the pipeline in another way)
Edit: Removed append per Bill_Stewart's suggestion. Will write out all entries for the the recursed folders in the run. Will overwrite on next run.
I don't see a need for appending the CSV file? How about:
Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525' | ForEach-Object {
Import-Csv $_.FullName | Where-Object { $_.'path/from' -like '*01May18.xlsx*' }
} | Export-Csv 'C:\Temp\Output.csv' -NoTypeInformation
Assuming your CSVs are in the same format and that your search text is not likely to be present in any other columns you could use a Select-String instead of Import-Csv. So instead of converting string to object and back to string again, you can just process as strings. You would need to add an additional line to fake the header row, something like this:
$files = Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525'
$result = #()
$result += Get-Content $files[0] -TotalCount 1
$result += ($files | Select-String -Pattern '01May18\.xlsx').Line
$result | Out-File 'c:\temp\output.csv'
The following code gives me the correct output to console. But I would need it in a csv file:
$array = #{}
$files = Get-ChildItem "C:\Temp\Logs\*"
foreach($file in $files){
foreach($row in (Get-Content $file | select -Last 2)){
if($row -like "Total peak job memory used:*"){
$sp_memory = $row.Split(" ")[5]
$array.Add(($file.BaseName),([double]$sp_memory))
break
}
}
}
$array.GetEnumerator() | sort Value -Descending |Format-Table -AutoSize
current output (console):
required output (csv):
In order to increase performance I would like to avoid the array and write output directly to csv (no append).
Thanks in advance!
Change your last line to this -
$array.GetEnumerator() | sort Value -Descending | select #{l='FileName'; e={$_.Name}}, #{l='Memory (MB)'; e={$_.Value }} | Export-Csv -path $env:USERPROFILE\Desktop\Output.csv -NoTypeInformation
This will give you a csv file named Output.csv on your desktop.
I am using Calculated properties to change the column headers to FileName and Memory (MB) and piping the output of $array to Export-Csv cmdlet.
Just to let you know, your variable $array is of type Hashtable which won't store duplicate keys. If you need to store duplicate key/value pairs, you can use arrays. Just suggesting! :)
Total Newbie with PowerShell, but used to use WSH with .vbs back in the day - so hopefully can structure this question correctly.
I would like to extract x number of columns from a .csv file, only if the row data equals a certain value - and then send the filtered data to a new .csv in another destination.
So taking a saved Windows event log as an example, I would like to extract Columns A-F but only on rows where column 'A' equals 'Error' - and then send that output to a new .csv in a child directory.
I think I am pretty close, but can only get it to save columns A-F but no rows with the data I need!
Can anyone help me figure this out or show me where I am going wrong please?
$folderPath = 'C:\DLA\'
$folderPathDest = 'C:\DLA\OUT\'
$desiredColumns = 'A','B','C','D','E','F'
$topics.Where({$desiredColumns.play -eq 'Error'}).topic
Get-ChildItem $folderPath -Name |
ForEach-Object {
$filePath = $folderPath + $_
$filePathdest = $folderPathDest + $_
Import-Csv $filePath | Select $desiredColumns | Select $topics |
Export-Csv -Path $filePathDest –NoTypeInformation
}
Just put the filter directly in your pipeline:
Import-Csv $filePath | Select $desiredColumns | where {$_.A -eq 'Error'} | Export-Csv -Path $filePathDest –NoTypeInformation
Below command worked for me to extract all the columns to a new file. This can be modified to select the desired columns:
import-csv $filePath | ? { $_.columnName -eq 'Error' } | export-csv $filePathDest -NoTypeInformation
columnName is the header title of the columns