Total Newbie with PowerShell, but used to use WSH with .vbs back in the day - so hopefully can structure this question correctly.
I would like to extract x number of columns from a .csv file, only if the row data equals a certain value - and then send the filtered data to a new .csv in another destination.
So taking a saved Windows event log as an example, I would like to extract Columns A-F but only on rows where column 'A' equals 'Error' - and then send that output to a new .csv in a child directory.
I think I am pretty close, but can only get it to save columns A-F but no rows with the data I need!
Can anyone help me figure this out or show me where I am going wrong please?
$folderPath = 'C:\DLA\'
$folderPathDest = 'C:\DLA\OUT\'
$desiredColumns = 'A','B','C','D','E','F'
$topics.Where({$desiredColumns.play -eq 'Error'}).topic
Get-ChildItem $folderPath -Name |
ForEach-Object {
$filePath = $folderPath + $_
$filePathdest = $folderPathDest + $_
Import-Csv $filePath | Select $desiredColumns | Select $topics |
Export-Csv -Path $filePathDest –NoTypeInformation
}
Just put the filter directly in your pipeline:
Import-Csv $filePath | Select $desiredColumns | where {$_.A -eq 'Error'} | Export-Csv -Path $filePathDest –NoTypeInformation
Below command worked for me to extract all the columns to a new file. This can be modified to select the desired columns:
import-csv $filePath | ? { $_.columnName -eq 'Error' } | export-csv $filePathDest -NoTypeInformation
columnName is the header title of the columns
Related
I have a csv file with two columns and multiple rows, which has the information of files with folder location and its corresponding size, like below
"Folder_Path","Size"
"C:\MSSQL\DATA\UsersData\FTP.txt","21345"
"C:\MSSQL\DATA\UsersData\Norman\abc.csv","78956"
"C:\MSSQL\DATA\UsersData\Market_Database\123.bak","1234456"
What i want do is remove the "C:\MSSQL\DATA\" part from every row in the csv and keep the rest of the folder path after starting from UsersData and all other data intact as this info is repetitive. So my csv should like this below.
"Folder_Path","Size"
"UsersData\FTP.txt","21345"
"UsersData\Norman\abc.csv","78956"
"UsersData\Market_Database\123.bak","1234456"
What i am running is as below
Import-Csv ".\abc.csv" |
Select-Object -Property #{n='Folder_Path';e={$_.'Folder_Path'.Split('C:\MSSQL\DATA\*')[0]}}, * |
Export-Csv '.\output.csv' -NTI
Any help is appreciated!
Seems like a job for a simple string replace:
Get-Content "abc.csv" | foreach { $_.replace("C:\MSSQL\DATA\", "") | Set-Content "output.csv"
or:
[System.IO.File]::WriteAllText("output.csv", [System.IO.File]::ReadAllText("abc.csv" ).Replace("C:\MSSQL\DATA\", ""))
This should work:
Import-Csv ".\abc.csv" |
Select-Object -Property #{n='Folder_Path';e={$_.'Folder_Path' -replace '^.*\\(.*\\.*)$', '$1'}}, Size |
Export-Csv '.\output.csv' -NoTypeInformation
I would like to read data from csv or another txt files. Data should been read only from row 1 and few columns on row 1 and save them to variables and after saving delete the row. Now I have done it like this:
Get-ChildItem -Path C:\path | ForEach-Object -Process {
$YourContent = Get-Content -Path $_.FullName
$YourVariable = $YourContent | Select-Object -First 1
$YourContent | Select-Object -Skip 1 | Set-Content -Path $_.FullName
My problem is that my variable prints out like this :
Elvis;867.5390;elvis#geocities.com
So I would like to save each variable to its own column. Example what csv could look:
Elvis | 867.5309 | Elvis#Geocities.com
Sammy | 555.1234 | SamSosa#Hotmail.com
Use Import-Csv instead of Get-Content:
Import-Csv file.csv -Delimiter ";" -Header A, B, C
here's one way to do what i think you want.
the 1st 8 lines make a file to work with. [grin]
line 10 reads in that file
lines 11-13 convert the 1st line into an object & remove the unwanted property
lines 14-15 grab all BUT the 1st line & send it to overwrite the source file
the remaining lines show what was done [grin]
Code:
$FileName = "$env:TEMP\Pimeydentimo.txt"
# create a file to work with
#'
Alfa;123.456;Some unwanted info;Alfa#example.com
Bravo;234.567;More info that can be dropped;Bravo#example.com
Charlie;345.678;This is also ignoreable;Charlie#example.com
'# | Set-Content -LiteralPath $FileName
$InStuff = Get-Content -LiteralPath $FileName
$TempObject = $InStuff[0] |
ConvertFrom-Csv -Delimiter ';' -Header 'Name', 'Number', 'DropThisOne', 'Email' |
Select-Object -Property * -ExcludeProperty DropThisOne
$InStuff[1..$InStuff.GetUpperBound(0)] |
Set-Content -LiteralPath $FileName
$InStuff
'=' * 30
$TempObject
'=' * 30
Get-Content -LiteralPath $FileName
output ...
Alfa;123.456;Some unwanted info;Alfa#example.com
Bravo;234.567;More info that can be dropped;Bravo#example.com
Charlie;345.678;This is also ignoreable;Charlie#example.com
==============================
Name Number Email
---- ------ -----
Alfa 123.456 Alfa#example.com
==============================
Bravo;234.567;More info that can be dropped;Bravo#example.com
Charlie;345.678;This is also ignoreable;Charlie#example.com
Thanks for the answers!
I try to clarify a bit more what i was trying to do. Answers might do it already, but I'm not yet that good in Powershell and learning still a alot.
If I have csv or any other txt file, i would want to read the first row of the file. The row contains more than one piece of information. I want also save each piece of information to Variables. After saving information to variables, I would like to delete the row.
Example:
Car Model Year
Ford Fiesta 2015
Audi A6 2018
In this example, i would like to save Ford, Fiesta and 2015 to variables (row 1)($Card, $Model, $Year) and after it delete the row. The 2nd row should not be deleted, because it is used later on
I have a directory on a server called 'servername'. In that directory, I have subdirectories whose name is a date. In those date directories, I have about 150 .csv file audit logs.
I have a partially working script that starts from inside the date directory, enumerates and loops through the .csv's and searches for a string in a column. Im trying to get it to export the row for each match then go on to the next file.
$files = Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525'
ForEach ($file in $files) {
$Result = If (import-csv $file.FullName | Where {$_.'path/from' -like "*01May18.xlsx*"})
{
$result | Export-CSV -Path c:\temp\output.csv -Append}
}
What I am doing is searching the 'path\from' column for a string - like a file name. The column contains data that is always some form of \folder\folder\folder\filename.xls. I am searching for a specific filename and for all instances of that file name in that column in that file.
My issue is getting that row exported - export.csv is always empty. Id also like to start a directory 'up' and go through each date directory, parse, export, then go on to the next directory and files.
If I break it down to just one file and get it out of the IF it seems to give me a result so I think im getting something wrong in the IF or For-each but apparently thats above my paygrade - cant figure it out....
Thanks in advance for any assistance,
RichardX
The issue is your If block, when you say $Result = If () {$Result | ...} you are saying that the new $Result is equal what's returned from the if statement. Since $Result hasn't been defined yet, this is $Result = If () {$null | ...} which is why you are getting a blank line.
The If block isn't even needed. you filter your csv with Where-Object already, just keep passing those objects down the pipeline to the export.
Since it sounds like you are just running this against all the child folders of the parent, sounds like you could just use the -Recurse parameter of Get-ChildItem
Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\' -Recurse |
ForEach-Object {
Import-csv $_.FullName |
Where-Object {$_.'path/from' -like "*01May18.xlsx*"}
} | Export-CSV -Path c:\temp\output.csv
(I used a ForEach-Object loop rather than foreach just demonstrate objects being passed down the pipeline in another way)
Edit: Removed append per Bill_Stewart's suggestion. Will write out all entries for the the recursed folders in the run. Will overwrite on next run.
I don't see a need for appending the CSV file? How about:
Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525' | ForEach-Object {
Import-Csv $_.FullName | Where-Object { $_.'path/from' -like '*01May18.xlsx*' }
} | Export-Csv 'C:\Temp\Output.csv' -NoTypeInformation
Assuming your CSVs are in the same format and that your search text is not likely to be present in any other columns you could use a Select-String instead of Import-Csv. So instead of converting string to object and back to string again, you can just process as strings. You would need to add an additional line to fake the header row, something like this:
$files = Get-ChildItem '\\servername\volume\dir1\audit\serverbeingaudited\20180525'
$result = #()
$result += Get-Content $files[0] -TotalCount 1
$result += ($files | Select-String -Pattern '01May18\.xlsx').Line
$result | Out-File 'c:\temp\output.csv'
The following code gives me the correct output to console. But I would need it in a csv file:
$array = #{}
$files = Get-ChildItem "C:\Temp\Logs\*"
foreach($file in $files){
foreach($row in (Get-Content $file | select -Last 2)){
if($row -like "Total peak job memory used:*"){
$sp_memory = $row.Split(" ")[5]
$array.Add(($file.BaseName),([double]$sp_memory))
break
}
}
}
$array.GetEnumerator() | sort Value -Descending |Format-Table -AutoSize
current output (console):
required output (csv):
In order to increase performance I would like to avoid the array and write output directly to csv (no append).
Thanks in advance!
Change your last line to this -
$array.GetEnumerator() | sort Value -Descending | select #{l='FileName'; e={$_.Name}}, #{l='Memory (MB)'; e={$_.Value }} | Export-Csv -path $env:USERPROFILE\Desktop\Output.csv -NoTypeInformation
This will give you a csv file named Output.csv on your desktop.
I am using Calculated properties to change the column headers to FileName and Memory (MB) and piping the output of $array to Export-Csv cmdlet.
Just to let you know, your variable $array is of type Hashtable which won't store duplicate keys. If you need to store duplicate key/value pairs, you can use arrays. Just suggesting! :)
I'm having trouble making some changes to a series of CSV files, all with the same data structure. I'm trying to combine all of the files into one CSV file or one tab delimited text file (don't really mind), however each file needs to have 2 empty rows removed and two of the columns removed, below is an example:
col1,col2,col3,col4,col5,col6 <-remove
col1,col2,col3,col4,col5,col6 <-remove
col1,col2,col3,col4,col5,col6
col1,col2,col3,col4,col5,col6
^ ^
remove remove
End Result:
col1,col2,col4,col6
col1,col2,col4,col6
This is my attempt at doing this (I'm very new to Powershell)
$ListofFiles = "example.csv" #this is an list of all the CSV files
ForEach ($file in $ListofFiles)
{
$content = Get-Content ($file)
$content = $content[2..($content.Count)]
$contentArray = #()
[string[]]$contentArray = $content -split ","
$content = $content[0..2 + 4 + 6]
Add-Content '...\output.txt' $content
}
Where am I going wrong here...
your example file should be read, before foreach to fetch the file list
$ListofFiles = get-content "example.csv"
Inside the foreach you are getting content of mainfile
$content = Get-Content ($ListofFiles)
instead of
$content = Get-Content $file
and for removing rows i will recommend this:
$obj = get-content C:\t.csv | select -Index 0,1,3
for removing columns (column numbers 0,1,3,5):
$obj | %{(($_.split(","))[0,1,3,5]) -join "," } | out-file test.csv -Append
According to the fact the initial files looks like
col1,col2,col3,col4,col5,col6
col1,col2,col3,col4,col5,col6
,,,,,
,,,,,
You can also try this one liner
Import-Csv D:\temp\*.csv -Header 'C1','C2','C3','C4','C5','C6' | where {$_.c1 -ne ''} | select -Property 'C1','C2','C5' | Export-Csv 'd:\temp\final.csv' -NoTypeInformation
According to the fact that you CSVs have all the same structure, you can directly open them providing the header, then remove objects with the missing datas then export all the object in a csv file.
It is sufficient to specify fictitious column names, with a column number that can exceed the number of columns in the file, change where you want and exclude columns that you do not want to take.
gci "c:\yourdirwithcsv" -file -filter *.csv |
%{ Import-Csv $_.FullName -Header C1,C2,C3,C4,C5,C6 |
where C1 -ne '' |
select -ExcludeProperty C3, C4 |
export-csv "c:\temp\merged.csv" -NoTypeInformation
}