In a nutshell, I have an excel file that I need :
- Only 2 Columns (ComputerName, Results)
- Only need rows that contain specific items (IE. Start with DriveLetter:\, HKLM, %windir%, etc.)
I'm just not sure on the proper keyword syntax here. The original file is an xlsx.
Please forgive the crudeness of my script. I gathered bits and pieces trying to get it to work.
#File Import to Variable
Function Remove-File($fileName) {
if(Test-Path -path $fileName) { Remove-Item -path $fileName }
}
$excelFile = ".\Computers.xlsx"
if(Test-Path -path $excelFile) {
$csvFile = ($env:temp + "\" + ((Get-Item -path $excelFile).name).Replace(((Get-Item -path $excelFile).extension),".csv"))
Remove-File $csvFile
$excelObject = New-Object -ComObject Excel.Application
$excelObject.Visible = $false
$workbookObject = $excelObject.Workbooks.Open($excelFile)
$workbookObject.SaveAs($csvFile,6) # http://msdn.microsoft.com/en-us/library/bb241279.aspx
$workbookObject.Saved = $true
$workbookObject.Close()
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($workbookObject) | Out-Null
$excelObject.Quit()
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($excelObject) | Out-Null
[System.GC]::Collect()
[System.GC]::WaitForPendingFinalizers()
$spreadsheetDataObject = Import-Csv -path $csvFile # Use the $spreadsheetDataObject for your analysis
Remove-File $csvFile
}
#Filter Out All Columns except ComputerName, Results and subseqently create a CSV file
$PathCSV = ".\Computers.csv"
$spreadsheetDataObject | Select-Object ComputerName,Results | Export-Csv -Path $PathCSV -NoTypeInformation
$Keywords = "*HKLM*","*C:\*","*%windir%*"
$Filter = "($(($Keywords|%{[RegEx]::Escape($_)}) -join "|"))"
Import-CSV $PathCSV | Where-Object{$_Results -match $Keywords} | Export-Csv -Path ".\Computers2.csv" -NoTypeInformation
I found the issue. I needed the _.Results..... and I needed to -match $Filter
Import-CSV $PathCSV | Where-Object{$_.Results -match $Filter} | Export-Csv -Path ".\Computers2.csv" -NoTypeInformation
Related
Below is the data I have in 2 csv
CSV_1
"Username","UserCreationStatus","GroupAdditionStatus"
"WA92J4063641OAD","Success","Success"
CSV_2
"GroupName","GroupCreationStatus"
"WA92GRP-ADAdminAccount-CAP-OAD","Already exist"
I need to merge them in to single csv file like below
"Username","UserCreationStatus","GroupAdditionStatus","GroupName","GroupCreationStatus"
"WA92J4063641OAD","Success","Success","WA92GRP-ADAdminAccount-CAP-OAD","Already exist"
I tried the below code
Get-ChildItem -Path $RootPath -Filter *.csv | Select-Object * | Import-Csv | Export-Csv $RootPath\merged.csv -NoTypeInformation -Append
But getting below error
Import-Csv : You must specify either the -Path or -LiteralPath parameters, but not both.
Please let me know what is wrong here
You could do simething like this.
It does not work but shows the logic.
Let me know, if you have any questions.
$CSV1 = ".\first.csv"
$CSV2 = ".\second.csv"
$NewCSV = ".\new.csv"
$Data1 = Get-Content -Path $CSV1
$Data2 = Get-Content -Path $CSV2
foreach ($Line in $CSV1)
{
Add-Content -Value "$($Line),$($CSV2[$index])" -Path $NewCSV
}
I am trying to output to a csv file the result that I am getting from "Get-ChildItem"
I need to save only the LastWriteTime and the Name.
This is the Get-ChildItem output
I am trying something like
$pc = Get-WmiObject Win32_ComputerSystem
$file = Get-ChildItem -Path S:\Docker\RCT\repo\*xml -Recurse -Force
$DeviceObj = New-Object PSObject -Property #{
DeviceName = $pc.Name
FileName = $file.Name
FileDate = $file.LastWriteTime
}
$DeviceObjList += $DeviceObj
$DeviceObjList | Export-Csv -Path "$current_path\FileStatus.csv" -NoTypeInformation -Encoding UTF8
But my csv is not saving the file name and also the timestamp
Use a ForEach-Object to loop though your $file object
$pc = Get-WmiObject Win32_ComputerSystem
$file = Get-ChildItem -Path S:\Docker\RCT\repo\*xml -Recurse -Force
$file | ForEach-Object {
$DeviceObj = New-Object PSObject -Property #{
DeviceName = $pc.Name
FileName = $_.Name
FileDate = $_.LastWriteTime
}
$DeviceObjList += $DeviceObj
}
$DeviceObjList | Export-Csv -Path "$current_path\FileStatus.csv" -NoTypeInformation -Encoding UTF8
..but in fact, this is more efficient:
$pc = Get-WmiObject Win32_ComputerSystem
$file = Get-ChildItem -Path S:\Docker\RCT\repo\*xml -Recurse -Force
$DeviceObjList = $file | ForEach-Object {
[pscustomobject]#{
DeviceName = $pc.Name
FileName = $_.Name
FileDate = $_.LastWriteTime
}
}
$DeviceObjList | Export-Csv -Path "$current_path\FileStatus.csv" -NoTypeInformation -Encoding UTF8
You Don't need WMI to get the computer name, use the built-in variable $env:COMPUTERNAME
Use Calculated Properties to add the Computer name to the results.
Iteration not required here, just add the Export-Csv to the pipeline
So you can do that:
$file = Get-ChildItem -Path S:\Docker\RCT\repo\*xml -Recurse -Force
$File | Select #{N="DeviceName";E={$env:COMPUTERNAME}},Name,LastWriteTime |
Export-Csv -Path "$current_path\FileStatus.csv" -NoTypeInformation -Encoding UTF8
And if you want to make it shorter, you can use aliases and don't define any variables
gci S:\Docker\RCT\repo\*xml -R -Fo |
select #{N="DeviceName";E={$env:COMPUTERNAME}},Name,LastWriteTime |
epcsv "$current_path\FileStatus.csv" -NoT -En UTF8
So I'm trying to process CSV files, then giving the output new name. I can do it with one file by explicitly specifying the file name. But is there a way / wildcard I can use to make the script to process multiple files at the same time? Let's just say I want to process anything with .csv as an extension. Here's my script that's used to process a specific file
$objs =#();
$output = Import-csv -Path D:\TEP\FilesProcessing\Test\file1.csv | ForEach {
$Object = New-Object PSObject -Property #{
Time = $_.READ_DTTM
Value = $_.{VALUE(KWH)}
Tag = [String]::Concat($_.SUBSTATION,'_',$_.CIRCUITNAME,'_',$_.PHASE,'_',$_.METERID,'_KWH')
}
$objs += $Object;
}
$objs
$objs | Export-CSv -NoTypeInformation D:\TEP\FilesProcessing\Test\file1_out.csv
You can combine Get-ChildItem and Import-Csv.
Here's an example that specifies different input and output directories to avoid name collisions:
$inputPath = "D:\TEP\FilesProcessing\Test"
$outputPath = "D:\TEP\FilesProcessing\Output"
Get-ChildItem (Join-Path $inputPath "*.csv") | ForEach-Object {
$outputFilename = Join-Path $outputPath $_.Name
Import-Csv $_.FullName | ForEach-Object {
New-Object PSObject -Property #{
"Time" = $_.READ_DTTM
"Value" = $_.{VALUE(KWH)}
"Tag" = "{0}_{1}_{2}_{3}_KWH" -f $_.SUBSTATION,$_.CIRCUITNAME,$_.PHASE,$_.METERID
}
} | Export-Csv $outputFilename -NoTypeInformation
}
Note that there's no need for creating an array and repeatedly appending it. Just output the custom objects you want and export afterwards.
Use the Get-Childitem and cut out all the unnecessary intermediate variables so that you code it in a more Powershell type way. Something like this:
Get-CHhilditems 'D:\TEP\FilesProcessing\Test\*.csv' | % {
Import-csv $_.FullName | % {
New-Object PSObject -Property #{
Time = $_.READ_DTTM
Value = $_.{VALUE(KWH)}
Tag = '{0}_{1}_{2}_{3}_KWH' -f $_.SUBSTATION, $_.CIRCUITNAME, $_.PHASE, $_.METERID
}
} | Export-CSv ($_.FullName -replace '\.csv', '_out.csv') -NoTypeInformation
}
The Get-ChildItem is very useful for situations like this.
You can add wildcards directly into the path:
Get-ChildItem -Path D:\TEP\FilesProcessing\Test\*.csv
You can recurse a path and use the provider to filter files:
Get-ChildItem -Path D:\TEP\FilesProcessing\Test\ -recurse -include *.csv
This should get you what you need.
$Props = #{
Time = [datetime]::Parse($_.READ_DTTM)
Value = $_.{VALUE(KWH)}
Tag = $_.SUBSTATION,$_.CIRCUITNAME,$_.PHASE,$_.METERID,'KWH' -join "_"
}
$data = Get-ChildItem -Path D:\TEP\FilesProcessing\Test\*.csv | Foreach-Object {Import-CSV -Path $_.FullName}
$data | Select-Object -Property $Props | Export-CSv -NoTypeInformation D:\TEP\FilesProcessing\Test\file1_out.csv
Also when using Powershell avoid doing these things:
$objs =#();
$objs += $Object;
I need to take a slew of csv files from a directory and get them into an array in Powershell (to eventually manipulate and write back to a CSV).
The problem is there are 5 file types. I need around 8 columns from each. The columns are essentially the same, but have different headings.
Is there an easy way to do this? I started creating a custom object with my 8 fields, looping through the files importing each one, looking at the filename (which tells me the column names I need) and then a bunch of ifs to add it to my custom object array.
I was wondering if there is a simpler way...like with a template saying which columns from each file.
wound up doing this. It may have not been the most efficient, but works. I wound up writing out each file separately and combining at the end as PS really got bogged down (over a million rows combined).
$Newcsv = #()
$path = "c:\scrap\BWFILES\"
$files = gci -path $path -recurse -filter *.csv | Where-Object { ! ($_.psiscontainer) }
$counter=1
foreach($file in $files)
{
$csv = Import-Csv $file.FullName
if ($file.Name -like '*SAV*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"SV"}},DMBRCH,DMACCT,DMSHRT
}
if ($file.Name -like '*TIME*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"TM"}},TMBRCH,TMACCT,TMSHRT
}
if ($file.Name -like '*TRAN*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"TR"}},DMBRCH,DMACCT,DMSHRT
}
if ($file.Name -like '*LN*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"LN"}},LNBRCH,LNNOTE,LNSHRT
}
$Newcsv | Export-Csv "C:\scrap\$file.name$counter.csv" -force -notypeinformation
$counter++
}
get-childItem "c:\scrap\*.csv" | foreach {
$filePath = $_
$lines = $lines = Get-Content $filePath
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 1}
}
$getFirstLine = $false
Add-Content "c:\scrap\combined.csv" $linesToWrite
}
With a hashtable for reference, a little RegEx matching, and using the automatic variable $Matches in a ForEach-Object loop (alias % used) that could all be shortened to:
$path = "c:\scrap\BWFILES\"
$Reference = #{
'SAV' = 'SV'
'TIME' = 'TM'
'TRAN' = 'TR'
'LN'='LN'
}
Set-Content -Value "PRODUCT,BRCH,ACCT,SHRT" -Path 'c:\scrap\combined.csv'
gci -path $path -recurse -filter *.csv | Where-Object { !($_.psiscontainer) -and $_.Name -match ".*(SAV|TIME|TRAN|LN).*"}|%{
$Product = $Reference[($Matches[1])]
Import-CSV $_.FullName | Select-Object #{Name="PRODUCT";Expression={$Product}},*BRCH,#{l='Acct';e={$_.LNNOTE, $_.DMACCT, $_.TMACCT|?{$_}}},*SHRT | ConvertTo-Csv -NoTypeInformation | Select -Skip 1 | Add-Content 'c:\scrap\combined.csv'
}
That should produce the exact same file. Only kind of tricky part was the LNNOTE/TMACCT/DMACCT field since obviously you can't just do the same as like *SHRT.
I have two csv files, each that contain a PATH column. For example:
CSV1.csv
PATH,Data,NF
\\server1\folderA,1,1
\\server1\folderB,1,1
\\server2\folderA,1,1
\\server2\folderB,1,1
CSV2.csv
PATH,User,Access,Size
\\server1\folderA\file1,don,1
\\server1\folderA\file2,don,1
\\server1\folderA\file3,sue,1
\\server2\folderB\file1,don,1
What I'm attempting to do is create a script that will result in separate csv exports based on the paths in CSV1 such that the new files contain file values from CSV2 that match. For example, from the above, I'd end up with 2 results:
result1.csv
\\server1\folderA\file1,don,1
\\server1\folderA\file2,don,1
\\server1\folderA\file3,sue,1
result2.csv
\\server2\folderB\file1,don,1
Previously I've used a script lime this when the two values are exact:
$reportfile = import-csv $apireportoutputfile -delimiter ';' -encoding unicode
$masterlist = import-csv $pathlistfile
foreach ($record in $masterlist)
{
$path=$record.Path
$filename = $path -replace '\\','_'
$filename = '.\Working\sharefiles\' + $filename + '.csv'
$reportfile | where-object {$_.path -eq $path} | select FilePath,UserName,LastAccessDate,LogicalSize | export-csv -path $filename
write-host " Creating files list for $path" -foregroundcolor red -backgroundcolor white
}
however since the two path values are not the same, it returns nothing. I found a -like operator but am not sure how to use it in this code to get the results I want. where-object is a filter while -like ends up returning a true/false. Am I on the right track? Any ideas for a solution?
Something like this, maybe?
$ht = #{}
Import-Csv csv1.csv |
foreach { $ht[$_.path] = New-Object collections.arraylist }
Import-Csv csv2.csv |
foreach {
$path = $_.path | Split-Path -Parent
$ht[$path].Add($_) > $null
}
$i=1
$ht.Values |
foreach { if ($_.count)
{
$_ | Export-Csv "result$i.csv" -NoTypeInformation
$i++
}
}
My suggestion:
$1=ipcsv .\csv1.CSV
$2=ipcsv .\csv2.CSV
$equal = diff ($2|select #{n='PATH';e={Split-Path $_.PATH}}) $1 -Property PATH -IncludeEqual -ExcludeDifferent -PassThru
0..(-1 + $equal.Count) | %{%{$i = $_}{
$2 | ?{ (Split-Path $_.PATH) -eq $equal[$i].PATH } | epcsv ".\Result$i.CSV"
}}