PowerShell - searching for existing files generates empty output - powershell

I wish to search for specific files listed in searchFiles and pipe their locations to TestFileLocation.CSV. However, my current script only generates an empty CSV. What am I missing?
My TestFindFile.csv is of the form:
Name
123.pdf
321.pdf
aaa.pdf
SNIPPET
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
ForEach($File in $searchFiles)
{
Get-ChildItem $source -Filter $File -rec | where {!$_.PSIsContainer} | select-object FullName | export-csv -notypeinformation -delimiter '|' -path c:\data\scripts\ps1\TestFileLocation.csv
}

You were overwriting the CSV for each iteration of the loop.
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
$outputPath = 'c:\data\scripts\ps1\TestFileLocation.csv'
$searchFiles | ForEach-Object {
# Silently continue to try to ignore error like
# not being able to read path's which are too long
Get-ChildItem $source -Filter $_ -rec -ErrorAction SilentlyContinue | where {!$_.PSIsContainer} | select-object FullName
} | export-csv -notypeinformation -delimiter '|' -path $outputPath
Example using AlphaFS
A comment asked for an example using AlphaFS because it claims to overcome the long path issue. I'm not going into all the details, but here is how I got it to work.
# download and unzip to c:\alpahfs
# dir C:\AlphaFS\* -Recurse -File | Unblock-File
[System.Reflection.Assembly]::LoadFrom('C:\AlphaFS\lib\net451\AlphaFS.dll')
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
$outputPath = 'c:\data\scripts\ps1\TestFileLocation.csv'
$searchFiles | ForEach-Object {
$files = [Alphaleonis.Win32.Filesystem.Directory]::EnumerateFiles($source,'*',[System.IO.SearchOption]::AllDirectories)
$files | ForEach-Object { [PSCustomObject] #{FileName = $_} }
} | export-csv -notypeinformation -delimiter '|' -path $outputPath
# type $outputPath

If your .csv file contains the header "Name", there is no need to again declare it when running Import-Csv.
The reason the output is empty is that you are searching for an Object which contains the property Name (imported from the TestFindFile.csv). Search for $File.Name. Also pull commands outside the loop that don't need to be there:
$searchFiles | Select -ExpandProperty Name | % {
Get-ChildItem $source -Filter $_ -Recurse | where {!$_.PSIsContainer}
} | select-object FullName | export-csv -notypeinformation -delimiter '|' -path c:\data\scripts\ps1\TestFileLocation.csv

Related

Run Powershell script on multiple files

I have the following code to add new columns to a csv file. I would like to amend the code to run on multiple csv files within a folder and output it to a different folder.
$source = "C:\input_folder\input.csv"
$destination = "C:\output_folder\output.csv"
(Import-CSV $source |
Select-Object *,#{Name='column1';Expression={'data1'}} |
Select-Object *,#{Name='column2';Expression={'data2'}} |
ConvertTo-csv -NoTypeInformation |
Select-Object -Skip 0) -replace '"' | Set-Content $destination
You could do something like this. You might also want to add a check that the files in the folder are .csv type and not something else.
$source = "C:\input_folder"
$destinationFolder = "C:\output_folder"
$folderContents = Get-ChildItem $source
foreach ($item in $folderContents) {
if (Test-Path -Path $item -PathType Leaf == True) {
(Import-CSV $item |
Select-Object *,#{Name='column1';Expression={'data1'}} |
Select-Object *,#{Name='column2';Expression={'data2'}} |
ConvertTo-csv -NoTypeInformation |
Select-Object -Skip 0) -replace '"' | Set-Content ("$destinationFolder\$item" + "_formatted.csv")
}
}

powershell / merging .txt-files into csv adding date and filename

it's me again, as i mentioned yesterday i'm new to Powershell (now 3 days) and i hope you can help me again.
What I want:
I want to merge different txt-files into one csv-file
PLUS every line which is added should start with the actual date (yyyy-mm-dd) and the filename.
Expectation_Image
WhatIamActuallyGetting_Image
So what I've got so far:
New-Item Shoppinglist_$(get-date -f yyyyMMdd_hhmm).csv -ItemType File
$txtFiles = Get-ChildItem -Name *.txt
$desiredColumns = 'Date','Filename','Substance','Information','Comment'
ForEach ($file in $txtFiles) {
$csv = Import-Csv -path $file -Delimiter "`t"
$outcsv=$csv | Select-Object $desiredColumns
#I Think the mistake is somewhere here, but i habe no idea to fix it. :(
Select-Object *, #{Name = 'Date'; Expression = {(Get-Date -format s)}}
Select-Object *, #{Name = 'Filename'; Expression = {(GetFileName)}}
$outcsv | Export-Csv Shoppinglist_$(get-date -f yyyyMMdd_hhmm).csv -NoTypeInformation -Delimiter ";" -Append
}
I hope there is someone outside in the world who can help me. :)
You are right to use calculated properties, but are overthinking this a bit.
Also, Get-ChildItem returns FileInfo or DirectoryInfo objects. (unless you specify switch -Name, in that case it returns only the names of the items in the path).
These objects have useful properties, such as FullName, Name, LastWriteTime, etc.
Since you only want files returned, you can use the -File switch.
This assumes both input files have the exact same columns as in your example:
# the folder where the input files are and where the output csv file should be saved
$path = 'D:\Test'
$today = '{0:yyyy-MM-dd}' -f (Get-Date)
$txtFiles = Get-ChildItem -Path $path -Filter '*.txt' -File
$csv = foreach ($file in $txtFiles) {
Import-Csv -Path $file.FullName -Delimiter "`t" |
Select-Object #{Name = 'Date'; Expression = {$today}},
#{Name = 'Filename'; Expression = {$file.Name}}, *
}
$fileOut = Join-Path -Path $path -ChildPath ('Shoppinglist_{0:yyyyMMdd_HHmm}.csv' -f (Get-Date))
$csv | Export-Csv -Path $fileOut -Delimiter ";" -NoTypeInformation
This assumes both input files have at least the 3 desired columns: 'Substance','Information' and 'Comment'
# the folder where the input files are and where the output csv file should be saved
$path = 'D:\Test'
$today = '{0:yyyy-MM-dd}' -f (Get-Date)
$txtFiles = Get-ChildItem -Path $path -Filter '*.txt' -File
$csv = foreach ($file in $txtFiles) {
Import-Csv -Path $file.FullName -Delimiter "`t" |
Select-Object #{Name = 'Date'; Expression = {$today}},
#{Name = 'Filename'; Expression = {$file.Name}},
Substance, Information, Comment
}
$fileOut = Join-Path -Path $path -ChildPath ('Shoppinglist_{0:yyyyMMdd_HHmm}.csv' -f (Get-Date))
$csv | Export-Csv -Path $fileOut -Delimiter ";" -NoTypeInformation
If you are using a PowerShell version below 3.0, you cannot use the -File switch. Instead then use: $txtFiles = Get-ChildItem -Path $path -Filter '*.txt' | Where-Object { !$_.PSIsContainer }

Merge multiple csv files, ignoring metadata line and handling different quotings

I'm trying to merge CSV files in Powershell. I've read numerous answers here but I'm stuck on this problem.
I have a list of csv files, 2 difficulties :
[A] each file has a metadataline, the headers are in the second line.
[B] each file has the same structure, but sometimes quotes surround the column to escape the content.
Thanks to this question : Merging multiple CSV files into one using PowerShell,
I'm able to solve these two problems individually.
However, I'm stuck at combining the solutions.
Partial solution A
Skips every metadata line as well as header for subsequent files
Adapting the answer from kemiller2002:
$sourcefilefolderPath = "C:\CSV_folder"
$destinationfilePath = "C:\appended_files.csv"
$getHeader = $true
Get-ChildItem -Path $sourcefilefolderPath -Filter *.csv -Recurse| foreach {
$filePath = $_.FullName
$lines = $lines = Get-Content $filePath
$linesToWrite = switch($getHeader) {
$true {$lines | Select -Skip 1} # skips only the metadata line
$false {$lines | Select -Skip 2} # skips both the metadata line as well as headers
}
$getHeader = False
Add-Content $destination_file $linesToWrite
}
The problem : Import-Csv $destination_file give inconsistent results, as the quoting can be different for each source file.
Partial solution B
handles successfully random quoted columns
Solution provided by stinkyfriend.
Import-Csv seems to import the data gracefully when the column quoting, however different from one column to the other, is consistent for each line of the source file.
I could not combine this solution with the one above.
Get-ChildItem -Path $sourcefilefolderPath -File -Filter *.csv -Recurse |
Select-Object -ExpandProperty FullName |
Import-Csv |
Export-Csv $destination_file -NoTypeInformation -Append
Thanks a lot for your help !
Solution C
produces blank file on my PC
using suggestion from Mathias R. Jessen
Get-ChildItem -Path $sourcefilefolderPath -File -Filter *.csv -Recurse | foreach {
Write-Host $_.FullName |
Get-Content $_.FullName | Select-Object -Skip 1 | ConvertFrom-Csv |
Export-Csv $destinationfilePath -NoTypeInformation -Append
--- EDIT ---
RESULT
I could solve the problem by creating appended_files.csv using the first matching source file and then append to it.
$pattern_sourceFile = "*.csv*"
$list_files = Get-ChildItem -Path $sourcefilefolderPath -File -Recurse | Where {
$_FullName -match $pattern_sourcefile }
Get-Content $list_files[0].FullName |
Select-Object -Skip 1 | # skips metadataline
ConvertFrom-Csv | Export-Csv $destinationfilePath -NoTypeInformation
$list_files |
Select-Object -Skip 1 | # skips $array_files[0]
foreach { Get-Content $_.FullName |
Select-Object -Skip 1 | # skips metadata line
ConvertFrom-Csv |
Export-Csv $destinationfilePath -NoTypeInformation -Append }
Use ConvertFrom-Csv instead of Import-Csv, this way you can still control how many lines to skip:
Get-Content $file |Select -Skip 1 |ConvertFrom-Csv
So you'll end up with something like:
$sourcefilefolderPath = "C:\CSV_folder"
$destinationfilePath = "C:\appended_files.csv"
Get-ChildItem -Path $sourcefilefolderPath -Filter *.csv -Recurse | foreach {
Get-Content $_.FullName |Select-Object -Skip 1 |ConvertFrom-Csv |Export-Csv -Path $destinationfilePath -NoTypeInformation -Append
}

Export to CSV using PowerShell 2.0?

I have the following code that prints the file system rights of each account enabled on the folder with path "C:\Temp\CSM\*" & "C:\Temp\CSM\*\*" . How do I write the output in a comma-separated CSV? As this is for PowerShell 2.0 Export-Csv -Append parameter cannot be used.
$FolderPath = dir -Directory -Path "C:\Temp\CSM\*", "C:\Temp\CSM\*\*" -Force
foreach ($Folder in $FolderPath) {
$Acl = Get-Acl -Path $Folder.FullName
foreach ($Access in $acl.Access) {
Write-Host $Folder.FullName "," $Access.IdentityReference "," $Access.FileSystemRights
}
}
Prior to PowerShell v3 if you wanted to append to an existing CSV you need something like this:
... | ConvertTo-Csv -NoType | Select-Object -Skip 1 | Add-Content
However, in your scenario that probably isn't necessary. If you replace your foreach loops with a pipeline you can write the CSV in one go without having to append to it in a loop (which isn't recommended anyway).
$folders = "C:\Temp\CSM\*", "C:\Temp\CSM\*\*"
Get-ChildItem -Path $folders -Directory -Force | ForEach-Object {
$path = $_.FullName
Get-Acl -Path $path |
Select-Object -Expand Access |
Select-Object #{n='Path';e={$path}}, IdentityReference, FileSystemRights
} | Export-Csv 'C:\output.csv' -NoType

Scan the properties of files on an excel sheet by Powershell

I have a task that requires to scan the property of all the files indicated by certain directories where the files are stored. I need my code to read the following line of information separated by the delimiter "," stored in a .txt file as follows (the directory is made up by myself on my own device and I went ahead making up some blank .xlsx files to test my code:
Jakarta,C:\\temp\Hfolder,C:\temp\Lfolder
I currently have code that looks like this:
$LocContent = Import-Csv "C:\temp\Location.txt" # -Header $fileHeaders
ForEach($line in $LocContent){C:\temp\test1.csv -NoTypeInformation
#split fields into values
$line = $LocContent -split (",")
$country = $line[0]
$hDrivePath = $line[1]
$lDrivePath = $line[2]
Get-ChildItem $hDrivePath -force -include *.xlsx, *.accdb, *.accde, *.accdt, *.accdr -Recurse
Get-ChildItem $lDrivePath -force -include *.xlsx, *.accdb, *.accde, *.accdt, *.accdr -Recurse
? {
$_.LastWriteTime -gt (Get-Date).AddDays(-5)
}
Select-Object -Property Name, Directory, #{Name="Owner";Expression={(Get-ACL $_.Fullname).Owner}}, CreationTime, LastAccessTime, #{N="Location";E={$country}}, #{N='size in MB';E={$_.Length/1024kb}} | Export-Csv
}
However there is no output on the .csv file I assigned to output the information. What is wrong in my code?
Thanks!
There are several flaws within your code:
The Select has neither an -InputObject nor is anything piped to it so there can't be an output
You should decide whether you treat C:\temp\Location.txt as
a text file with Get-Contentand a split
or as a csv with headers
or without headers and supply them to the import.
The Get-ChildItem output isn't piped anywhere nor stored in a variable so it goes to the screen.
Export-Csv needs a file name to export to.
Try this untested script:
## Q:\Test\2018\06\26\SO_51038180.ps1
$fileHeaders = #('country','hDrivePath','lDrivePath')
$extensions = #('*.xlsx','*.accdb','*.accde','*.accdt','*.accdr')
$LocContent = Import-Csv "C:\temp\Location.txt" -Header $fileHeaders
$NewData = ForEach($Row in $LocContent){
Get-ChildItem $Row.hDrivePath,$Row.lDrivePath -Force -Include $extensions -Recurse |
Where-Object LastWriteTime -gt (Get-Date).AddDays(-5) |
Select-Object -Property Name,
Directory,
#{Name="Owner";Expression={(Get-ACL $_.Fullname).Owner}},
CreationTime,
LastAccessTime,
#{N="Location";E={$Row.country}},
#{N='size in MB';E={$_.Length/1024kb}}
}
# you choose what to do with the result uncomment the desired
$NewData | Format-Table -Auto
# $NewData | Out-Gridview
# $NewData | Export-Csv '.\NewData.csv' -NoTypeInformation