I have a task that requires to scan the property of all the files indicated by certain directories where the files are stored. I need my code to read the following line of information separated by the delimiter "," stored in a .txt file as follows (the directory is made up by myself on my own device and I went ahead making up some blank .xlsx files to test my code:
Jakarta,C:\\temp\Hfolder,C:\temp\Lfolder
I currently have code that looks like this:
$LocContent = Import-Csv "C:\temp\Location.txt" # -Header $fileHeaders
ForEach($line in $LocContent){C:\temp\test1.csv -NoTypeInformation
#split fields into values
$line = $LocContent -split (",")
$country = $line[0]
$hDrivePath = $line[1]
$lDrivePath = $line[2]
Get-ChildItem $hDrivePath -force -include *.xlsx, *.accdb, *.accde, *.accdt, *.accdr -Recurse
Get-ChildItem $lDrivePath -force -include *.xlsx, *.accdb, *.accde, *.accdt, *.accdr -Recurse
? {
$_.LastWriteTime -gt (Get-Date).AddDays(-5)
}
Select-Object -Property Name, Directory, #{Name="Owner";Expression={(Get-ACL $_.Fullname).Owner}}, CreationTime, LastAccessTime, #{N="Location";E={$country}}, #{N='size in MB';E={$_.Length/1024kb}} | Export-Csv
}
However there is no output on the .csv file I assigned to output the information. What is wrong in my code?
Thanks!
There are several flaws within your code:
The Select has neither an -InputObject nor is anything piped to it so there can't be an output
You should decide whether you treat C:\temp\Location.txt as
a text file with Get-Contentand a split
or as a csv with headers
or without headers and supply them to the import.
The Get-ChildItem output isn't piped anywhere nor stored in a variable so it goes to the screen.
Export-Csv needs a file name to export to.
Try this untested script:
## Q:\Test\2018\06\26\SO_51038180.ps1
$fileHeaders = #('country','hDrivePath','lDrivePath')
$extensions = #('*.xlsx','*.accdb','*.accde','*.accdt','*.accdr')
$LocContent = Import-Csv "C:\temp\Location.txt" -Header $fileHeaders
$NewData = ForEach($Row in $LocContent){
Get-ChildItem $Row.hDrivePath,$Row.lDrivePath -Force -Include $extensions -Recurse |
Where-Object LastWriteTime -gt (Get-Date).AddDays(-5) |
Select-Object -Property Name,
Directory,
#{Name="Owner";Expression={(Get-ACL $_.Fullname).Owner}},
CreationTime,
LastAccessTime,
#{N="Location";E={$Row.country}},
#{N='size in MB';E={$_.Length/1024kb}}
}
# you choose what to do with the result uncomment the desired
$NewData | Format-Table -Auto
# $NewData | Out-Gridview
# $NewData | Export-Csv '.\NewData.csv' -NoTypeInformation
Related
I'm trying (badly) to work through combining CSV files into one file and prepending a column that contains the file name. I'm new to PowerShell, so hopefully someone can help here.
I tried initially to do the well documented approach of using Import-Csv / Export-Csv, but I don't see any options to add columns.
Get-ChildItem -Filter *.csv | Select-Object -ExpandProperty FullName | Import-Csv | Export-Csv CombinedFile.txt -UseQuotes Never -NoTypeInformation -Append
Next I'm trying to loop through the files and append the name, which kind of works, but for some reason this stops after the first row is generated. Since it's not a CSV process, I have to use the switch to skip the first title row of each file.
$getFirstLine = $true
Get-ChildItem -Filter *.csv | Where-Object {$_.Name -NotMatch "Combined.csv"} | foreach {
$filePath = $_
$collection = Get-Content $filePath
foreach($lines in $collection) {
$lines = ($_.Basename + ";" + $lines)
}
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 1}
}
$getFirstLine = $false
Add-Content "Combined.csv" $linesToWrite
}
This is where the -PipelineVariable parameter comes in real handy. You can set a variable to represent the current iteration in the pipeline, so you can do things like this:
Get-ChildItem -Filter *.csv -PipelineVariable File | Where-Object {$_.Name -NotMatch "Combined.csv"} | ForEach-Object { Import-Csv $File.FullName } | Select *,#{l='OriginalFile';e={$File.Name}} | Export-Csv Combined.csv -Notypeinfo
Merging your CSVs into one and adding a column for the file's name can be done as follows, using a calculated property on Select-Object:
Get-ChildItem -Filter *.csv | ForEach-Object {
$fileName = $_.Name
Import-Csv $_.FullName | Select-Object #{
Name = 'FileName'
Expression = { $fileName }
}, *
} | Export-Csv path/to/merged.csv -NoTypeInformation
I'm trying to merge CSV files in Powershell. I've read numerous answers here but I'm stuck on this problem.
I have a list of csv files, 2 difficulties :
[A] each file has a metadataline, the headers are in the second line.
[B] each file has the same structure, but sometimes quotes surround the column to escape the content.
Thanks to this question : Merging multiple CSV files into one using PowerShell,
I'm able to solve these two problems individually.
However, I'm stuck at combining the solutions.
Partial solution A
Skips every metadata line as well as header for subsequent files
Adapting the answer from kemiller2002:
$sourcefilefolderPath = "C:\CSV_folder"
$destinationfilePath = "C:\appended_files.csv"
$getHeader = $true
Get-ChildItem -Path $sourcefilefolderPath -Filter *.csv -Recurse| foreach {
$filePath = $_.FullName
$lines = $lines = Get-Content $filePath
$linesToWrite = switch($getHeader) {
$true {$lines | Select -Skip 1} # skips only the metadata line
$false {$lines | Select -Skip 2} # skips both the metadata line as well as headers
}
$getHeader = False
Add-Content $destination_file $linesToWrite
}
The problem : Import-Csv $destination_file give inconsistent results, as the quoting can be different for each source file.
Partial solution B
handles successfully random quoted columns
Solution provided by stinkyfriend.
Import-Csv seems to import the data gracefully when the column quoting, however different from one column to the other, is consistent for each line of the source file.
I could not combine this solution with the one above.
Get-ChildItem -Path $sourcefilefolderPath -File -Filter *.csv -Recurse |
Select-Object -ExpandProperty FullName |
Import-Csv |
Export-Csv $destination_file -NoTypeInformation -Append
Thanks a lot for your help !
Solution C
produces blank file on my PC
using suggestion from Mathias R. Jessen
Get-ChildItem -Path $sourcefilefolderPath -File -Filter *.csv -Recurse | foreach {
Write-Host $_.FullName |
Get-Content $_.FullName | Select-Object -Skip 1 | ConvertFrom-Csv |
Export-Csv $destinationfilePath -NoTypeInformation -Append
--- EDIT ---
RESULT
I could solve the problem by creating appended_files.csv using the first matching source file and then append to it.
$pattern_sourceFile = "*.csv*"
$list_files = Get-ChildItem -Path $sourcefilefolderPath -File -Recurse | Where {
$_FullName -match $pattern_sourcefile }
Get-Content $list_files[0].FullName |
Select-Object -Skip 1 | # skips metadataline
ConvertFrom-Csv | Export-Csv $destinationfilePath -NoTypeInformation
$list_files |
Select-Object -Skip 1 | # skips $array_files[0]
foreach { Get-Content $_.FullName |
Select-Object -Skip 1 | # skips metadata line
ConvertFrom-Csv |
Export-Csv $destinationfilePath -NoTypeInformation -Append }
Use ConvertFrom-Csv instead of Import-Csv, this way you can still control how many lines to skip:
Get-Content $file |Select -Skip 1 |ConvertFrom-Csv
So you'll end up with something like:
$sourcefilefolderPath = "C:\CSV_folder"
$destinationfilePath = "C:\appended_files.csv"
Get-ChildItem -Path $sourcefilefolderPath -Filter *.csv -Recurse | foreach {
Get-Content $_.FullName |Select-Object -Skip 1 |ConvertFrom-Csv |Export-Csv -Path $destinationfilePath -NoTypeInformation -Append
}
I've researched this and haven't been able to come up with a solid solution. Basically, I have a separate hard drive containing thousands of music files. I have a CSV list with the names of all the files that should be in the hard drive. Example:
My List
I want to be able to test if each of the files on my list exist in the hard drive, and if not, export it to a separate "missing files" list. The thing is each of the files in the hard drive exist under multiple folders.
As my script is now, I am trying to test if the path exists by using join-path. Here is my code right now - it's returning all of the files in the directory instead of just the missing files:
$documents = 'C:\Users\Me\Documents\ScriptTest'
$CSVexport = 'C:\Users\Me\Documents\ScriptTest\TestResults.csv'
$obj = #()
Write-host "`n_____Results____`n" #Write the results and export to a CSV file
$NewCsv = Import-CSV -Path 'C:\Users\Me\Documents\ScriptTest\Test.csv' |
Select-Object ID,'File Path' |
ForEach-Object {
if (!(Test-Path (Join-Path $documents $_.'File Path'))){
write-host "`n$($_.'File Path') is missing from the folder.`n"
$ObjectProperties = #{
ID = $_.ID
'File Path' = $_.'File Path'
}
$obj += New-Object PSObject -Property $ObjectProperties
}
}
$obj | export-csv $CSVexport -NoTypeInformation
How do I account for the sub-directories that vary with each file?
Edit - Resolved
$myFolder = 'C:\Users\Me\Documents\ScriptTest'
$CSVexport = 'C:\Users\Me\Documents\ScriptTest\Results.csv'
$csvPath = 'C:\Users\Me\Documents\ScriptTest\Test.csv'
$FileList = Get-ChildItem $myFolder -Recurse *.wav | Select-Object -ExpandProperty Name -Unique
Import-CSV -Path $csvPath |
Where-Object {$FileList -notcontains $_.'File Path'} |
export-csv $CSVexport -NoTypeInformation
You could generate a list of filenames from the recursed folders, then check if the file is in that list.
$documents = 'C:\Users\Me\Documents\ScriptTest'
$CSVexport = 'C:\Users\Me\Documents\ScriptTest\TestResults.csv'
$FileList = Get-ChildItem $documents -Recurse |
Where-Object { -not $_.PSIsContainer } |
Select-Object -ExpandProperty Name -Unique
Import-CSV -Path 'C:\Users\Me\Documents\ScriptTest\Test.csv' |
Where-Object {$FileList -notcontains $_.File} |
Export-CSV $CSVexport -NoTypeInformation
Edit: Answer updated to work with PowerShell 2.0 with suggestions from Bruce Payette and mklement0
I am working on creating a script that will read a .csv document containing a single column of filenames (one per cell) and search a larger folder for each of the files matching the filenames provided and identify the 'owner' using:
(get-acl $file).owner
Currently I have several bits of code that can do individual parts, but I am having a hard time tying it all together. Ideally, a user can simply input file names into the .csv file, then run the script to output a second .csv or .txt identifying each file name and it's owner.
csv formatting will appear as below (ASINs is header):
ASINs
B01M8N1D83.MAIN.PC_410
B01M14G0JV.MAIN.PC_410
Pull file names without header:
$images = Get-Content \\path\ASINs.csv | Select -skip 1
Find images in larger folder to pull full filename/path (not working):
ForEach($image in $images) {
$images.FullName | ForEach-Object
{
$ASIN | Get-ChildItem -Path $serverPath -Filter *.jpg -Recurse -ErrorAction SilentlyContinue -Force | Set-Content \\path\FullNames.csv
}
}
At that point I would like to use the full file paths provided by FullNames.csv to pull the owners from the files in their native location using the above mentioned:
(get-acl $file).owner
Does anyone have any ideas how to tie these together into one fluid script?
EDIT
I was able to get the following to work without the loop, reading one of the filenames, but I need it to loop as there are multiple filenames.
New CSV Format:
BaseName
B01LVVLSCM.MAIN.PC_410
B01LVY65AN.MAIN.PC_410
B01MAXORH6.MAIN.PC_410
B01MTGEMEE.MAIN.PC_410
New Script:
$desktopPath = [System.Environment]::GetFolderPath([System.Environment+SpecialFolder]::Desktop)
$images = $desktopPath + '\Get_Owner'
Get-ChildItem -Path $images | Select BaseName | Export-Csv $desktopPath`\Filenames.csv -NoTypeInformation
$serverPath = 'C:\Users\tuggleg\Desktop\Archive'
$files = Import-Csv -Path $desktopPath`\Filenames.csv
While($true) {
ForEach ($fileName in $files.BaseName)
{
Get-ChildItem -Path $serverPath -Filter "*$fileName*" -Recurse -ErrorAction 'SilentlyContinue' |
Select-Object -Property #{
Name='Owner'
Expression={(Get-Acl -Path $_.FullName).Owner}
},'*' |
Export-Csv -Path $desktopPath`\Owners.csv -NoTypeInformation
}
}
Any ideas on the loop issue? Thanks everyone!
This example assumes your csv contains partial filenames. It will search the filepath and filter for those partials.
Example.csv
"ASINs"
"B01M8N1D83.MAIN.PC_410"
"B01M14G0JV.MAIN.PC_410"
Code.ps1
$Files = Import-Csv -Path '.\Example.csv'
ForEach ($FileName in $Files.ASINs)
{
Get-ChildItem -Path $serverPath -Filter "*$FileName*" -Recurse -ErrorAction 'SilentlyContinue' |
Select-Object -Property #{
Name='Owner'
Expression={(Get-Acl -Path $_.FullName).Owner}
},'*' |
Export-Csv -Path '\\path\FullNames.csv' -NoTypeInformation
}
I wish to search for specific files listed in searchFiles and pipe their locations to TestFileLocation.CSV. However, my current script only generates an empty CSV. What am I missing?
My TestFindFile.csv is of the form:
Name
123.pdf
321.pdf
aaa.pdf
SNIPPET
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
ForEach($File in $searchFiles)
{
Get-ChildItem $source -Filter $File -rec | where {!$_.PSIsContainer} | select-object FullName | export-csv -notypeinformation -delimiter '|' -path c:\data\scripts\ps1\TestFileLocation.csv
}
You were overwriting the CSV for each iteration of the loop.
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
$outputPath = 'c:\data\scripts\ps1\TestFileLocation.csv'
$searchFiles | ForEach-Object {
# Silently continue to try to ignore error like
# not being able to read path's which are too long
Get-ChildItem $source -Filter $_ -rec -ErrorAction SilentlyContinue | where {!$_.PSIsContainer} | select-object FullName
} | export-csv -notypeinformation -delimiter '|' -path $outputPath
Example using AlphaFS
A comment asked for an example using AlphaFS because it claims to overcome the long path issue. I'm not going into all the details, but here is how I got it to work.
# download and unzip to c:\alpahfs
# dir C:\AlphaFS\* -Recurse -File | Unblock-File
[System.Reflection.Assembly]::LoadFrom('C:\AlphaFS\lib\net451\AlphaFS.dll')
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
$outputPath = 'c:\data\scripts\ps1\TestFileLocation.csv'
$searchFiles | ForEach-Object {
$files = [Alphaleonis.Win32.Filesystem.Directory]::EnumerateFiles($source,'*',[System.IO.SearchOption]::AllDirectories)
$files | ForEach-Object { [PSCustomObject] #{FileName = $_} }
} | export-csv -notypeinformation -delimiter '|' -path $outputPath
# type $outputPath
If your .csv file contains the header "Name", there is no need to again declare it when running Import-Csv.
The reason the output is empty is that you are searching for an Object which contains the property Name (imported from the TestFindFile.csv). Search for $File.Name. Also pull commands outside the loop that don't need to be there:
$searchFiles | Select -ExpandProperty Name | % {
Get-ChildItem $source -Filter $_ -Recurse | where {!$_.PSIsContainer}
} | select-object FullName | export-csv -notypeinformation -delimiter '|' -path c:\data\scripts\ps1\TestFileLocation.csv