Import-Csv and Export-Csv with same name and path - powershell

This should be a fairly simple question to answer, but I haven't been able to find a resource online. I need to remove rows from a csv with out changing the filename or path. Currently I am trying to import the file, filter the items I need and export that data to the same location with the same file name:
foreach ($csv in $csvLists){
Import-Csv $csv | Where-Object {[DateTime]$_.'Date' -ge $2WeeksOld} | Export-Csv $csv -NoType -Force
}
Also tried:
foreach ($csv in $csvLists){
Import-Csv $csv | Where-Object {[DateTime]$_.'Date' -ge $2WeeksOld} | Export-Csv $csv.PSParentPath -NoType -Force
}
I am getting access denied errors:
Export-Csv : Access to the path 'D:\Test' is denied.
At C:\script.ps1:20 char:71
+ ... .'Date' -ge $2WeeksOld} | Export-Csv $csv.PSParentPath -NoType -Force
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OpenError: (:) [Export-Csv], UnauthorizedAccessException
+ FullyQualifiedErrorId : FileOpenFailure,Microsoft.PowerShell.Commands.ExportCsvCommand
What am I missing here? Is the issue that powershell is reading the file and cant export over it at the same time? Should I try setting the imported data as a variable and then exporting the csv outside the import-csv pipeline?
As always thanks for any help.

The problem here is that you're trying to do everything in one line, and consequently you're opening a file (which locks it) and trying to overwrite the same file (which needs to lock it) at the same time. That won't work.
Try:
foreach ($csv in $csvLists){
$Data = Import-Csv $csv | Where-Object {[DateTime]$_.'Date' -ge $2WeeksOld}
$Data | Export-Csv $csv -NoType -Force
}
Or, if $csvLists is an array of FileInfo objects (such as from the output of Get-ChildItem):
foreach ($csv in $csvLists){
$Data = Import-Csv $csv.FullName | Where-Object {[DateTime]$_.'Date' -ge $2WeeksOld}
$Data | Export-Csv $csv.FullName -NoType -Force
}

Use $csv.FullName. This contains the full path for a file/folder.
Here are a few common properties of System.IO.FileInfo:
Property Contents
BaseName MyFile
Directory* <returns System.IO.DirectoryInfo object of "C:\Path\To">
Name MyFile.txt
FullName C:\Path\To\MyFile.txt
PSParentPath Microsoft.PowerShell.Core\FileSystem::C:\Path\To
I'm not aware of a use case for .PSParentPath. Looking at the content, I assume it's used when you need to specify the PSProvider as well as the string value of the parent. Otherwise you could just use .Directory.Name
You may find the class definitions for FileInfo and DirectoryInfo useful. They are the objects returned by Get-ChildItem. The links list Methods and Properties; they do not list Code/Note/ScriptProperty types.
tip: run $csv | Get-Member to see members of this object, look at MemberType.
* For System.IO.DirectoryInfo objects, use Parent. Have got some different behaviour in Win10 and am sure an alias was instroduced so that Parent works for FileInfo or Directory works for DirectoryInfo.

Related

List all files with size using powershell char limit issue

I have an applications folder that have more than 10 applications in. I want to list all files (including sub-folders) with directory and size info and save it under each application folder.
Here is the script (it is in C:\Users\xxxxxx\Desktop\apps)
$FolderList = Get-ChildItem -Directory
foreach ($folder in $FolderList)
{
$thisPath = Get-Location
Get-ChildItem -File -Filter * -Path $folder -Recurse |
Sort-Object -Property Length -Descending|
Select-Object -Property FullName, Length, #{l='Length (MB)';e={$_.Length/1MB}} |
Format-Table -AutoSize |
Out-File $thisPath\fileList_$folder.txt
}
Output:
FullName - Length - Length (MB)
C:\Users\xxxxxx\Desktop\apps\3\VSCodeUserSetup-x64-1.62.2.exe 79944464 76.2409820556641
C:\Users\xxxxxx\Desktop\apps\3\son.zip 18745870 17.8774547576904
It does what I want but in some of the outputs where the path is too long it didn't write the length or even full path.
FullName
C:\Users\xxxxxx\Desktop\apps\3\xxxxxxxxxxx/xxxxxx/xxxxxxxxxxxxxx/xxxxxxxxxxx/VSCodeUserSetu...
C:\Users\xxxxxx\Desktop\apps\3\son.zip
As I searched I saw that there is a char limit. How can I overcome this issue?
The -Path parameter is defined as a string array, but you are feeding it a DirectoryInfo object.
The second part of your question is about truncation, which happens because you use Format-Table -AutoSize on the data.
Format-* cmdlets are designed to display output on the console window only which has a limited width and all items longer are truncated.
Rule of Thumb: never use Format-* cmdlets when you need to process the data later.
Instead of saving a formatted table to a text file, I would suggest saving the (object) info as structured CSV file which makes working with the data later MUCH easier and without truncation. (you can for instance open it in Excel)
# just get an array of Full pathnames
$FolderList = (Get-ChildItem -Directory).Name #instead of Fullname
foreach ($folder in $FolderList) {
# create the path for the output
$fileOut = Join-Path -Path $folder -ChildPath ('filelist_{0}.csv' -f $folder)
Get-ChildItem -File -Path $folder -Recurse |
Sort-Object -Property Length -Descending |
Select-Object -Property FullName, Length, #{l='Length (MB)';e={$_.Length/1MB}} |
Export-Csv -Path $fileOut -NoTypeInformation -UseCulture
}
I added switch -UseCulture to the Export-Csv cmdlet so you can afterwards simply double-click the csv file to open it in your Excel

How to add a string to the output from powershell

I use this to get a list of folders containing .h files.
**
$type = "*.h"
$HDIRS = dir .\$type -Recurse |
Select-Object Directory -Unique |
Format-Table -HideTableHeaders
** It gives me a list of folders.
Now I want "-I " before every foldername. Common string manipulation doesn't seem to work
You still have rich objects after the select so to manipulate the strings you have to reference the one property you've selected "Directory"
$type = "*.h"
$HDIRS = Dir .\$type -Recurse |
Select-Object Directory -Unique |
ForEach-Object{
$_.Directory = "-I" + $_.Directory
$_
} |
Format-Table -HideTableHeaders
This will result in $HDIRS looking like a list of folder paths like -IC:\temp\something\something...
However, the format output objects are generally not suitable for further consumption. It looks like you're interested in the strings you could simply make this a flat array of strings like:
$type = "*.h"
$HDIRS = Dir .\$type" -Recurse |
Select-Object -ExpandProperty Directory -Unique |
ForEach-Object{ "-I" + $_ }
The mantra goes filter left, format right.
Our data:
$type = '.\*.h'
Get-ChildItem -Path $type -Recurse
The manipulation (filter):
Select-Object { "-I $($_.Directory)" } -Unique
And the format:
Format-Table -HideTableHeaders
In many cases, PowerShell cmdlets allow you to pass scriptblocks (closures) to evaluate to values and that's what I'm doing above with the Select-Object call.

Auditing Permissions with Powershell

A little background first: I've been asked as part of a Scrum team at work to prepare a Powershell script that will collate a list of SMB Shares that have been flagged as safe to archive from the CSV files that were given to me nested in a folder structure (each share equates to one CSV file, with a list of files within, but the only relevance for this is that they are titled SHARENAME.csv), and the current permissions of each, in case they need to be restored at a later date.
At the moment, I have the following for the first step (nice and easy):
get-childitem -path 'C:\users\adam.lane\desktop\_with adam\_remove' | select BaseName | export-csv -path "..\test.csv"
This gives me a single CSV file with the names of all of the 188 shares in a single column. The second step, exporting a CSV file for each with a list of permissions, is not going as well. So far I have the following:
$share = Import-Csv -path 'C:\users\adam.lane\desktop\_with adam\test.csv' -Header 'BaseName'
ForEach ($share in $share) {
get-smbshareaccess -name $share -cimsession euukpopdfs005
}
Obviously at the moment there's no export-csv command in there, but essentially I'll want the script to use the same share name that it calls with 'get-smbshareaccess -name' as the file name for the CSV file. Unfortunately I hit 188 errors in the following format at this point:
get-smbshareaccess : euukpopdfs005: No MSFT_SMBShare objects found with property 'Name' equal to '#{BaseName=Brand}'. Verify the value of the property and retry.
At line:3 char:1
+ get-smbshareaccess -name $share -cimsession euukpopdfs005
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (#{BaseName=Brand}:String) [Get-SmbShareAccess], CimJobException
+ FullyQualifiedErrorId : CmdletizationQuery_NotFound_Name,Get-SmbShareAccess
+ PSComputerName : euukpopdfs005
So rather than calling the BaseName (for instance "Brand" in this first case), it's calling (#{BaseName=Brand}:String), which it then doesn't find anything for.
Is there anyway I can tidy this up? Ideally in some way that will work for both the Get-SmbShareAccess command and the export-csv command, but I'm not against doing a little more jiggery pokery as needed.
This is my first post, so I apologise if this is too long, lacking in detail, etc etc. Let me know if you need anything else! Thanks in advance.
Final (working) code after suggestions:
get-childitem -path 'C:\users\adam.lane\desktop\_with adam\_remove' | select BaseName | export-csv -path "..\test.csv"
$share = Import-Csv -path 'C:\users\adam.lane\desktop\_with adam\test.csv' -Header 'BaseName'
$share | Format-List
ForEach ($item in $share) {
get-smbshareaccess -name $item.BaseName -cimsession euukpopdfs005
}
Final working code (Thanks everyone!):
get-childitem -path 'C:\users\adam.lane\desktop\_with adam\_remove' | select BaseName | export-csv -path "..\test.csv"
$share = Import-Csv -path 'C:\users\adam.lane\desktop\_with adam\test.csv' -Header 'BaseName'
$share | Format-List
ForEach ($item in $share) {
get-smbshareaccess -name $item.BaseName -cimsession euukpopdfs005
}

Powershell - Out of Memory Error When Running Script Against Large Directory

So I have a script that gets the filename of songs contained in a CSV list and checks a directory to see if the file exists, then exports the missing information if there is any. The CSV file looks something like this:
Now, my script seems to work when I test on a smaller directory but when I run it against my actual directory contained on an external drive (about 10TB of files), I get a "system.outofmemoryexception" error before the script can complete.
$myPath = 'Z:\Music\media'
$myCSV = 'C:\Users\Me\Documents\Test.csv'
$CSVexport = 'C:\Users\Me\Documents\Results.csv'
$FileList = Get-ChildItem $myPath -Recurse *.wav | Select-Object -ExpandProperty Name -Unique
Import-CSV -Path $myCSV |
Where-Object {$FileList -notcontains $_.Filename} |
Select ID, AlbumTitle, TrackNo, Filename | Export-CSV $CSVexport -NoTypeInformation
$missing = Import-CSV $CSVexport | Select-Object -ExpandProperty Filename
If(!([string]::IsNullOrEmpty($missing))){
Write-Output "Missing files:`n" $missing}
Is there a way to make this script consume less memory or a more efficient way to do this against a large directory of files? I am new to Powershell scripting and am having trouble finding a way around this.
When #TheIncorrigible says iteratively he means something like this. Please note I am using different file paths since I don't have Z: drive. The best way would be to load up your csv items in a variable then iterate through that variable using a foreach loop, then for each one of those items testing to see if file exist, then if it does not add that item to a new variable. Once complete then export the new variable containing the missing items to csv.
$myPath = "C:\temp\"
$myCsv = "C:\temp\testcsv.csv"
$CSVexport = "C:\temp\results.csv"
$CsvItems = Import-Csv -Path $myCsv
$MissingItems
foreach($item in $CsvItems)
{
$DoesFileExist = Test-Path ($myPath + $item.Filename)
If($DoesFileExist -eq $false)
{
$MissingItems = $MissingItems + $item
}
}
$MissingItems | Export-Csv $CSVexport -NoTypeInformation

PowerShell - Copy specific lines from multiple specific nested files

So, the folder structure looks like this:
RootFolder
RootFolder
SubFolder1
SubSubFolder1
totals.txt
SubSubFolder2
totals.txt
SubFolder2
SubSubFolder1
totals.txt
SubSubFolder2
totals.txt
What I want to do is recursively walk through these Subfolders for the totals.txt file. Read content, and copy lines 22,26,30,34,38,and 42 (with first line being 0 not 1)into a single combined file.
I started with this code:
Get-ChildItem -Recurse | Where-Object {$_ -like "totals.txt" } | Get-Content | Select-Object -Index 22,26,30,34,38,42 | Add-Content "DataInportFile.txt"
However this only finds RootFolder\SubFolder\SubSubFolder\totals.txt and then exits script. Not what I'm looking for...
What I need is the above script to continue searching recursively for the next file and next until all directories have been searched in structure. So I used this:
Get-ChildItem -Recurse | ForEach-Object {$_ -like "totals.txt" } | Get-Content | Select-Object -Index 22,26,30,34,38,42 | Add-Content "DataInportFile.txt"
However, this script errors
C:\Users\user1\scripts\Untitled1.ps1:1 char:69
+ ... urse | ForEach-Object {$_ -like "totals.txt" } | $_.filename #| Get- ...
+ ~~~~~~~~~~~
Expressions are only allowed as the first element of a pipeline.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : ExpressionsMustBeFirstInPipeline
I could use some help figuring out the rest of this powershell script. So close but no joy. Thanks for the help.
Realized I needed to walk through the array
For other's reference, this code works as required:
Get-ChildItem -Recurse | Where-Object {$_ -like "totals.txt" } |
%{
(Get-Content $_.FullName) | Select-Object -Index 22,26,30,34,38,42 | Add-Content "DataInportFile.txt"
}