Powershell - Export Multiple CSV's into unique folders - powershell

I have been working on a PowerShell script for the better part of well a week or two. I've been able to get some parts of it working however I'm unable to fully get this automated.
I deal with a lot of CSV files on a daily basis, I have been tasked with uploading them into our software however sometimes they're too large to handle so I break them down based upon their "type" (it's a column in the CSV) and I export it to a single CSV per "type". I've been able to accomplish this with the following:
$file = gci -Filter "*.csv";
Import-Csv $file `
| Group-Object –Property “type” `
| Foreach-Object `
{
$path=$_.name+”.csv” ; $_.group `
| Export-Csv –Path $path –NoTypeInformation
}
So this works wonderfully, for each individual CSV. Unfortunately I don't have the time to do this for each individual CSV. Now I come to my other PowerShell script:
get-childitem -Filter "*.csv" `
| select-object basename `
| foreach-object{ $path=$_.basename+".csv" #iterate through files.
if(!(Test-Path -path $_.basename)) #If the folder of the file can't be found then it will attempt to create it.
{
New-Item $_.basename -type directory; $file=$_.basename+".csv";
Import-Csv $file `
| Group-Object -Property "Type" `
| Foreach-Object {
$path=$_.name+".csv"; $_.group `
| `
if(!(Test-Path -path $path2))
{
New-Item $path2 -type directory
Export-Csv -Path $path2 + "\" + $path -NoTypeInformation
}
else
{
"Failed on: " + $_.basename
#Export-Csv -Path $_.basename + "\" + $path -NoTypeInformation
}
}
}
else
{
Import-Csv $path `
| Group-Object -Property "Type" `
| Foreach-Object {$path=$_.basename+".csv" ; $_.group
if(Test-Path -path $._)
{
New-Item $path2 -type directory
Export-Csv -Path $path2 + "\" + $path -NoTypeInformation
}
#else
#{
Write-Host "Failed on: $_.basename"
#Export-Csv -Path $_.basename + "\" + $path -NoTypeInformation
#}
}
}
}
I just can't wrap my head around "why" this isn't working effectively. I have two conditionals. Is there a folder for the CSV? If no create one. I have to have another one because one of the "types" contains a \ which errors out if I don't have the folder, so I automatically try to create it. When I run the script I get the Path is null.
The Error is:
The term ' ' is not recognized as the name of a cmdlet, function,
script file, or operable program. Check the spelling of the name, or
if a path was included, verify that the path is correct and try again.
At C:\Users\c.burkinshaw\foldermake.ps1:11 char:26
+ | ` <<<<
+ CategoryInfo : ObjectNotFound: ( :String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
Test-Path : Cannot bind argument to parameter 'Path' because it is null.
At C:\Users\c.burkinshaw\foldermake.ps1:12 char:45
+ if(!(Test-Path -path <<<< $path2))
+ CategoryInfo : InvalidData: (:) [Test-Path], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.TestPathCommand
Any help would be greatly appreciated, if you have questions please don't hesitate to ask.

You have not defined $path2 anywhere, so something like test-path -path $path2 will say path is null. And in one place you are using $._ which will again give errors.
Edit after question updated with error message:
Your error message also says the same
Test-Path : Cannot bind argument to parameter 'Path' because it is
null. At C:\Users\c.burkinshaw\foldermake.ps1:12 char:45
+ if(!(Test-Path -path <<<< $path2))
Also the other error is in:
$path=$_.name+".csv"; $_.group `
| `
what are you trying to do here with the $_.group?
It is not proper. You cannot do $_.group | and provide some if statement.
Other comments:
Why are using $_.basename and then appending .csv? You could have just used $_.name. Try to not use the select-object basename - I don't see the value.
Extract the common import-csv and export-csv part into a function.

Related

storing filename and path in variable to use with Copy-Item

I'm trying to write a script to find the most recent .bak in a certain directory and copy the file to another location. When I use Get-ChildItem the file name isn't enlcosed in single quotes so when i try to copy it Copy-Item cant find it. ( I think)
$dir = 'E:\Backups\'
$dest = 'F:\'
$filename = Get-ChildItem -path $dir -filter *.bak | Sort-Object CreationTime -Descending | Select-Object -First 1 | select FullName | Format-Table -HideTableHeaders
echo #filename
copy-Item -path #filename -destination #dest
echo #filename returns E:\Backups\company.bak but i think need 'E:\Backups\company.bak' for it to work?
PS C:\Users\prodadmin> copy-Item -path #filename -destination #dest
Copy-Item : A positional parameter cannot be found that accepts argument 'Microsoft.PowerShell.Commands.Internal.Format.GroupStartData'.
At line:1 char:1
+ copy-Item -path #filename -destination #dest
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Copy-Item], ParameterBindingException
+ FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.CopyItemCommand
As above, typed it all in one go.
Get-ChildItem returns FileInfo (and also DirectoryInfo) objects.
By using select FullName you tell PowerShell to return an new object with one property called FullName and you lose the Name property.
Format-Table is a cmdlet used for display purposes only, so never use that on data you want to process further.
Lastly, you prefix the variables you have with # instead of $
Try
$dir = 'E:\Backups\'
$dest = 'F:\'
$file = Get-ChildItem -Path $dir -Filter '*.bak' -File | # do not return directories
Sort-Object CreationTime -Descending | # or did you mean LastWriteTime (=> last modified date) ?
Select-Object -First 1 # return just 1 FileInfo object
Write-Host "Found $($file.Name)" # write to console
Copy-Item -Path $file.FullName -Destination $dest

Unable to work with lines i csv created by export-csv

After running this script:
# Input path
$input_path = 'C:\Users\larry\Pictures\2003'
# Export file
$documents_export_file = 'C:\Users\larry\Documents\Temp\ffmpeg\fullnames.csv'
# Copy-to folder
$copy_to = "C:\Users\larry\Documents\Temp\ffmpeg\movies"
# List filenames in $documents_export_file
Get-ChildItem -Attributes !Directory -Path $input_path -Recurse -force | select-object -Property fullName | Export-Csv $documents_export_file
# Copy the files listed in $documents_export_file to $copy_to folder
get-content $documents_export_file | Foreach-Object { copy-item -Path $_ -Destination $copy_to}
I get the following error:
copy-item : Cannot find drive. A drive with the name '"C' does not exist.
At line:18 char:55
+ ... ort_file | Foreach-Object { copy-item -Path $_ -Destination $copy_to}
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: ("C:String) [Copy-Item], DriveNotFoundException
+ FullyQualifiedErrorId : DriveNotFound,Microsoft.PowerShell.Commands.CopyItemCommand
copy-item : Cannot find drive. A drive with the name '"C' does not exist.
At line:18 char:55
+ ... ort_file | Foreach-Object { copy-item -Path $_ -Destination $copy_to}
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: ("C:String) [Copy-Item], DriveNotFoundException
+ FullyQualifiedErrorId : DriveNotFound,Microsoft.PowerShell.Commands.CopyItemCommand
As far as I can figure out, it has something to do with my Export-Csv, bur I can´t figure it out.
Please give me a hand.
There's no need to export the list of files to CSV in the first place - assign the output from Get-ChildItem to a variable instead, and then pipe the contents directly to Copy-Item:
$files = Get-ChildItem -Attributes !Directory -Path $input_path -Recurse -Force
$files |Copy-Item -Destination $copy_to

Search directory for unique path and patterns

I am trying to search a directory structure, and files for all instances of where a pattern exists. Than I want that file location recorded in a log file that I can review latter. I looked at various posts, but I have not found a similar example where this is happening. Reviewed posts include:
PowerShell Scripting - Get-ChildItem
Search List for unique pattern
Search directory and sub-directories for pattern in a file
Use an Easy PowerShell Command to Search Files for Information
Get full path of the files in PowerShell
Here is the code I am using to recuse through the folder structure:
#Set variables for paths
$Results = "C:\Results"
$Source = "C:\Test\*"
$Destination = "C:\MyTest\"
#Create file name for each report with date and time of run
$ReportDate = (Get-Date).ToString("dd-MM-yyyy-hh-mm-ss")
$CustomPattern = Read-Host 'What pattern are you looing for?'
$CustomPatternLog = New-Item -itemType File -Path C:\Results -Name $("CustomerPattern_" + $ReportDate + ".txt")
$CustomPattern = foreach ($file in Get-ChildItem -Path $Destination -Recurse | Select-String -pattern $CustomPattern | Select-Object -Unique Path) {$file.path}
$CustomPattern > "$($Results)\$($CustomPatternLog)"
However, this code is returning the following error:
Get-ChildItem : The specified path, file name, or both are too long.
The fully qualified file name must be less than 260 characters, and
the directory name must be less than 248 characters. At line:19
char:36
+ $CustomPattern = foreach ($file in Get-ChildItem -Path $Destination -Recurse | S ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ReadError: (C:\Test\Mor...ofiles\Customer:St ring) [Get-ChildItem],
PathTooLongException
+ FullyQualifiedErrorId : DirIOError,Microsoft.PowerShell.Commands.GetChil dItemCommand
Do you have a better way to do the same operation?
replace this
$CustomPatternLog = New-Item -itemType File -Path C:\Results -Name $("CustomerPattern_" + $ReportDate + ".txt")
$CustomPattern = foreach ($file in Get-ChildItem -Path $Destination -Recurse | Select-String -pattern $CustomPattern | Select-Object -Unique Path) {$file.path}
$CustomPattern > "$($Results)\$($CustomPatternLog)"
with this
$files = Get-ChildItem -Path $Destination -Recurse
#in case you would need the path replace FullName with PsParentPath
$result = ($files | ?{$_.name -like "*$CustomPattern*"}).FullName
$result | out-file ($CustomPattern + "_" + $ReportDate + ".txt")
and since its shell you can do the same with one liner
(Get-ChildItem -Path $Destination -Recurse | ?{$_.name -like "*$CustomPattern*"}).FullName | out-file ($CustomPattern + "_" + $ReportDate + ".txt")

Powershell to Delete files based on CSV

I am trying to delete bunch of files based on a csv file. Files listed in the csv file should be deleted from the target file share. When I run the script I am in the target folder but I am getting the following error:
Remove-Item : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties
do not match any of the parameters that take pipeline input.
At line:9 char:56
+ Get-Childitem | where {$_.Name -match $filename} | Remove-Item -verbose -$fi ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (gp.html:PSObject) [Remove-Item], ParameterBindingException
+ FullyQualifiedErrorId : InputObjectNotBound,Microsoft.PowerShell.Commands.RemoveItemCommand
Code below:
$csvFile = Import-csv "C:\\DeleteFiles.csv"
foreach($user in $csvFile)
{
$filename = $user.FileName
write-host $filename
Get-Childitem | where {$_.Name -match $filename} | Remove-Item -verbose -$fileName
}
According to your comment, you have some empty filenames in the csv file. This solution should be more robust:
$csvFile = Import-csv "C:\\DeleteFiles.csv"
foreach($user in $csvFile)
{
$filename = $user.FileName
write-host "The filename is '$filename'"
if (test-path $filename)
{
Remove-Item -verbose $fileName
}
else
{
Write-Host "File '$filename' not found"
}
}

Get data from CSV to delete users folders from multiple servers

I am very new to powershell and situation is that I have some unneeded users folder that must be deleted on different servers. Path on all servers is f.e. "\server1\hiddenshare$\username" My CSV looks like:
ServerName,FolderName
SERVER1,AAAA
SERVER2,AAA1
SERVER3,AAA2
And I am trying to run this code:
$path = "C:\temp\servers_folders.csv"
$ServerName = Import-Csv -Path $path
$FolderName = Import-CSV -Path $path
Import-Csv -Path $path | % {
Remove-Item -Path \\$ServerName\hiddenshare$\$FolderName -Recurse
}
After all I get this error:
Remove-Item : Cannot find path '\\ \$\ ' because it does not exist.
At C:\temp\servers_folders.csv.ps1:5 char:3
+ Remove-Item -Path \\$ServerName\hiddenshare$\$FolderName -Recurse
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (\\ \usr$\ :String) [Remove-Item], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand
you dont need to import your csv more than once.
Inside the foreach block you have to reference the current item by $_
try this, remove -whatif if it's ok
$servers=import-csv .\servers.csv
$servers | %{
remove-item -path \\$($_.servername)\users$\$($_.FolderName) -whatif
}