How to copy a file then save the destination path to CSV - powershell

I'm trying to copy a batch of files (those whose filename begins with 6 digits) from a temp folder to a permanent location, excluding those that already exist in the new location.
Once the copy is done, I want to export the filename and new path of the copied file into a CSV.
It's pretty easy to get the old file location and export to CSV, I'm just not quite sure how to get the new file location.
My script looks like this:
# Prompt for file origin
$file_location = Read-Host -Prompt "Where do you want your files to come from?"
# Prompt for file destination
$file_destination = Read-Host -Prompt "Where do you want your files to go? `n(They won't be copied if they're already there)"
# Save contents of file destination - used to check for duplicates
$dest_contents = Get-ChildItem $file_destination
<# For all of the files of interest (those whose names begin with 6 digits) in $file_location,
determine whether that filename already exists in the target directory ($file_destination)
If it doesn't, copy the file to the destination
Then save the filename and the new** filepath to a CSV #>
Get-ChildItem -Path $file_location |
Where-Object { $_.Name -match '^\d{6}' -And !($dest_contents -Match $_.Name ) } |
Copy-Item -Destination $file_destination -PassThru |
Select-Object -Property Name, FullName | # **Note: This saves the old filepath, not the new one
Export-CSV -Path "$file_location\files_copied.csv" -NoClobber

You can do this with a few changes to the code
# Prompt for file origin
$file_location = Read-Host -Prompt "Where do you want your files to come from?"
# Prompt for file destination
$file_destination = Read-Host -Prompt "Where do you want your files to go? `n(They won't be copied if they're already there)"
# Save contents of file destination - used to check for duplicates
$dest_contents = Get-ChildItem $file_destination | Select-Object -ExpandProperty Name
<# For all of the files of interest (those whose names begin with 6 digits) in $file_location,
determine whether that filename already exists in the target directory ($file_destination)
If it doesn't, copy the file to the destination
Then save the filename and the new** filepath to a CSV #>
Get-ChildItem -Path $file_location |
Where-Object { $_.Name -match '^\d{6}' -and ($dest_contents -notcontains $_.Name ) } |
ForEach-Object {
Copy-Item -Path $_.FullName -Destination $file_destination
# emit a PSObject storing the Name and (destination) FullName of the file that has been copied
# This will be used to generate the output in the 'files_copied.csv'
New-Object -TypeName PSObject -Property ([ordered]#{ Name = $_.Name; FullName = (Join-Path $file_destination $_.Name)})
} |
Export-CSV -Path "$file_location\files_copied.csv" -NoTypeInformation -Force
Note that i only gather the Names of the files already in the destination path instead of the fileInfo objects. This makes it a lot 'leaner' since the only reason for gathering is to have a collection of file names to compare with.
As it is now, you have a fixed name for the 'files_copied.csv' and personally i think it would be a good idea to make that more generic by adding the current date to it for instance like
Export-CSV -Path ("{0}\files_copied_{1}.csv" -f $file_location, (Get-Date).ToString("yyyy-MM-dd")) -NoTypeInformation -Force
P.s. I'm using the [ordered] here so the output will always have the properties in the same order. This however requires PowerShell v3 or better.
Also, i suggest looking at the -File or -Attributes switches on the Get-ChildItem command if you need to make sure the code only copies Files, not Directories. If your version of PowerShell is 2.0, you can use the Where-Object { !$_.PSIsContainer } construct to filter out files only.

I'd use a different approach testing if destination exist while iterating source files.
To append to a present csv (log) file you need to remove the header of the new one.
As Get-ChildItem allows ranges [0-9] you can directly select leading numbers
## Q:\Test\2018\08\08\SO_51738853.ps1
# Prompt for file origin
$SrcDir = Read-Host -Prompt "Where do you want your files to come from?"
# Prompt for file destination
$DstDir = Read-Host -Prompt "Where do you want your files to go? `n(They won't be copied if they're already there)"
$SrcFiles = Join-Path (Get-Item $SrcDir).Fullname [0-9][0-9][0-9][0-9][0-9][0-9]*
$CopiedFiles = ForEach ($SrcFile in (Get-ChildItem -Path $SrcFiles)){
$DstFile = Join-Path $DstDir $SrcFile.Name
If (!(Test-Path $DstFile)){
$SrcFile | Copy-Item -Destination $DstFile -PassThru |
Select-Object -Property Name, FullName
}
}
# Check if log file exists, if yes append, otherwise export.
$LogFile = Join-Path $SrcDir "files_copied.csv"
If ($CopiedFiles){
If (!(Test-Path $LogFile)){
Export-CSV -Path $LogFile -InputObject $CopiedFiles -NoTypeInformation
} else {
# We need to remove the Header appending to the present csv
$CopiedFiles | ConvertTo-Csv -NoType | Select-Object -Skip 1 | Add-Content -Path $LogFile
}
}

Related

Copy files into newly created folders on partial filename match

Hi all reaching out because I've reached the limits of my powershell knowledge.
I have a directory that has over 200,000 files, I need to copy all files that have a partial match to the filename into folders that I have already created using this script
Set-Location "C:\Users\joshh\Documents\Testing Environment"
$Folders = Import-Csv "C:\Users\joshh\Documents\Weichert.csv"
ForEach ($Folder in $Folders) {
New-Item "myfilepathhere\$($Folder.folderName)" -type directory
}
UPDATED:
Here is a sample of the filenames:
TH-246-02050-LOL-SQ-ALT2.png
TH-246-02050-WHT-H.png
TH-247-02050-EMB-LOD.png
TH-246-02050-LOL-H-ALT2.png
TH-246-02050-LOL-SQ.png
TH-246-02050-LOL-SQ-ALT.png
TH-247-02050-EMB-LOD-ALT.png
TH-247-02050-EMB-LOL.png
TH-247-02050-EMB-LOL-ALT.png
TH-247-02050-LOD-H.png
Above is an example of what the filenames look like, I need to copy all files containing -EMB- and move them into folders in another directory that match the first 12 characters of that filename (ex. TH-247-02050)
UPDATED:
And if a folder doesn't exist create a folder with the first 12 characters of the filename.
Mind you the first 12 characters have many variants some start with RM, KW, etc.
This is what I have so far and what I know but I know the Move-Item portion isn't exactly what I want it to do
$source = "targetPath"
$destination = "targetPath2"
$embFiles = #(Get-ChildItem ${source}/*EMB* -File | Select-Object -ExpandProperty FullName)
foreach($file in $embFiles) {
if($file | Where-Object { $_ -clike "*EMB*" }){
Move-Item -Path $source -Destination $destination
}
}
Any and all help would be GREATLY appreciated!
Here is one way you could do it:
Get all files that contain -EMB- in their names: -Filter *-EMB-* -File.
Group all this files by everything before -EMB-, here we can use Group-Object -AsHashTable and a calculated expression using Regex.Match. See https://regex101.com/r/iOoBJS/1 for details.
Loop through the Keys of the hash table, each Key will be the Name Destination folder of the group of files (i.e.: TH-247-02050).
Join the destination path ($destinationPath2) with the name of the destination folder ($folder), here we can use Join-Path and check if this joined path exists, if it doesn't, create a new folder with New-Item.
Lastly, we can move all the files (the Values of each Key from the hash table) to their corresponding destination.
$source = "targetPath"
$destination = "targetPath2"
$map = Get-ChildItem $source -Filter *-EMB-* -File | Group-Object -AsHashTable -AsString {
[regex]::Match($_.BaseName, '(?i).+(?=-EMB-)').Value
}
foreach($folder in $map.Keys) {
$d = Join-Path $destination -ChildPath $folder
$d = New-Item $d -ItemType Directory -Force
# -WhatIf can be removed once you have checked the script is doing what you want
$map[$folder] | Move-Item -Destination $d -WhatIf -Verbose
}
-AsString is needed in Windows PowerShell due to a bug.

Merge CSV files in subfolders

I have found a script which does everything that I need, but it's only useful if you run it in a single folder. What I'd like is:
Script is located in c:/temp/. Upon running the script, it would go into each subfolder and execute. Each subfolder would then have a separate Final.csv.
Somebody mentioned just add -Recurse, but it doesn't complete the job as described. With -Recurse added, it goes into each subfolder and creates a Final.csv final in the root dir (C:/temp/) instead of creating a Final.csv in each subfolder.
$getFirstLine = $true
get-childItem *.csv | foreach {
$filePath = $_
$lines = Get-Content $filePath
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 2}
}
$getFirstLine = $false
Add-Content Final.csv $linesToWrite
}
If you are certain the csv files combined this way will leave you a valid 'Final.csv', you need to use Group-Object in order to create a combined file in each of the directories where the csv files to combine are found.
Suppose you have a folder with subfolders 'Folder1' and 'Folder2', both having csv files in them like these:
first.csv
Lorem,Ipsum,Dolor,Sic,Amet
data1-1,data1-2,data1-3,data1-4,data1-5
data2-1,data2-2,data2-3,data2-4,data2-5
second.csv
Lorem,Ipsum,Dolor,Sic,Amet
something,blah,whatever,very important,here's more..
Then this should do it for you:
$targetFileName = 'Final.csv'
# loop over the CSV files, but exclude the Final.csv file
# Group the files by their DirectoryNames
Get-ChildItem -Path 'D:\Test' -Filter '*.csv' -File -Recurse -Exclude $targetFileName | Group-Object DirectoryName | ForEach-Object {
# reset the $getFirstLine variable for each group
$getFirstLine = $true
# create the target path for the combined csv inside this folder.
# ($_.Name is the name of the group, which is the Directory name of the files inside the group)
$target = Join-Path -Path $_.Name -ChildPath $targetFileName
foreach ($file in $_.Group) {
if ($getFirstLine) {
# copy the first CSV as a whole
Get-Content -Path $file.FullName | Set-Content -Path $target
$getFirstLine = $false
}
else {
# add the content of the next file(s) without the header line
Get-Content -Path $file.FullName | Select-Object -Skip 1 | Add-Content -Path $target
}
}
}
The end result is that each subfolder will have a new 'Final.csv' file containing
Lorem,Ipsum,Dolor,Sic,Amet
data1-1,data1-2,data1-3,data1-4,data1-5
data2-1,data2-2,data2-3,data2-4,data2-5
something,blah,whatever,very important,here's more..
Of course I'm just showing an example for one of the subfolders.. Other subfolders will contain different 'Final.csv' content

How to copy files using a txt list to define beginning of file names

Hello awesome community :)
I have a list containing a bunch of SKU's. All the filenames of the files, that I need to copy to a new location, starts with the corresponding SKU like so
B6BC004-022_10_300_f.jpg
In this case "B6BC004" is the SKU and my txt list contains "B6BC004" along with many other SKU's.
Somewhere in the code below I know I have to define that it should search for files beginning with the SKU's from the txt file but I have no idea how to define it.
Get-Content .\photostocopy.txt | Foreach-Object { copy-item -Path $_ -Destination "Z:\Photosdestination\"}
Thanks in advance :)
If all files start with one of the SKU's, followed by a dash like in your example, this should work:
$sourceFolder = 'ENTER THE PATH WHERE THE FILES TO COPY ARE'
$destination = 'Z:\Photosdestination'
# get an array of all SKU's
$sku = Get-Content .\photostocopy.txt | Select-Object -Unique
# loop through the list of files in the source folder and copy all that have a name beginning with one of the SKU's
Get-ChildItem -Path $sourceFolder -File -Recurse |
Where-Object { $sku -contains ($_.Name -split '\s*-')[0] } |
ForEach-Object { $_ | Copy-Item -Destination $destination }
I haven't tested this so please proceed with caution!
What is does it loops through all the items in your photostocopy.txt file, searches the $source location for a file(s) with a name like the current item from your file. It then checks if any were found before outputting something to the console and possibly moving the file(s).
$source = '#PATH_TO_SOURCE'
$destination = '#PATH_TO_DESTINATION'
$photosToCopy = Get-Content -Path '#PATH_TO_TXT_FILE'
$photosToCopy | ForEach-Object{
$filesToCopy = Get-ChildItem -Path $source -File | Where-Object {$_.Name -like "$_*"}
if ($fileToCopy.Count -le 0){
Write-Host "No files could be found for: " $_
}else{
$filesToCopy | ForEach-Object{
Write-Host "Moving: " $_.Name
Copy-Item -Path $_.FullName -Destination $destination
}
}
}
Let me know how if this helps you :)

How do I create a new files automatically depend on an existing variable in PowerShell?

I have many file in a folder, I would like to check the existing and matching of the file with variable that I initialize. Then, if the file exit and match, I want to get some information from the files (many file), then create a new file depend on how many file exist and match.
I tried this code, I can check the matching and existing file. I can create a new file and get the information from the file, but I only can create 1 file.
The information that I get from the file, each file is different.
$ID = "123"
$Pre = "ABC"
$Path = "C:\Folder"
$PO = Get-ChildItem -Path $Path
foreach ($File in $PO) {
if (($File.Name -match $ID) -and ($File.Name -match $Pre)) {
Write-Host ">>POfile Found: $File"
} else {
Write-Host ">>Check Again!"
}
}
# CREATE FILE
$Jb_Path = "C:\Folder\Jb"
## GET INFORMATION
$count = 1
$Get_PO = Get-ChildItem -Path $Path\$File -File -Recurse
$POfile = Get-Random -InputObject $Get_PO -Count $count
Write-Host ">>Selected POfile= $POfile"
$FilteredContents = Get-Content $POfile | Where-Object {$_ -like "*;INFO*"}
$Get_INFO = $FilteredContents.Substring(5,2)
## NEW FILE
New-Item -Path $Jb_Path\NEW_$Pre$ID-$Get_INFO.txt -Force
In the section # CREATE FILE you are referencing the variable $File which has the last value iterated in the previous foreach (even if it didn't match the if condition).
Asuming the $Pre is for prefix and comes first in a file name simply do a
Get-ChildItem "$Path\*$Pre*$ID*"
to only get file names for your criteria.
As $File contains only one file name a Get-Random doesn't make sense, especially as it might not contain a line with ;INFO
Assuming the two characters to extract are in front of ;INFO this untested script might do:
$Pre = "ABC"
$ID = "123"
$Path = "C:\Folder"
$Jb_Path= "C:\Folder\Jb"
Get-ChildItem "$Path\*$Pre*$ID*" | Get-Content |
Select-String -Pattern '^.....(..).*;INFO' |
Get-Random | ForEach-Object {
$NewFile = Join-Path $Jb_Path ('NEW_{0}{1}-{2}.txt' -f $Pre,
$ID,$_.Matches.Groups[1].Value)
New-Item -Path $NewFile -ItemType File -Force -WhatIf
}
It will only output what it would do without the -WhatIf parameter.
If no file matching the criteria and RegEx pattern is found it will silently continue.
If my assumptions led me wrong, enhance your question be editing it with more details.

Looping through directories from CSV file

I have a working PS script that's used to compare the contents of two directories and report any missing files or files that have different contents. This is what it currently does:
Takes two specific directories.
Pulls all of the files from each directory (minus any excluded paths/files).
Checks to see if any files are missing from one or the other.
For each file that is in both directories, it does a hash comparison of that file's contents.
Places the results in variables based on files in the source but not the destination and vice-versa, and files that were in both the source and destination but had different contents.
Below is the main chunk of code that does all this. What I need to change/add is the ability to list ONLY the specific paths I want to be compared between the two servers. For example, say I want the following paths to be used for the comparison:
D:\Files\Stuff
D:\Files\Contents\Folders\MoreStuff
D:\Files\Archive
I would want each of those directories compared between their counterpart on the other server. Meaning it would compare the D:\Files\Stuff path between server 1 and server 2. It would NOT compare the paths with each other, though. Meaning, I DON'T want it to compare D:\Files\Stuff with D:\Files\Archive, regardless of the server.
What's the best way I can achieve this?
$SourceDir = "\\12345-serverP1\D$\Files";
$DestDir = "\\54321-serverP2\D$\Files";
#The excluded array holds all of the specific paths, files, and file types you don't want to be compared.
$ExcludedPaths = #(Import-Csv -LiteralPath 'D:\ExclusionList.csv') |Select-Object -Expand ExcludedPaths;
$ExcludedFiles = #(Import-Csv -LiteralPath 'D:\ExclusionList.csv') |Select-Object -Expand ExcludedFiles;
#Script block which stores the filter for the Where-Object used to exclude chosen paths.
#This script is called with the -FilterScript parameter below.
$Filter = {
$FullName = $_.FullName
-not($ExcludedPaths | Where-Object {$FullName -like "$_*";})
}
#Grabs all files from each directory, minus the excluded paths and files, and assigns them to variables based on Source and Destination.
try {$SourceFiles = Get-ChildItem -Recurse -Path $SourceDir -Exclude $ExcludedFiles -Force -ErrorAction Stop | Where-Object -FilterScript $Filter;}
catch {Write-Output "$(Get-Date) The following Source path was not found: $SourceDir" | Out-File $ErrorLog -Append;}
try {$DestFiles = Get-ChildItem -Recurse -Path $DestDir -Exclude $ExcludedFiles -Force -ErrorAction Stop | Where-Object -FilterScript $Filter;}
catch {Write-Output "$(Get-Date) The following Destination path was not found: $DestDir" | Out-File $ErrorLog -Append;}
#Pulls the name of each file and assigns it to a variable.
$SourceFileNames = $SourceFiles | % { $_.Name };
$DestFileNames = $DestFiles | % { $_.Name };
#Empty variables to be used in the loops below.
$MissingFromDestination = #();
$MissingFromSource = #();
$DifferentFiles = #();
$IdenticalFiles = #();
#Iterates through each file in the Source directory and compares the name against each file in the Destination directory.
#If the file is missing from the Destination, it is added to the MissingFromDestination variable.
#If the file appears in both directories, it compares the hash of both files.
#If the hash is the same, it adds it to the IdenticalFiles variable. If the hash is different, it adds the Source file to the DifferentFiles variable.
try {
foreach ($f in $SourceFiles) {
if (!$DestFileNames.Contains($f.Name)) {$MissingFromDestination += $f;}
elseif ($DestFileNames.Contains($f.Name)) {$IdenticalFiles += $f;}
else {
$t = $DestFiles | Where { $_.Name -eq $f.Name };
if ((Get-FileHash $f.FullName).hash -ne (Get-FileHash $t.FullName).hash) {$DifferentFiles += $f;}
}
}
}
catch {Write-Output "$(Get-Date) The Destination variable is null due to an incorrect path." | Out-File $ErrorLog -Append;}
#Iterates through each file in the Destination directory and compares the name against each file in the Source directory.
#If the file is missing from the Source, it is added to the MissingFromSource variable.
try {
foreach ($f in $DestFiles) {
if (!$SourceFileNames.Contains($f.Name)) {$MissingFromSource += $f;}
}
}
catch {Write-Output "$(Get-Date) The Source variable is null due to an incorrect path." | Out-File $ErrorLog -Append;}