Ouline -
I have 300,000+ folders containing subfolders and files.
I am trying to flatten each directory so that subfolders are removed and all files are brought to their respective parent directory.
Unfortunately, the Get-ChildItem cmdlet runs in the location of the .ps1 file and not those specified in the .txt file.
I have been trying to troubleshoot this for hours, any help would be greatly apprecieated.
Process -
First, I run a .ps1 file that retrieves the parent folder locations from a text file and calls a custom module:
[System.IO.File]::ReadLines("C:\Users\ccugnet\Desktop\test.txt") | ForEach-Object {
Import-Module MyFunctions
fcopy -SourceDir $line -DestinationDir $line
Remove-Module MyFunctions
}
Second, the custom module moves the child items to the parent folder, appending an incrementing digit to the end of the file name for duplicate files:
function fcopy ($SourceDir,$DestinationDir)
{
Get-ChildItem $SourceDir -Recurse | Where-Object { $_.PSIsContainer -eq $false } | ForEach-Object {
$SourceFile = $_.FullName
$DestinationFile = $DestinationDir + $_
if (Test-Path $DestinationFile) {
$i = 0
while (Test-Path $DestinationFile) {
$i += 1
$DestinationFile = $DestinationDir + $_.basename + $i + $_.extension
}
} else {
Move-Item -Path $SourceFile -Destination $DestinationFile -Verbose -Force -WhatIf
}
Move-Item -Path $SourceFile -Destination $DestinationFile -Verbose -Force -WhatIf
}
}
Text file contents:
"C:\Users\ccugnet\Desktop\Dir_Flatten\Fox, Hound & Hunter"
"C:\Users\ccugnet\Desktop\Dir_Flatten\Cat, Hat"
"C:\Users\ccugnet\Desktop\Dir_Flatten\Alice"
"C:\Users\ccugnet\Desktop\Dir_Flatten\Beetle | Juice"
Your $line is empty
PS P:\> [System.IO.File]::ReadLines("C:\Users\ccugnet\Desktop\test.txt") | ForEach-Object {
$line
}
Try $_
PS P:\> [System.IO.File]::ReadLines("C:\Users\ccugnet\Desktop\test.txt") | ForEach-Object {
$_
}
"C:\Users\ccugnet\Desktop\Dir_Flatten\Fox, Hound & Hunter"
"C:\Users\ccugnet\Desktop\Dir_Flatten\Cat, Hat"
"C:\Users\ccugnet\Desktop\Dir_Flatten\Alice"
"C:\Users\ccugnet\Desktop\Dir_Flatten\Beetle | Juice"
Related
I have a folder with a number of subfolders containing files and want to copy all files to the root folder but only overwrite if newer.
In powershell I can do -
Get-ChildItem D:\VaM\Custom\Atom\Person\Morphs\temp2\female -Recurse -file | Copy-Item -Destination D:\VaM\Custom\Atom\Person\Morphs\female
But this will overwrite all files, I only want to overwrite files if the copied file is newer.
robocopy can overwrite only older this but keeps the folder structure.
Try this
$root = 'D:\VaM\Custom\Atom\Person\Morphs\temp2\female'
[bool]$Delete = $false
Get-ChildItem $root -Recurse -File |
Where-Object {$_.DirectoryName -ne $root } | # Do not touch files already seated in root
ForEach-Object {
$rootNameBrother = Get-Item "$root\$($_.Name)" -ea 0
if($rootNameBrother -and $rootNameBrother.LastWriteTime -lt $_.LastWriteTime) {
# RootFile with same name exists and is Older - Copy and override
Copy-Item -Path $_.FullName -Destination $rootNameBrother.FullName -Force
}
elseif ($rootNameBrother -and $rootNameBrother.LastWriteTime -ge $_.LastWriteTime) {
# RootFile with same name exists and is Newer or same Age
# Delete non root File if allowed
if($Delete) { Remove-Item $_.FullName -Force }
}
}
Set...
$Delete = $true
...if you wish to delete non root files that could not be copied because there already was a file with the same name and greater modiefydate in root.
You also can set the
$VerbosePreference = "Continue"
$WhatIfPreference = "Continue"
variables, just to be safe when you execute the script for the first time.
If you wish to delete all empty subfolder, you can run this:
$allFolders =`
Get-ChildItem $root -Recurse -Directory |
ForEach-Object {
# Add now Depth Script Property
$_ | Add-Member -PassThru -Force -MemberType ScriptProperty -Name Depth -Value {
# Get Depth of folder by looping through each letter and counting the backshlashes
(0..($this.FullName.Length - 1) | ForEach {$this.FullName.Substring($_,1)} | Where-Object {$_ -eq "\"}).Count
}
}
# Sort all Folder by new Depth Property annd Loop throught
$allFolders | Sort -Property Depth -Descending |
ForEach-Object {
# if .GetFileSystemInfos() method return null, the folder is empty
if($_.GetFileSystemInfos().Count -eq 0) {
Remove-Item $_.FullName -Force # Remove Folder
}
}
You can do it like this:
$source = 'D:\VaM\Custom\Atom\Person\Morphs\temp2\female'
$destination = 'D:\VaM\Custom\Atom\Person\Morphs\female'
Get-ChildItem -Path $source -Recurse -File | ForEach-Object {
# try and get the existing file in the destination folder
$destFile = Get-Item -Path (Join-Path -Path $destination -ChildPath $_.Name) -ErrorAction SilentlyContinue
if (!$destFile -or $_.LastWriteTime -gt $destFile.LastWriteTime) {
# copy the file if it either did not exist in the destination or if this file is newer
Write-Host "Copying file $($_.Name)"
$_ | Copy-Item -Destination $destination -Force
}
}
I ended up doing this:
Get-ChildItem G:\VaM\Custom\Atom\Person\Morphs\temp2\ -Recurse |
Where-Object { $_.PSIsContainer -eq $true } |
Foreach-Object { robocopy $_.FullName G:\VaM\Custom\Atom\Person\Morphs\female /xo /ndl /np /mt /nfl}
it runs through the directory structure and copys the contents of each directory to the destination but only overwrites older files.
I am trying to write a script that will find duplicate file inside a compressed files.
The compressed files can be ZIP or CAB (Need help to extract CAB file also because currently its not working).
What I have so far is to extract the zips to a temp folder (don't know how to extract cab) and if there is a vip inside I need to extract him also to the same folder. currently all the files are extracted to the same temp folder what I need is to extract each zip/cab into a folder with the original name even if he has a vip inside. (the zip/cab are not flat) in the next step I need to find duplication files and display all the duplication and where they found.
The script below is not working...
$tempFolder = Join-Path ([IO.Path]::GetTempPath()) (New-GUID).ToString('n')
$compressedfiles = Get-ChildItem -path C:\Intel\* -Include "*.zip","*.CAB"
foreach ($file in $compressedfiles) {
if ($file -like "*.zip") {
$zip = [System.IO.Compression.ZipFile]::ExtractToDirectory($file, $tempFolder)
$test = Get-ChildItem -path $tempFolder\* -Include "*.vip"
if ($test) {
$zip = [System.IO.Compression.ZipFile]::ExtractToDirectory($test, $tempFolder)
}
}
}
$Files=gci -File -Recurse -path $tempFolder | Select-Object -property FullName
$MatchedSourceFiles=#()
ForEach ($SourceFile in $Files)
{
$MatchingFiles=#()
$MatchingFiles=$Files |Where-Object {$_.name -eq $SourceFile.name}
if ($MatchingFiles.Count -gt 0)
{
$NewObject=[pscustomobject][ordered]#{
File=$SourceFile.FullName
#MatchingFiles=$MatchingFiles
}
$MatchedSourceFiles+=$NewObject
}
}
$MatchedSourceFiles
Remove-Item $tempFolder -Force -Recurse
Building on what you have already tried, you could do this like:
Add-Type -AssemblyName System.IO.Compression.FileSystem
$tempFolder = Join-Path -Path ([IO.Path]::GetTempPath()) -ChildPath (New-GUID).Guid
$compressedfiles = Get-ChildItem -Path 'C:\Intel' -Include '*.zip','*.CAB' -File -Recurse
$MatchedSourceFiles = foreach ($file in $compressedfiles) {
switch ($file.Extension) {
'.zip' {
# the destination folder should NOT already exist here
$null = [System.IO.Compression.ZipFile]::ExtractToDirectory($file.FullName, $tempFolder)
Get-ChildItem -Path $tempFolder -Filter '*.vip' -File -Recurse | ForEach-Object {
$null = [System.IO.Compression.ZipFile]::ExtractToDirectory($_.FullName, $tempFolder)
}
}
'.cab' {
# the destination folder MUST exist for expanding .cab files
$null = New-Item -Path $tempFolder -ItemType Directory -Force
expand.exe $file.FullName -F:* $tempFolder > $null
}
}
# now see if there are files with duplicate names
Get-ChildItem -Path $tempFolder -File -Recurse | Group-Object Name |
Where-Object { $_.Count -gt 1 } | ForEach-Object {
foreach ($item in $_.Group) {
# output objects to be collected in $MatchedSourceFiles
[PsCustomObject]#{
SourceArchive = $file.FullName
DuplicateFile = '.{0}' -f $item.FullName.Substring($tempFolder.Length) # relative path
}
}
}
# delete the temporary folder
$tempFolder | Remove-Item -Force -Recurse
}
# display on screen
$MatchedSourceFiles
# save as CSV file
$MatchedSourceFiles | Export-Csv -Path 'X:\DuplicateFiles.csv' -UseCulture -NoTypeInformation
The output would be something like this:
SourceArchive DuplicateFile
------------- -------------
D:\Test\test.cab .\test\CA2P30.BA0
D:\Test\test.cab .\test\dupes\CA2P30.BA0
D:\Test\test.zip .\test\CA2P3K.DAT
D:\Test\test.zip .\test\dupes\CA2P3K.DAT
D:\Test\test.zip .\test\CA2P60.BA0
D:\Test\test.zip .\test\dupes\CA2P60.BA0
Below is my scenario which I need to achieve:
I have a file test.txt .This file contains file names. So suppose, test.txt has below two lines in it:
file1.txt
file2.txt
Please note that these two files (file1.txt, file2.txt) are present in a folder (src_folder).
Below is the action that I need to perform:
I need to read this test.txt file
For every file entry found in test.txt file (in our case file1.txt and file2.txt), copy these two files from src_folder to a different folder (say suppose tgt_folder).
How can I achieve this using powershell script?
Appreciate help on this! Thanks in advance.
This shouldn't be too difficult:
$sourceFolder = 'D:\Test\src_folder'
$destination = 'D:\Test\tgt_folder'
Get-Content -Path 'D:\Path\To\test.txt' | ForEach-Object {
Copy-Item -Path (Join-Path -Path $sourceFolder -ChildPath $_) -Destination $destination
}
If you're worried that test.txt may contain empty lines, do:
Get-Content -Path 'D:\Path\To\test.txt' | Where-Object { $_ -match '\S' } | ForEach-Object { .. }
as per your comment you need to have two destinations, depending on the file extension, use below code:
$sourceFolder = 'D:\Test\src_folder'
$csvDestination = 'D:\Test\tgt_folder'
$txtDestination = 'D:\Test\new_tgt_folder'
# test if the destination folders exist. If not create them first
if (!(Test-Path -Path $csvDestination)) {
$null = New-Item -Path $csvDestination -ItemType Directory
}
if (!(Test-Path -Path $txtDestination)) {
$null = New-Item -Path $txtDestination -ItemType Directory
}
Get-Content -Path 'D:\Path\To\test.txt' | Where-Object { $_ -match '\S' } | ForEach-Object {
$file = Join-Path -Path $sourceFolder -ChildPath $_.Trim()
switch ([System.IO.Path]::GetExtension($file)) {
# you can add more options here if there are other extensions to handle differently
'.csv' {$destination = $csvDestination; break}
default {$destination = $txtDestination} # this is for .txt or any other extension
}
if (Test-Path -Path $file -PathType Leaf) {
Copy-Item -Path $file -Destination $destination
}
else {
Write-Warning "File '$file' not found"
}
}
We need to know in code what files were copied and some files were old and were not copied.
$date = (get-date).AddDays(-1)
get-childitem -File c:\t\*.*,c:\f\*.*,c:\u\*.*,c:\s\*.* | where-object {$_.LastWriteTime -gt $date} |
Copy-Item -Destination c:\t\1 ```
If you're on PowerShell 4.0 or newer, you could use the .Where({}) extension method in "Split" mode to split new and old files into two groups:
$new,$old = #(Get-ChildItem -File C:\t\*.*).Where({$_.LastWriteTime -gt $date}, 'Split')
# Write file names to log files
$new.Name > newfiles.txt
$old.Name > oldfiles.txt
$new | Copy-Item -Destination C:\t\1\
If by 'showing progress' you mean writing some info to the console, then this could be what you want.
$date = (Get-Date).AddDays(-1)
$dest = 'C:\t\1'
# if the destination folder does not exist, create it first
if (!(Test-Path $dest -PathType Container)) {
New-Item -Path $dest -ItemType Directory | Out-Null
}
Get-ChildItem -Path 'C:\t','C:\f','C:\u','C:\s' -File | ForEach-Object {
if ($_.LastWriteTime -gt $date) {
Write-Host "Copying file '$($_.FullName)'" -ForegroundColor Green
$_ | Copy-Item -Destination $dest
}
else {
Write-Host "File '$($_.FullName)' is too old.. Skipped" -ForegroundColor Yellow
}
}
I am a complete novice when it comes to powershell, but I have been given a script that I need to improve so that we can move updated or new files from one server to another. I've managed to get to grips with the current script but am struggling to find the right cmdlets and paramters to achieve the desired behaviour.
The script I have is successful at detecting changed files and moving them to a location ready for transfer to another server, but it doesn't detect any new files.
Can anyone give me some guidance as to how I would be able to achieve both behaviours?
$CurrentLocation = "C:\current"
$PreviousLocation = "C:\prev"
$DeltaLocation = "C:\delta"
$source = #{}
#
# Get the Current Location file information
#
Get-ChildItem -recurse $CurrentLocation | Foreach-Object {
if ($_.PSIsContainer) { return }
$source.Add($_.FullName.Replace($CurrentLocation, ""), $_.LastWriteTime.ToString())
}
Write-Host "Content of Source"
$source
$changesDelta = #{}
$changesPrevious = #{}
#
# Get the Previous Directory contents and compare the dates against the Current Directory contents
#
Get-ChildItem -recurse $PreviousLocation | Foreach-Object {
if ($_.PSIsContainer) { return }
$File = $_.FullName.Replace($PreviousLocation, "")
if ($source.ContainsKey($File)) {
if ($source.Get_Item($File) -ne $_.LastWriteTime.ToString()) {
$changesDelta.Add($CurrentLocation+$File, $DeltaLocation+$File)
$changesPrevious.Add($CurrentLocation+$File, $PreviousLocation+$File)
}
}
}
Write-Host "Content of changesDelta:"
$changesDelta
Write-Host "Content of changesPrevious:"
$changesPrevious
#
# Copy the files into a temporary directory
#
foreach ($key in $changesDelta.Keys) {
New-Item -ItemType File -Path $changesDelta.Get_Item($key) -Force
Copy-Item $key $changesDelta.Get_Item($key) -Force
}
Write-Host $changesDelta.Count "Files copied to" $DeltaLocation
#
# Copy the files into the Previous Location to match the Current Location
#
foreach ($key in $changesPrevious.Keys) {
Copy-Item $key $changesDelta.Get_Item($key) -Force
}
Here's a simplified approach to your needs. One thing to note is that some of the constructs I've used require PSv3+. This does not copy directory structure, just the files. Additionally, it compares the basenames (ignoring extensions) which may or may not do what you want. It can be expanded to include extensions by using .Name instead of .BaseName
#requires -Version 3
$CurrentLocation = 'C:\current'
$PreviousLocation = 'C:\prev'
$DeltaLocation = 'C:\delta'
$Current = Get-ChildItem -LiteralPath $CurrentLocation -Recurse -File
$Previous = Get-ChildItem -LiteralPath $PreviousLocation -Recurse -File
ForEach ($File in $Current)
{
If ($File.BaseName -in $Previous.BaseName)
{
If ($File.LastWriteTime -gt ($Previous | Where-Object { $_.BaseName -eq $File.BaseName }).LastWriteTime)
{
Write-Output "File has been updated: $($File.FullName)"
Copy-Item -LiteralPath $File.FullName -Destination $DeltaLocation
}
}
Else
{
Write-Output "New file detected: $($File.FullName)"
Copy-Item -LiteralPath $File.FullName -Destination $DeltaLocation
}
}
Copy-Item -Path "$DeltaLocation\*" -Destination $PreviousLocation -Force