I am trying to copy all files recursively from a:\ to b:\, except those whose metadata is present in a:\list.txt. The list.txt pattern is LastWriteTimeYYYY-MM-DD HH:MM:SS,size,.fileextension, for example:
2001-01-31 23:59:59,12345,.doc
2001-01-31 23:59:59,12345,.txt
2001-01-31 23:59:00,456,.csv
...so any and all files, anywhere in the a:\ dir tree, matching these metadata should not be copied.
I seem to be having trouble with the Where-Object in order to exclude the items on the list.txt, but copy everything else:
$Source = "C:\a"
$Target = "C:\b"
$List = Import-Csv list.txt -Header LastWriteTime,Size,Name
$Hash = #{}
ForEach ($Row in $List){
$Key = ("{0},{1},.{2}" -F $Row.LastWriteTime,$Row.Size,$Row.Name.Split('.')[-1].ToLower())
IF (!($Hash[$Key])) {$Hash.Add($Key,$Row.Name)}
}
$Hash | Format-Table -Auto
Get-Childitem -Path $Source -Recurse -File | Where-Object {$Hash -eq $Hash[$Key]}| ForEach-Object {$Key = ("{0},{1},{2}" -F ($_.LastWriteTime).ToString('yyyy-MM-dd HH:mm:ss'),$_.Length,$_.Extension.ToLower())
#$Key
If ($Hash[$Key]){
$Destination = $_.FullName -Replace "^$([RegEx]::Escape($Source))","$Target"
If (!(Test-Path (Split-Path $Destination))){MD (Split-Path $Destination)|Out-Null}
$_ | Copy-Item -Destination $Destination
}
}
I propose you a simplification of your code :
$Source = "C:\a\"
$Target = "C:\b\"
New-Item -ItemType Directory $Target -Force | Out-Null
$List = Import-Csv list.txt -Header LastWriteTime,Length,Extension
Get-Childitem $Source -Recurse -File | %{
$File=$_
$exist=$List | where {$_.LastWriteTime -eq $File.LastWriteTime.ToString('yyyy-MM-dd HH:mm:ss') -and $_.Length -eq $File.Length -and $_.Extension -eq $File.Extension} | select -first 1
if ($exist -ne $null) {continue}
New-Item -ItemType Directory $File.DirectoryName.Replace($Source, $Target) -Force | Out-Null
Copy-Item $File.FullName $File.FullName.Replace($Source, $Target) -Force
}
Related
I want to get the duplicates from a folder structure and copy all of them to a single folder, while renaming (so they don't overwrite). I would like the first file from a duplicates group to be copied with it's original name, and for the rest to add "_X" at the end of the name.
I wrote a code that almost works, but at some point it just overwrites the first file copied. Only one file is being overwritten, the rest are renamed and copied like intended.
Get-ChildItem $SourcePath -Recurse -File -Force | Group-Object -Property Name | Where-Object {$_.Count -gt 1} | Select-Object -ExpandProperty Group |
ForEach-Object {
$SourceFile = $_.FullName
$FileName = $($_.BaseName + $_.Extension)
$DestFileName = Join-Path -Path $DestinationPath -ChildPath $FileName
if (Test-Path -Path $DestFileName) {
$DestinationFile = "$DestinationPath\" + $_.BaseName + "_" + $i + $_.Extension
$i+=1
} else {
$DestinationFile = $DestFileName
}
Copy-Item -Path $SourceFile -Destination $DestinationFile
}
I don't see the actual problem but you could rewrite the code without using Test-Path. Remove Select-Object -ExpandProperty Group too, then iterate over each group's elements. Increment a counter and append it to all file names except the first one.
Get-ChildItem $SourcePath -Recurse -File -Force | Group-Object -Property Name | Where-Object Count -gt 1 |
ForEach-Object {
$i = 0
foreach( $dupe in $_.Group ) {
$SourceFile = $dupe.FullName
$DestinationFile = Join-Path -Path $DestinationPath -ChildPath $dupe.BaseName
if( $i -gt 0 ) { $DestinationFile += "_$i" }
$DestinationFile += $dupe.Extension
Copy-Item -Path $SourceFile -Destination $DestinationFile
$i++
}
}
I found a piece of code here that almost does what I need, which is extract just one folder from an archived file.
The only issue I have is that the archive name changes month on month, therefore I wanted to use a wildcard. Once a wildcard is specified (* in $zipfile), the script does not work for me.
I would be grateful for any suggestions.
[Reflection.Assembly]::LoadWithPartialName('System.IO.Compression.FileSystem') | Out-Null
$zipfile = 'C:\ALL\Debtor*.zip'
$folder = 'tmp\st\sd'
$dst = 'C:\ALL\ZipOutput'
[IO.Compression.ZipFile]::OpenRead($zipfile).Entries | ? {
$_.FullName -like "$($folder -replace '\\','/')/*.*"
} | % {
$file = Join-Path $dst $_.FullName
$parent = Split-Path -Parent $file
if (-not (Test-Path -LiteralPath $parent)) {
New-Item -Path $parent -Type Directory | Out-Null
}
[IO.Compression.ZipFileExtensions]::ExtractToFile($_, $file, $true)
Try this out for size. Just use Get-ChildItem to locate the zip file in your ALL directory.
[Reflection.Assembly]::LoadWithPartialName('System.IO.Compression.FileSystem') | Out-Null
$zipfile = Get-ChildItem -Path C:\ALL\ -Filter *.zip | Where-Object {$_.Name -like "Debtor*"} | Select-Object -ExpandProperty FullName
$folder = 'tmp\st\sd\'
$dst = 'C:\ALL\ZipOutput'
[IO.Compression.ZipFile]::OpenRead($zipfile).Entries | Where-Object {
$_.FullName -like "$($folder -replace '\\','/')/*.*"
} | ForEach-Object {
$file = Join-Path $dst $_.FullName
$parent = Split-Path -Parent $file
if(-not (Test-Path -LiteralPath $parent)) {
New-Item -Path $parent -Type Directory | Out-Null
}
[IO.Compression.ZipFileExtensions]::ExtractToFile($_, $file, $true)
}
I am also assuming that the archive name changes but there are not multiple archives with that name. If there are you will need to wrap everything in a Foreach($zip in $zipfile){ ... }
I am trying to move all files under each subfolder into another folder with subfoldername_mmm_yyyy name.
Below codes only move all files in all sub folders into one folder with the name subfoldername_subfoldername_mmm_yyyy.
I know my for each loops are incorrect but I don't know how to fix them, would someone please help?
$curr_date = Get-Date
$folder_path = "C:\Logs\"
$file_type = "C:\Logs\*.log*"
$destination = "C:\Archive\"
# delete tmp files if existed
$deletefile = Get-ChildItem $destination -recurse -include *.7z.tmp -force | remove-item
# set min age of files
$max_days = "-1"
# determine how far back we go based on current date
$zip_date = $curr_date.AddDays($max_days)
Get-ChildItem -Path $folder_path | Where-Object { $_.Attributes -eq "Directory" } | foreach {
# obtain all subfolders
$servername = Get-ChildItem -Path $folder_path | Where-Object { $_.Attributes -eq "Directory" }
# move all files into servername_mmm_yyyy folder
Get-ChildItem $file_type -Recurse | Where-Object { ($_.LastWriteTime -lt $zip_date) -and ($_.psIsContainer -eq $false)}| foreach {
$x = $_.LastWriteTime.ToShortDateString()
$month_year = Get-Date $x -Format MMM_yyyy
$file_destination = ($destination) + ($servername) + "_" + ($month_year)
if (test-path $file_destination) {
move-item $_.fullname $file_destination
}
else {
new-item -ItemType directory -Path $file_destination
move-item $_.fullname $file_destination
}
}
}
Change:
Get-ChildItem $file_type -Recurse | Where-Object { ($_.LastWriteTime -lt $zip_date) -and ($_.psIsContainer -eq $false)} ...
to
Get-ChildItem $file_type -Recurse | Where-Object { ($_.LastWriteTime -lt $zip_date) -and ($_.psIsContainer -eq $false) -and $_.Name.EndsWith(".log")}
and that should get you just the files you want to target
I figured out my problem. Here is how I do it.
I just need to obtain the folder name of each file and build the destination path. Therefore, I only need one foreach loop.
# move all files into servername_mmm_yyyy folder
Get-ChildItem $file_path -Recurse | Where-Object { ($_.LastWriteTime -lt $zip_date) -and ($_.psIsContainer -eq $false)}| foreach {
$x = $_.LastWriteTime.ToShortDateString()
$month_year = Get-Date $x -Format MMM_yyyy
$servername = $_.Directory.Name
$file_destination = ($destination)+($servername)+"_"+($month_year)
if (test-path $file_destination){
copy-item $_.fullname $file_destination
}
else {
new-item -ItemType directory -Path $file_destination
copy-item $_.fullname $file_destination
}
}
$getFiles = Get-ChildItem -Path $readDir -File -Include "*.doc","*.docx","*.xlsx"-Recurse | %{
if(($_ -match "\.doc$") -or ($_ -match "\.docx$")){
$Doc = $word.Documents.Open($_.fullname)
$nameDoc = $fileSaveLoc + $_.Name.Replace(".docx",".txt").replace(".doc",".txt")
$Doc.saveas([ref] $nameDoc, [ref] 5)
$Doc.close()
if((Get-ChildItem "I:\temp\").length -ne 0){
$locations = (Get-Item "I:\temp\"), (Get-ChildItem "I:\temp\" -Directory -recurse) | % {
Get-ChildItem -File $_.FullName | Select-String -List -Pattern '^\d{3}-?\d{2}-?\d{4}$' |
% Path
}
if($locations -ne $null){
$locations | out-file "I:\temp\SSN_FILES.txt"
#Get-ChildItem "I:\temp\" -exclude "fullpath.txt","SSN_FILES.txt" | remove-item
}else{
Get-ChildItem "I:\temp\" -exclude "fullpath.txt","SSN_FILES.txt" | remove-item
}
}
}
elseif($_ -match "\.xlsx$"){
$workbook = $excel.Workbooks.Open($_.FullName)
$csvFilePath = "I:\temp\" + $_.Name.Replace(".xlsx",".csv")
#$csvFilePath = $_.FullName -replace "\.xlsx$", ".csv"
$workbook.SaveAs($csvFilePath, [Microsoft.Office.Interop.Excel.XlFileFormat]::xlCSV)
$workbook.Close()
if((Get-ChildItem "I:\temp\").length -ne 0){
$locations = (Get-Item "I:\temp\"), (Get-ChildItem "I:\temp\" -Directory -recurse) | % {
Get-ChildItem -File $_.FullName | Select-String -List -Pattern '^\d{3}-?\d{2}-?\d{4}$' |
% Path
}
if($locations -ne $null){
$locations | out-file "I:\temp\SSN_FILES.txt"
#Get-ChildItem "I:\temp\" -exclude "fullpath.txt","SSN_FILES.txt" | remove-item
}else{
Get-ChildItem "I:\temp\" -exclude "fullpath.txt","SSN_FILES.txt" | remove-item
}
}
}
}
So this basically says:
check for a file matching doc/docx/xlsx
convert them into a file that can be parsed through
parse through each file at every iteration and compare it to a regex
if the regex is not null, then output it to a file with the file path
otherwise, delete it and anything else that was created except for two files
restart the process at the next file
Now the problem I am encountering is that the files aren't being deleted. I can't get them to be removed when they are created, when I know they don't match the regex. Setting ($locations -eq $true) doesn't solve that issue because it never goes into the first conditional statement.
The folder should contain only the two files that were created and possibly the ones that match the regex.
I am working with a script to clear old files off our file server. We are using this line in the script to find all files older than a certain date:
$oldFiles = Get-ChildItem $oldPath -Recurse | Where-Object { $_.lastwritetime -le $oldDate }
My question is, how do I ignore a certain directory in the $oldPath? For instance, if we had the following:
root
dir1
dir 2
subdir 1
subdir 2
dir 3
subdir 1
dir 4
And we want to ignore dir 2 and all subdirectories when building the list
Final working script:
$oldPath = "\\server\share"
$newDrive = "I:"
$oldDate = Get-Date -Date 1/1/2012
$oldFiles = Get-ChildItem $oldPath -Recurse -File | Where-Object {($_.PSParentPath -notmatch '\\Ignore Directory') -and $_.lastwritetime -le $oldDate }
$oldDirs = Get-ChildItem $oldPath -Recurse | Where-Object {$_.PSIsContainer -and ($_.PSParentPath -notmatch '\\Ignore Directory')} | select-object FullName
$oldDirs = $oldDirs | select -Unique
foreach ($oldDir in $oldDirs) {
$strdir = $newDrive + "\" + ($oldDir | Split-Path -NoQualifier | Out-String).trim().trim("\")
if (!(Test-Path $strdir)) {
Write-Host "$strdir does not exist. Creating directory..."
mkdir $strdir | Out-Null
} # end if
} # end foreach
foreach ($file in $oldFiles) {
$strfile = $newDrive + "\" + ($file.FullName | Split-Path -NoQualifier | Out-String).trim().trim("\")
Write-Host "Moving $file.FullName to $strfile..."
Move-Item $file.FullName -Destination $strfile -Force -WhatIf
} # end foreach
$oldfiles | select pspath | Split-Path -NoQualifier | Out-File "\\nelson\network share\ArchivedFiles.txt"
Modify your Where-Object condition to:
... | Where-Object {($_.PSParentPath -notmatch '\\dir 2') -and ($_.lastWriteTime -le $oldDate)}
Also, you probably want to filter out directory items as well so that $oldFiles contains only files e.g.:
$oldFiles = Get-ChildItem $oldPath -Recurse | Where {!$_.PSIsContainer -and ($_.PSParentPath -notmatch '\\dir 2') -and ($_.lastWriteTime -le $oldDate)}
If you're on PowerShell v3 you can use a new parameter on Get-ChildItem to simplify this to:
$oldFiles = Get-ChildItem $oldPath -Recurse -File | Where {($_.PSParentPath -notmatch '\\dir 2') -and ($_.lastWriteTime -le $oldDate)}
Something like this should work:
$exclude = Join-Path $oldPath 'dir 2'
$oldFiles = Get-ChildItem $oldPath -Recurse | ? {
-not $_.PSIsContainer -and
$_.FullName -notlike "$exclude\*" -and
$_.LastWriteTime -le $oldDate
}
Try $oldFiles = Get-ChildItem $oldPath -Recurse -Exclude "dir 2" | Where-Object { $_.lastwritetime -le $oldDate}