I have thousands of files spanning 5 years which I would like to move into year/month folders. The file names all end with
_yyyy_mm_dd_wxyz.dat
I'm looking for ideas on how I can generate such file folders and move the files into the appropriate folders yyyy/mm using the windows command shell.
You'll need a Regular Expression with (capture groups) to extract year/month from the filename.
Assuming the year/month folder should be placed directly in files parent location.
untested with -Version 2
## Q:\Test\2018\07\23\SO_51485727.ps1
Push-Location 'x:\folder\to\start'
Get-ChildItem *_*_*_*_*.dat |
Where-Object {$_.BaseName -match '_(\d{4})_(\d{2})_\d{2}_[a-z]+$'} | ForEach-Object {
$TargetDir = "{0}\{1}" -f $Matches[1],$Matches[2]
if (!(Test-Path $TargetDir)){MD $TargetDir | Out-Null}
$_ | Move -Destination $TargetDir
}
Sample tree /f after running the script on my ramdriive:
PS A:\> tree /F
A:.
├───2017
│ └───07
│ test_2017_07_24_xyz.dat
└───2018
└───07
test_2018_07_24_xyz.dat
I have created this little quick and dirty script.
Things have been put in more variables than strictly needed, they could be combined in a single line, but I feel this adds clarity which I hope help you understand what happens.
As a note, I have used the date the item was last written to (created or edited).
If you want only the date the file was created and not the time the file was last edited, you could change LastWriteTime to CreationTime
#Load all files from the folder you wish to move on
$items = Get-ChildItem -Path "C:\SomeFolder\RestofPathToYourFiles"
foreach($item in $items) {
#Creates variables for year, month and day
$FolderYear = "$($item.LastWriteTime.Year)"
$FolderMonth = "$($item.LastWriteTime.Month)"
$FolderDay = "$($item.LastWriteTime.Day)"
#create variable with the new directory path
$NewPath = $item.Directory.FullName + "\" + $FolderYear + "\" + $FolderMonth + "\" + $FolderDay
#create variable with the new full path of the file
$NewFullPath = $NewPath + "\" + $item.Name
#test if the folder already is created, if not, create it
if((Test-Path -Path $NewPath) -eq $false) {
New-Item -Force -path $NewPath -Type Directory
}
#move the item to the new folder
Move-Item -Path $item.FullName -Destination $NewFullPath -Force
}
At the simplest, I'd do something like the following:
Determine the year and month related to a file
See if a folder exists already. If not, create it
Move the file
Example...
foreach ($file in $(ls .\stuff.txt)) {
$m = $file.LastWriteTime.Month.ToString("00")
$y = $file.LastWriteTime.Year
$dir = "{0}-{1}" -f $y, $m
New-Item -Name $dir -ItemType directory -ErrorAction SilentlyContinue | Out-Null
Move-Item -Path $file.Fullname -Destination $dir
}
Related
The code below has been wonderful so far for organising my hard-drives.
I do face this error when I transfer large amounts of data:
Move-Item : Cannot create a file when that file already exists.
This happens when I move a file that is duplicate, is there a way to rename the duplicate file in some sort of sequence?
That would be much appreciated :))
# Get all files
Get-ChildItem "C:\zAa" -File -Recurse | ForEach-Object {
# Get the modified date
$dt = Get-Date $_.LastWriteTime
$year = $dt.Year
$month = $dt.Month
# This adds "0" in front of the 1-9 months
if($dt.Month -lt 10) {
$month = "0" + $dt.Month.ToString()
} else {
$month = $dt.Month
}
# Remove leading '.' from the extension
$extension = $_.Extension.Replace(".", "")
# Where we want to move the file
$destinationFolder = "C:\zBb\$extension\$year\$month\"
# Ensure full folder path exists
if(!(Test-Path $destinationFolder)) {
New-Item -ItemType Directory -Force -Path $destinationFolder
}
# Copy/Move the item to it's new home
Move-Item $_.FullName $destinationFolder
}
I haven't been able to do much, I normally go find the duplicates and rename them manually.
Probably the easiest way to move a file with a unique name is to use a Hashtable that stores the filenames already present.
Then a simple loop can add a sequence number to its file name until it is no longer found in the Hashtable.
Next simply move the file under that new name.
Your code modified:
# Where we want to move the file
$destinationFolder = 'C:\zBb\{0}\{1:yyyy}\{1:MM}' -f $_.Extension.TrimStart("."), $_.LastWriteTime
# Ensure full folder path exists
$null = New-Item -Path $destinationFolder -ItemType Directory -Force
# create a Hashtable and store the filenames already present in the destination folder
$existing = #{}
Get-ChildItem -Path $destinationFolder -File | ForEach-Object { $existing[$_.Name] = $true }
# Get all source files
Get-ChildItem "C:\zAa" -File -Recurse | ForEach-Object {
# Copy/Move the item to it's new home
# construct the new filename by appending an index number in between brackets
$newName = $_.Name
$count = 1
while ($existing.ContainsKey($newName)) {
$newName = "{0}({1}){2}" -f $_.BaseName, $count++, $_.Extension
}
# add this new name to the Hashtable so it exists in the next run
$existing[$newName] = $true
# use Join-Path to create a FullName for the file
$newFile = Join-Path -Path $destinationFolder -ChildPath $newName
Write-Verbose "Moving '$($_.FullName)' as '$newFile'"
$_ | Move-Item -Destination $newFile -Force
}
enter image description hereI have a folder which has a bunch of files named: WBF123456, WBF135464, etc. These files need to be moved to the corresponding folder. At the moment I am using the commandline to manually enter the numbers of each file so they get moved, using this code:
$files = $args[0]
mv O:\SCAN\SecSur\*$files.pdf O:\SPG\G*\*\*$files
How can I automate this process?
It needs to identify the number in the filename, then move it to the folder containing the same number.
Any help would be great. Thanks.
I need to get the files on the left, inside the corresponding folders on the right.
Maybe the below solution will help you. You should change $origin_path and $destination_path
$origin_path= "C:\Users\geralexgr\Desktop\kati\files"
$destination_path = "C:\Users\geralexgr\Desktop\kati\folders"
Get-ChildItem $origin_path -Recurse -Include *.txt | ForEach-Object {
$folder = [regex]::Matches($_.Name, "\d+(?!.*\d+)").value
Move-Item $_.FullName $destination_path\$folder
}
The example will place files under the folders that match the numeric regex.
After powershell execution file WBF12 gets inside 12 folder
Apparently the files to move are .pdf files, so what you can do is get a list of those files in the source folder and then loop over that list to create (if needed) the destination subfolder and move the file there.
Try:
$destinationRoot = 'O:\SPG\G\SomeWhere' # enter the root folder destination path here
$filesToMove = Get-ChildItem -Path 'O:\SCAN\SecSur' -Filter '*.pdf' -File
foreach ($file in $filesToMove) {
$numName = $file.BaseName -replace '\D+' # leaving only the numbers
# create the target path for the file
$targetFolder = Join-Path -Path $destinationRoot -ChildPath $numName
# create that subfolder if it does not already exist
$null = New-Item -Path $targetFolder -ItemType Directory -Force
# now, move the file
$file | Move-Item -Destination $targetFolder
}
Seeing your screenshots, this might be a better approach for you.
$destinationRoot = 'O:\SPG\G\SomeWhere' # enter the root folder destination path here
# get a list of target folders for the files to be moved to and create a lookupHashtable from their names
$targets = #{}
Get-ChildItem -Path $destinationRoot -Directory | Where-Object {$_.Name -match '(\d+)'} | ForEach-Object {
$targets[$matches[1]] = $_.FullName # key is the number, value is the directory fullname
}
# get a list of files to move
$filesToMove = Get-ChildItem -Path 'O:\SCAN\SecSur' -Filter '*.pdf' -File | Where-Object {$_.Name -match '\d+'}
foreach ($file in $filesToMove) {
$numName = $file.BaseName -replace '\D+' # leaving only the numbers
# see if we have a target folder with that same number in its name
if ($targets.ContainsKey($numName)) {
$targetFolder = $targets[$numName]
Write-Host "Moving file $($file.Name) to $targetFolder"
$file | Move-Item -Destination $targetFolder
}
else {
Write-Warning "Could not find a destination folder for file $($file.Name).."
}
}
I am trying to create a powershell script that does the following:
Move the latest 3 subfolders(a-b-c with files) into a new folder with today's date without moving older files
Security has 3 subfolders generated today along with subfolders from previous days.
$localpath ="D:\Security\"
Get-ChildItem $localpath -Recurse | foreach {
$DateofFile = $_.LastWriteTime.ToShortDateString()
$Todaysfolder = Get-Date $DateofFile -Format yyyMMdd
$targetpath = $Todaysfolder
Create directory if it doesn't exsist
if (!(Test-Path $targetpath))
{
New-Item $targetpath -type directory
}
Get-ChildItem $localpath | Move-Item -Destination $targetpath
}
Process right now is grabbing all files - even the ones that were not created today and grouping them into one folder. This script will run at the end of the day to move those 3 subfolders (A-B-C) into today's folder "20200520" example
There's no need to recurse Get-ChildItem when moving the entire folder - saves you looping through contained files.
$FromPath ="C:\fromtest"
$ToPath = "C:\totest\"+(Get-Date -Format yyyMMdd)
$FoldersToMove = Get-ChildItem $FromPath |
Where-Object{$_.LastWriteTime -ge (Get-Date).date}
#Create directory if it doesn't exsist
if (!(Test-Path $ToPath)){
New-Item $ToPath -type directory
}
foreach ($Folder in $FoldersToMove){
$Folder | Move-Item -Destination $ToPath
}
I've finally have given up googling and come here out of desperation. Go easy on me I'm fairly new to Powershell.
So, the objective of the code below was to first look through the source folder, then read through each .zip file and move to the directory specified by the value in the hashtable. Unfortunately, this is not how they want it to work anymore.
Now I need to retain the parent folder from source: for example "DAL" and then create the proceeding folders based on the file names and finally move each .zip to its file specified folder. Also, it needs to go through each folder under source which will be at least 20 other folders with a unique 3 character names.
$srcRoot = "C:\Cloud\source\dal"
$dstRoot = "C:\Cloud\Destination"
##$map = #{}; dir -recurse | ? { !$_.psiscontainer} | % { ##$map.add($_.name,$_.PSChildName) }
# DAT and DEV will have to be excluded from folder creation
$map = {
#AEODDAT_201901 = "AEOD\2019\01"
#AEOMDEV_201902 = "AEOM\2019\01"
#AEOYDAT_201902 = "AEOY\2019\01"
}
$fileList = Get-ChildItem -Path $srcRoot -Filter "*.zip*" -File -Force -Recurse
foreach ($file in $fileList)
{
#Go through each file up to mapped string
$key = $file.BaseName.Substring(0,14)
if ($key -in $map.Keys)
{
$fileName = $file.Name
$dstDir = Join-Path -Path $dstRoot -ChildPath $map[$key]
#create direcotory if not in path
if (-not (Test-Path -Path $dstDir))
{
mkdir -Path $dstDir
}
Write-Verbose "Moving $($file.FullName)"
if (Test-Path -Path (Join-Path -Path $dstDir -ChildPath $fileName))
{
#Write error if name exists
Write-Error -Message "File $fileName already exists at $dstDir"
#move path
} else {
Move-Item -Path $($file.FullName) -Destination $dstDir
}
}
}
So C:\Cloud\source\DAL\AEODDAT20190101.zip should create folders in C:\Cloud\Destination\DAL\AEOD\2019\01\AEODDAT20190101.zip would be my desired output.
Welcome, Matt! (no pun intended) One of the habits I have in similar situations with destination folders is to Set-Location $dstRoot and create folders from the relative path. You can execute New-Item with the relative path and the syntax is simpler. For example, your If statement could look like this and it would work the same way (with a slightly different error message):
if ($key -in $map.Keys){
Set-Location $dstRoot
New-Item -ItemType Directory $map[$key] -ErrorAction Ignore #won't raise an error if it exists
Write-Verbose "Moving $($file.FullName)"
#this will raise an error if the file already exists, unless you specify -Force
Move-Item "$($file.FullName)" $map[$key]
}
EDIT: Found 2 issues.
$map is a Hashtable literal that should be preceded with #:
$map = #{
AEODDAT20190101 = "AEOD\2019\01"
You were missing the last character of the base file name by taking only the first 14 characters. AEODDAT2019010 didn't match AEODDAT20190101. This should fix it:
$key = $file.BaseName.Substring(0,15)
Wrote the following code to move files to a specific Year-Month folder on a drive. However, I would also like to zip the folder I wrote to at the end of the operation. How do I do that?
# Get the files which should be moved, without folders
$files = Get-ChildItem 'D:\NTPolling\InBound\Archive' -Recurse | where {!$_.PsIsContainer}
# List Files which will be moved
# $files
# Target Filder where files should be moved to. The script will automatically create a folder for the year and month.
$targetPath = 'D:\SalesXMLBackup'
foreach ($file in $files)
{
# Get year and Month of the file
# I used LastWriteTime since this are synced files and the creation day will be the date when it was synced
$year = $file.LastWriteTime.Year.ToString()
$month = $file.LastWriteTime.Month.ToString()
# Out FileName, year and month
$file.Name
$year
$month
# Set Directory Path
$Directory = $targetPath + "\" + $year + "\" + $month
# Create directory if it doesn't exsist
if (!(Test-Path $Directory))
{
New-Item $directory -type directory
}
# Move File to new location
$file | Move-Item -Destination $Directory
}
The intention is to move these files into a folder and zip them and archive them for later use. So I will schedule this once a month to run for the previous month
If you're using PowerShell v5 then you can use the Compress-Archive function:
Get-ChildItem $targetPath | Compress-Archive -DestinationPath "$targetPath.zip"
This will compress D:\SalesXMLBackup to D:\SalesXMLBackup.zip
This is the code I am using for unzipping all the files in a directory. You just need to modify it enough to zip instead of unzipping.
$ZipReNameExtract = Start-Job {
#Ingoring the directories that a search is not require to check
$ignore = #("Tests\","Old_Tests\")
#Don't include "\" at the end of $loc - it will stop the script from matching first-level subfolders
$Files=gci $NewSource -Fecurse | Where {$_.Extension -Match "zip" -And $_.FullName -Notlike $Ignore}
Foreach ($File in $Files) {
$NewSource = $File.FullName
#Join-Path is a standard Powershell cmdLet
$Destination = Join-Path (Split-Path -parent $File.FullName) $File.BaseName
Write-Host -Fore Green $Destination
#Start-Process needs the path to the exe and then the arguments passed seperately.
Start-Process -FilePath "C:\Program Files\7-Zip\7z.exe" -ArgumentList "x -y -o $NewSource $Destination" -Wait
}
}
Wait-Job $ZipReNameExtract
Receive-Job $ZipReNameExtract
Let me know if it helps.
The UnderDog...