ForEach-Object to look in subfolders - powershell

I Have this powershell script “copFiles.ps1” that looks in a txt file "Filestocopy.txt" for a list and copies them to a destination
$source = "C:\Data\Filestocopy.txt"
$destination = "C:\Data\Models"
Get-Content $source | ForEach-Object {copy-item $_ $destination}
It’ll only copy the files if they’re in the same folder as the .ps1 file and it ignores subfolders, how can I get it to look in subfolders of the folder that its in, I gather I need to use the -recurse option but don’t know how to rewrite it so it works.
The .ps1 file is fired by a bat file.
Many thanks

I don't know how fast this will be, but you can give an array as the argument for the -Path parameter of Get-ChildItem add the -Recurse switch to dig out the files in subdirectories and simply pipe them along to Copy-Item. something like:
Get-ChildItem (Get-Content $Source) -Recurse |
Copy-Item -Destination $destination
You may also want to add the -File switch.
Update
Based on your comment I played around with this a a little more:
$source = "C:\Data\Filestocopy.txt"
$Destination = "C:\data\Models"
# Get-ChildItem (Get-Content $Source) -Recurse |
Get-ChildItem (Get-Content $Source) -Recurse -File |
ForEach-Object{
If( $_.Directory.FullName -eq $Destination )
{ # Don't work on files already present in the destination
# when the destination is under the current directory...
Continue
}
$FileNum = $null
$NewName = Join-Path -Path $Destination -ChildPath $_.Name
While( (Test-Path $NewName) )
{
++$FileNum
$NewName = Join-Path -Path $Destination -ChildPath ($_.BaseName + "_" + $FileNum + $_.Extension)
}
Copy-Item $_.FullName -Destination $NewName
}
This will increment the destination file name in cases where a file by that name already exists in the destination. If the destination is under the current directory it will prevent analyzing those files by comparing the path of the file to the destination. Files must have unique names in a given folder so I'm not sure how else it can be handled.

Related

Excluding Folders with Get-ChildItem - Need Help Debugging a Script

I've searched through both StackOverflow and SuperUser to try to figure this out, and I'm still getting plagued by a problem I can't figure out how to fix. I know it's something simple, but after playing with it for an hour I'm still stumped. Simple question: how the heck do I tell Get-Childitem to exclude folders?
Right up front here's the code that doesn't work:
$sourceDir="E:\Deep Storage"
$targetDir="W:\Deep Storage"
$excludeThese = 'Projects2','Projects3','Projects4';
Get-ChildItem -Path $sourceDir -Directory -Recurse |
where {$_.fullname -notin $excludeThese} |
Get-ChildItem -Path $sourceDir | ForEach-Object {
$num=1
$nextName = Join-Path -Path $targetDir -ChildPath $_.name
while(Test-Path -Path $nextName)
{
$nextName = Join-Path $targetDir ($_.BaseName + "_$num" + $_.Extension)
$num+=1
}
$_ | Move-Item -Destination $nextName -Force -Verbose -WhatIf
}
}
The underlying concept here already works:
$sourceDir="E:\Deep Storage"
$targetDir="W:\Deep Storage"
Get-ChildItem -Path $sourceDir -File -Recurse | ForEach-Object {
$num=1
$nextName = Join-Path -Path $targetDir -ChildPath $_.name
while(Test-Path -Path $nextName)
{
$nextName = Join-Path $targetDir ($_.BaseName + "_$num" + $_.Extension)
$num+=1
}
$_ | Copy-Item -Destination $nextName -Verbose
}
Basically what this does is it moves folders from one place to another, and if files exist in both places, it renames files coming in. It helps keep my archive drive clear. But there are three folders there that I want to exclude because I still pull assets from them regularly, so I don't need those files moved.
Hence the difference between the two code samples: in the first one, I'm trying to get Get-Childitem to exclude a specific trio of folders, while this second one just grabs everything all at once.
I tried just doing a straight -Exclude with $excludeThese as the variable, without success; I tried skipping the variable approach altogether and just putting the folder names in after -Exclude. Still didn't work. I also tried putting in the entire path to the folders I wanted to exclude. No good--no matter what I did, the -WhatIf showed that the script was trying to move everything, including the folders I was theoretically excluding.
The last trick I tried was one I came across here on SO, and that was to go a gci with the exclude argument first, then do another gci after it. That still failed, so now I have to turn to the experts for help.
I would use a regex string created from the (escaped) directory names to exclude to make sure files withing these folders are ignored.
Also, by using a lookup Hashtable of all file names already present in the target folder, figuring out if a file with a certain name already exists is extremely fast.
$sourceDir = 'E:\Deep Storage'
$targetDir = 'W:\Deep Storage'
$excludeThese = 'Projects2','Projects3','Projects4';
# create a regex string with all folder names to exclude combined with regex OR (|)
$excludeDirs = ($excludeThese | ForEach-Object { [Regex]::Escape($_) }) -join '|'
# create a lookup Hashtable and store the filenames already present in the destination folder
$existingFiles = #{}
Get-ChildItem -Path $targetDir -File | ForEach-Object { $existingFiles[$_.Name] = $true }
Get-ChildItem -Path $sourceDir -File -Recurse |
Where-Object {$_.DirectoryName -notmatch $excludeDirs} |
ForEach-Object {
# construct the new filename by appending an index number if need be
$newName = $_.Name
$count = 1
while ($existingFiles.ContainsKey($newName)) {
$newName = "{0}_{1}{2}" -f $_.BaseName, $count++, $_.Extension
}
# add this new name to the Hashtable so it exists in the next run
$existingFiles[$newName] = $true
# use Join-Path to create a FullName for the file
$newFile = Join-Path -Path $targetDir -ChildPath $newName
$_ | Move-Item -Destination $newFile -Force -Verbose -WhatIf
}
Assuming the excluded directories are at the top:
$sourceDir="E:\Deep Storage"
$excludeThese = 'Projects2','Projects3','Projects4'
get-childitem $sourcedir -exclude $excludethese | get-childitem -recurse

Moving Files based on filename

Im looking to move files based on the last half of the filename. Files look like this
43145123_Stuff.zip
14353135_Stuff.zip
2t53542y_Stuff.zip
422yg3hh_things.zip
I am only looking to move files that end in Stuff.zip
I have this in PowerShell so far but it only will move files according to the first half of a file name.
#set Source and Destination folder location
$srcpath = "C:\Powershelltest\Source"
$dstpath = "C:\Powershelltest\Destination"
#Set the files name which need to move to destination folder
$filterLists = #("stuff.txt","things")
#Get all the child file list with source folder
$fileList = Get-ChildItem -Path $srcpath -Force -Recurse
#loop the source folder files to find the match
foreach ($file in $fileList)
{
#checking the match with filterlist
foreach($filelist in $filterLists)
{
#$key = $file.BaseName.Substring(0,8)
#Spliting value before "-" for matching with filterlists value
$splitFileName = $file.BaseName.Substring(0, $file.BaseName.IndexOf('-'))
if ($splitFileName -in $filelist)
{
$fileName = $file.Name
Move-Item -Path $($file.FullName) -Destination $dstpath
}
}
}
There seems to be some differences between the state goal and what the code actually does. This will move the files to the destination directory. When you are confident that the files will be moved correctly, remove the -WhatIf from the Move-Item command.
$srcpath = "C:\Powershelltest\Source"
$dstpath = "C:\Powershelltest\Destination"
Get-ChildItem -File -Recurse -Path $srcpath |
ForEach-Object {
if ($_.Name -match '.*Stuff.zip$') {
Move-Item -Path $_.FullName -Destination $dstpath -WhatIf
}
}
Actually this can be written in PowerShell very efficiently (I hope I got the details right, let me know):
Get-ChildItem $srcpath -File -Force -Recurse |
where { ($_.Name -split "_" | select -last 1) -in $filterLists } |
Move-Item $dstpath
Alternatively, if you only want to look for this one particular filter, you can specify that directly, using wildcards:
Get-ChildItem $srcpath -Filter "*_Stuff.zip"

Renaming/Copying file with complicated name requirements

I have to create a script that searches for file, takes part of the folder name and move the file to a new location with that new name.
I am planning to use powershell for this but would be up willing to look for other options. This used for millions of files.
Example 1
sourcefolder\a\b\test_123456\example.txt -> \destinationfolder\example_123456.txt
Problem is I don't know how many folders deep the file is and the amount of folder name changes, I need everything after the last _
Example 2
sourcefolder\a\b\c\test_test_1234\example.txt -> \destinationfolder\example_1234.txt
I am researching how to script and will update question when I when I have some progress
FileInfo objects include many properties. One of these is the .Directory property which returns the directory (as DirectoryInfo object) that represents the parent folder the file is in. This Directory also has properties, like .Name.
You can use this like below:
$sourceFolder = 'D:\Test' # the root folder to search through
$destinationFolder = 'X:\Archive' # the destinationpath for the moved files
# make sure the destination folder exists
$null = New-Item -Path $destinationFolder -ItemType Directory -Force
# get a collection of FileInfo objects
# if you need more file extensions like both .txt and .log files, replace -Filter '*.txt' with -Include '*.txt', '*.log'
# this will be slower than using -Filter though..
$filesToMove = Get-ChildItem -Path $sourceFolder -File -Filter '*.txt' -Recurse | Where-Object {$_.Directory.Name -like '*_*'}
# using a foreach(..) is a bit faster than 'ForEach-Object'
foreach ($file in $filesToMove) {
# get the last part after '_' of the parent directory name
$suffix = ($file.Directory.Name -split '_')[-1]
# combine to create the new path and filename
$target = Join-Path -Path $destinationFolder -ChildPath ('{0}_{1}{2}' -f $file.BaseName, $suffix, $file.Extension)
$file | Move-Item -Destination $target -Force -WhatIf
}
Take off the WhatIf safety switch if you are satisfied what is displayed on screen about what would be moved is correct.
You don't even need the foreach loop because Move-Item can handle a scriptblock as parameter for the Destination like this:
$sourceFolder = 'D:\Test' # the root folder to search through
$destinationFolder = 'X:\Archive' # the destinationpath for the moved files
# make sure the destination folder exists
$null = New-Item -Path $destinationFolder -ItemType Directory -Force
# get a collection of FileInfo objects
# if you need more file extensions like both .txt and .log files, replace -Filter '*.txt' with -Include '*.txt', '*.log'
# this will be slower than using -Filter though..
$filesToMove = Get-ChildItem -Path $sourceFolder -File -Filter '*.log' -Recurse |
Where-Object {$_.Directory.Name -like '*_*'} |
Move-Item -Destination {
$suffix = ($_.Directory.Name -split '_')[-1]
Join-Path -Path $destinationFolder -ChildPath ('{0}_{1}{2}' -f $_.BaseName, $suffix, $_.Extension)
} -Force
Here, the $_ Automatic variable is used instead of a variable you define in a foreach loop.
P.S. If you only need files from subfolders with a name ending in _ followed by numbers only as opposed to names like sub_folder, change the Where-Object {...} clause in the code to
Where-Object {$_.Directory.Name -match '_\d+$'}

Rename After Copy - Powershell

I am trying to use powershell to copy one type of file (.xlsx) from one folder to another.
Once the copy is completed, I would like the extension on the original file to be changed. (.xlsx to .cmp)
I have the copy part down (below) but I am lost when it comes to the rename. Can you guys please help. I am a PS noob! Thank you.
$src = "C:\Users\x\Documents\Test1"
$dst = "C:\Users\x\Documents\Test2"
Get-ChildItem $src -Filter "*.xlsx" | Move-Item -Destination $dst -Force
As far as I know, you'll have to iterate over your files to be able to perform this rename.
# Set-up variables
$sourcePath = "C:\temp"
$sourceExtension = "txt"
$destinationPath = "C:\temp2"
$destinationExtension = "cmp"
# Grab the list of files
$files = Get-ChildItem -Path $sourcePath -Filter "*.$sourceExtension"
# Loop over the files
foreach ($file in $files) {
# Construct the new file name
$newFileName = (Join-Path -Path $destinationPath -ChildPath $file.BaseName) + ".$destinationExtension"
Write-Output "New File Name = $newFileName"
# Move the file to the new destination with its new name!
Move-Item -Path $file.FullName -Destination $newFileName
}
Note: BaseName = filename without extension
This should do it:
$src = "C:\Users\x\Documents\Test1"
$dst = "C:\Users\x\Documents\Test2"
Get-ChildItem $src -Filter "*.xlsx" | ForEach-Object {
Copy-Item $_ -Destination $dst
Rename-Item $_ -NewName ($_.Name -Replace '.xlsx','.cmp')
}
Uses a ForEach-Object loop to go through each item in the $src folder. Then for each item (represented inside the loop as $_) we use Copy-Item to copy it to the destination Then use Rename-Item with a -Replace to change the file extension.

Move-Item moves entire source directory instead of specified files

In PowerShell, I'm trying to use Move-Item to move all files containing the string "2017", located in $Source, to $MovieDst
The files that I want to move in $Source are individual .mkv files with the string "2017" in the filename, located at the root of $Source - they are not in a directory.
If I hash out Move-Item and un-hash Write-Host $NoDirMovie, it outputs the following:
ExampleFile.2017.mkv
ExampleFile2.2017.mkv
So, it definitely knows which are the correct files to be referencing. But as soon as I use Move-Item, it moves the entire $source directory into $MovieDst. I can't use -Exclude *COMPLETED_DOWNLOADS* as it includes the very files that I want to move.
So, I'm a bit stuck - It can't move my files, because it FIRST wants to move the entire directory containing them. Is there a way to stop it from moving the entire $Source directory, and JUST move only the contents of $Source that contain the string "2017" ?
$Source = "C:\TV Shows 7\COMPLETED_DOWNLOADS"
$MovieDst = "C:\TV Shows 7\COMPLETED DOWNLOADS_Sorted\MOVIES"
$NoDirMovies = dir $Source *2017*
foreach($NoDirMovie in $NoDirMovies | where {$_ -ne $Source})
{
Move-Item -Path $Source -Destination $MovieDst
#Write-Host $NoDirMovie
}
Why are you storing the files you want in $NoDirMovie, then passing $Source to Move-item?
Surely you want:
foreach($NoDirMovie in $NoDirMovies | where {$_ -ne $Source})
{
Move-Item -Path $NoDirMovie -Destination $MovieDst
#Write-Host $NoDirMovie
}
Edit
That's a simple fix, which I haven't tried. A few things:
dir is an alias for the built-in PowerShell Cmdlet Get-ChildItem
Your loop Where is illogical. $NoDirMovies are files in directory $Source, so they will never equal $Source (assume this was an attempt to not copy directory?)
Worth checking if the directory you are trying to copy to exists.
V2 (I tried this one)
$Source = "C:\TV Shows 7\COMPLETED_DOWNLOADS"
$MovieDst = "C:\TV Shows 7\COMPLETED DOWNLOADS_Sorted\MOVIES"
$filter = "2017"
If(!(Test-Path $MovieDst)){
New-Item -ItemType Directory -Path $MovieDst
}
foreach($file in (Get-ChildItem $Source | Where-object {$_.Name -match $filter})){
Move-Item -Path $file.FullName -Destination $MovieDst
}
IMHO your approach is overcomplicated. Move-Item accepts piped input, so
$Source = "C:\TV Shows 7\COMPLETED_DOWNLOADS"
$MovieDst = "C:\TV Shows 7\COMPLETED DOWNLOADS_Sorted\MOVIES\"
Get-ChildItem -Path $Source -Filter "*2017*.mkv" |Where {!($_.PSIsContainer)}
Move-Item -Destination $MovieDst -WhatIf
If the outpout looks OK, remove the -WhatIf