I apologize for the naivety of this post, please forgive my newness.
I have approximately 20,000 network files to filter through and copy certain ones to a local drive.
File List Requirements:
Excel files of various type (.xls, .xlsx, .xlsm)
Only files modified after 4/1/2022
Only files that contain "2022" in the filename
If the file meets those requirements then:
Copy the file to a local folder (original folder path structure doesn't matter, all files can go in one folder)
Output the original path and filename to a txt file, along with the lastwritedate
I have created the following code, which successfully obtains all excel files and creates the filename list
Get-ChildItem "D:\network_folder\" -Filter *.xls -Recurse | Select-Object -Property FullName, LastWriteTime |
Export-Csv -Path "C:\local_folder\file_list.csv" -Force -NoTypeInformation
However I cannot figure out the following issues:
how and where to filter for the lastwritetime
how and where to filter for the "2022" in the name
how and where to copy the files to the local folder
right now I'm just putting this all in the command line, do I need to make some file to run this process?
Thank you for any assistance you can provide!
I guess you want something like this.
It searches for files in the source folder with 2022 in the name and having .xls (or anything following xls) as extension.
It then loops over these items, creates the subfolder structure where they were found in the destination folder, copies the files and finally writes out a CSV file with information of the original file.
$sourcePath = 'D:\network_folder'
$destination = 'D:\dest_folder'
$refDate = [datetime]::new(2022,4,2) # --> next day date as of midnight
Get-ChildItem -Path $sourcePath -Filter '*2022*.xls*' -File -Recurse |
Where-Object {$_.LastWriteTime -ge $refDate} | ForEach-Object {
# create the destination folder if it does not already exist
$target = Join-Path -Path $destination -ChildPath $_.DirectoryName.Substring($sourcePath.Length)
$null = New-Item -Path $target -ItemType Directory -Force
# copy the file
$_ | Copy-Item -Destination $target
# output the wanted properties from the original file
$_ | Select-Object Name, FullName, LastWriteTime
} | Export-Csv -Path "C:\local_folder\file_list.csv" -Force -NoTypeInformation
Related
Got another multi-step process I'm looking to streamline. Basically, I'm looking to build a Powershell script to do three things:
Get-Childitem to look for folders with a specific name (we'll call it NAME1 as a placeholder)
For each folder it finds that has the name, I want it to output the full directory to a TXT file (so that in the end I wind up with a text file that has a list of the results it found, with their full paths; so if it finds folders with "NAME1" in five different subdirectories of the folder I give it, I want the full path beginning with the drive letter and ending with "NAME1")
Then I want it to take the list from the TXT file, and copy each file path to another drive and preserve directory structure
So basically, if it searches and finds this:
D:\TEST1\NAME1
D:\TEST7\NAME1
D:\TEST8\NAME1\
That's what I want to appear in the text file.
Then what I want it to do is to go through each line in the text file and plug the value into a Copy-Item (I'm thinking the source directory would get assigned to a variable), so that when it's all said and done, on the second drive I wind up with this:
E:\BACKUP\TEST1\NAME1
E:\BACKUP\TEST7\NAME1
E:\BACKUP\TEST8\NAME1\
So in short, I'm looking for a Get-Childitem that can define a series of paths, which Copy-Item can then use to back them up elsewhere.
I already have one way to do this, but the problem is it seems to copy everything every time, and since one of these drives is an SSD I only want to copy what's new/changed each time (not to mention that would save time when I need to run a backup):
$source = "C:\"
$target = "E:\BACKUP\"
$search = "NAME1"
$source_regex = [regex]::escape($source)
(gci $source -recurse | where {-not ($_.psiscontainer)} | select -expand fullname) -match "\\$search\\" |
foreach {
$file_dest = ($_ | split-path -parent) -replace $source_regex,$target
if (-not (test-path $file_dest)){mkdir $file_dest}
copy-item $_ -Destination $file_dest -force -verbose
}
If there's a way to do this that wouldn't require writing out a TXT file each time I'd be all for that, but I don't know a way to do this the way I'm looking for except a Copy-Item.
I'd be very grateful for any help I can get with this. Thanks all!
If I understand correctly, you want to copy all folders with a certain name, keeping the original folder structure in the destination path and copy only files that are newer than what is in the destination already.
Try
$source = 'C:\'
$target = 'E:\BACKUP\'
$search = 'NAME1'
# -ErrorAction SilentlyContinue because in the C:\ disk you are bound to get Access Denied on some paths
Get-ChildItem -Path $source -Directory -Recurse -Filter $search -ErrorAction SilentlyContinue | ForEach-Object {
# construct the destination folder path
$dest = Join-Path -Path $target -ChildPath $_.FullName.Substring($source.Length)
# copy the folder including its files and subfolders (but not empty subfolders)
# for more switches see https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy
robocopy $_.FullName $dest /XO /S /R:0
}
If you don't want console output of robocopy you can silence it by appending 2>&1, so neither stdout nor stderr is echoed
If you want to keep a file after this with both the source paths and the destinations, I'd suggest doing
$source = 'C:\'
$target = 'E:\BACKUP\'
$search = 'NAME1'
$output = [System.Collections.Generic.List[object]]::new()
# -ErrorAction SilentlyContinue because in the C:\ disk you are bound to get Access Denied on some paths
Get-ChildItem -Path $source -Directory -Recurse -Filter $search -ErrorAction SilentlyContinue | ForEach-Object {
# construct the destination folder path
$dest = Join-Path -Path $target -ChildPath $_.FullName.Substring($source.Length)
# add an object to the output list
$output.Add([PsCustomObject]#{Source = $_.FullName; Destination = $dest })
# copy the folder including its files and subfolders (but not empty subfolders)
# for more switches see https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy
robocopy $_.FullName $dest /XO /S /R:0
}
# write the output to csv file
$output | Export-Csv -Path 'E:\backup.csv' -NoTypeInformation
so trying to find a way to combine a couple of things the Stack Overflow crowd has helped me do in the past. So I know how to find folders with a specific name and move them where I want them to go:
$source_regex = [regex]::escape($sourceDir)
(gci $sourceDir -recurse | where {-not ($_.psiscontainer)} | select -expand fullname) -match "\\$search\\" |
foreach {
$file_dest = ($_ | split-path -parent) -replace $source_regex,$targetDir
if (-not (test-path $file_dest)){mkdir $file_dest}
move-item $_ -Destination $file_dest -force -verbose
}
And I also know how to find and delete files of a specific file extension within a preset directory:
Get-ChildItem $source -Include $searchfile -Recurse -Force | foreach{ "Removing file $($_.FullName)"; Remove-Item -force -recurse $_}
What I'm trying to do now is combine the two. Basically, I'm looking for a way to tell Powershell:
"Look for all folders named 'Draft Materials.' When you find a folder with that name, get its full path ($source), then run a command to delete files of a given file extension ($searchfile) from that folder."
What I'm trying to do is create a script I can use to clean up an archive drive when and if space starts to get tight. The idea is that as I develop things, a lot of times I go through a ton of incremental non-final drafts (hence folder name "Draft Materials"), and I want to get rid of the exported products (the PDFs, the BMPs, the AVIs, the MOVs, atc.) and just leave the master files that created them (the INDDs, the PRPROJs, the AEPs, etc.) so I can reconstruct them down the line if I ever need to. I can tell the script what drive and folder to search (and I'd assign that to a variable since the backup location may change and I'd like to just change it once), but I need help with the rest.
I'm stuck because I'm not quite sure how to combine the two pieces of code that I have to get Powershell to do this.
If what you want is to
"Look for all folders named 'Draft Materials.' When you find a folder with that name, get its full path ($source), then run a command to delete files of a given file extension ($searchfile) from that folder."
then you could do something like:
$rootPath = 'X:\Path\To\Start\Searching\From' # the starting point for the search
$searchFolder = 'Draft Materials' # the folder name to search for
$deleteThese = '*.PDF', '*.BMP', '*.AVI', '*.MOV' # an array of file patterns to delete
# get a list of all folders called 'Draft Materials'
Get-ChildItem -Path $rootPath -Directory -Filter $searchFolder -Recurse | ForEach-Object {
# inside each of these folders, get the files you want to delete and remove them
Get-ChildItem -Path $_.FullName -File -Recurse -Include $deleteThese |
Remove-Item -WhatIf
}
Or use Get-ChildItem only once, having it search for files. Then test if their fullnames contain the folder called 'Draft Materials'
$rootPath = 'X:\Path\To\Start\Searching\From'
$searchFolder = 'Draft Materials'
$deleteThese = '*.PDF', '*.BMP', '*.AVI', '*.MOV'
# get a list of all files with extensions from the $deleteThese array
Get-ChildItem -Path $rootPath -File -Recurse -Include $deleteThese |
# if in their full path names the folder 'Draft Materials' is present, delete them
Where-Object { $_.FullName -match "\\$searchFolder\\" } |
Remove-Item -WhatIf
In both cases I have added safety switch -WhatIf so when you run this, nothing gets deleted and in the console is written what would happen.
If that info shows the correct files are being removed, take off (or comment out) -Whatif and run the code again.
I'm trying to get the names of the files within the directory and the sub directories within the "parent" directory.
I managed to solve my problem in parts, but in the last directory it is multiplying the files (as if I was checking this folder twice).
If I have 2 items, he is exporting me 4 to that last directory.
I managed to get the files from the directories, but if I have any files in the root directory (parent directory), it won't list.
Parent dir "\XML"
File1.xlsx
File2.txt
Folder2
Dir "Folder2"
File1.xml
File2.xml
So, in the CSV file exported will be 4 rows, which is duplicate files from "Folder2" folder. And it won't bring me the files in the "XML" folder, which is the parent folder.
Powershell script:
# To execute the script without agreeing with the execution policy
Set-ExecutionPolicy Bypass -Scope Process
# Defines the parent directory, where all files and folders inside it will be
$DirPai = 'D:\Users\F02579\Desktop\XML'
# Variable to store all directories within the parent directory
$DirPastas = (Get-ChildItem -Directory -Recurse $DirPai).FullName
# Variable that will keep the final result
$results = #()
foreach ($Dir in $DirPastas)
{
# Write the directory name
Write-Host $Dir
# Get the file details
$Arquivos = Get-ChildItem -Path $Dir -Recurse | Select-Object Directory, Name, Lenght, LastWriteTime, #{Name="Extension";Expression={$_.Extension}} #| Where-Object "Extension"-ne ''
# Store the result for each path
$results += $Arquivos
}
# Defines the directory to which the final file will be exported
$DiretorioExportacao = 'D:\Users\F02579\Desktop\XML\I_PI_LISTFILES.csv'
# Export the result to CSV in the previously informed directory
$results | Export-Csv -Path $DiretorioExportacao -NoTypeInformation -Encoding UTF8
You could reduce your code to the following.
Keep only the first Get_ChildItem. Remove the -Dir switch (ie. get files and folders)
Pipe directly to Select-Object. Don't bother with an explicit loop.
This will give a result with five items: four files and one folder.
You also had a typo with 'length'.
Cheers.
# To execute the script without agreeing with the execution policy
Set-ExecutionPolicy Bypass -Scope Process
# Defines the parent directory, where all files and folders inside it will be
$DirPai = 'D:\Users\F02579\Desktop\XML'
$results = (Get-ChildItem -Recurse $DirPai) | Select-Object Directory, Name, Length, LastWriteTime, #{Name="Extension";Expression={$_.Extension}} #| Where-Object "Extension"-ne ''
# Defines the directory to which the final file will be exported
$DiretorioExportacao = 'D:\Users\F02579\Desktop\XML\I_PI_LISTFILES.csv'
# Export the result to CSV in the previously informed directory
$results | Export-Csv -Path $DiretorioExportacao -NoTypeInformation -Encoding UTF8
You already have all the directories captured so in your loop you need to specify the -File parameter and there is no need for the -Recurse parmaeter.
$Arquivos = Get-ChildItem -Path $Dir -File |
Select-Object #{Name="Directory",Expression($Dir),
Name, Lenght, LastWriteTime,
#{Name="Extension";Expression={$_.Extension}}
HTH
Ultimately, I need a solid PowerShell script that will take a folder with several hundred video files, import the existing file names into the program, lookup the new file name in a CSV, and rename it. The old filename is simply (ie. File1.mp4, File2.mp4, etc.) I would like to appended a date to the front of the file in the format of (YYYY-MM-DD).
For testing, I created a folder on my desktop with (10) text files, each with a unique file name.
My CSV file appears as follows:
Image of CSV
The "newfilename" column, was created by using the Concatenate command in Excel.
`(=CONCATENATE(TEXT(A2, "yyyy-mm-dd")," ", B2)`
As much as I would just like PowerShell to handle everything, I feel using Excel for most of this might be the best way.
In my testing, everything was in one folder. However, at work, I will have video files on one drive, and the script will have to be in a folder on my desktop. Because I am in a corporate network, I need a special batch file to run my scripts, which is nothing new. I just modify the script name, and away it goes!
So what commands do I need to do in order to have the script separate from the video files AND the CSV file?
Here is the code that I have so far. Everything works when it's in one folder.
PS C:\Users\ceran\Desktop\Rename Project> Import-Csv -Path .\MyFileList.csv | ForEach-Object {
>> $Src = Join-Path -Path $TargetDir -ChildPath $_.filename
>> $Dst = Join-Path -Path $TargetDir -ChildPath $_.newfilename
>> Rename-Item -Path $Src -NewName $Dst
>> }
Thanks in advance for the help!
Chris
I'm not sure what the date column is in your Excel file and if you want to rename all files in the folder, but if that is the case, you don't need a csv file at all and can do this:
$sourceFolder = 'X:\Path\to\the\video\files' # change this to the real path
Get-ChildItem -Path $sourceFolder -Filter '*.mp4' -File | # iterate through the files in the folder
Where-Object {$_.Name -notmatch '^\d{4}-\d{2}-\d{2}'} | # don't rename files that already start with the date
Rename-Item -NewName { '{0:yyyy-MM-dd} {1}' -f $_.LastWriteTime, $_.Name } -WhatIf
This uses parameter -Filter '*.mp4', to get only files with an .mp4 extension. For the files in your testfolder (Desktop\Rename Project), change this to -Filter '*.txt'.
If you want all files renamed, no matter what the extension, simply remove the Filter from the cmdlet.
Because of the -WhatIf switch, no file is actually renamed and the code just shows in the console what would happen. Once satisfied that this is OK, remove the -WhatIf
Hope that helps.
$targetdir="C:\path\to\where\our\file\directory\is"
$pathtocsv="c:\path\to\csv.csv"
Import-Csv -Path $pathtocsv | ForEach-Object {
$Src = Join-Path -Path $TargetDir -ChildPath $_.filename
$Dst = Join-Path -Path $TargetDir -ChildPath $_.newfilename
Rename-Item -Path $Src -NewName $Dst
}
Why would this not work in any situation?
By the way, if the csv had the columns path and newname, it could be piped directly to rename-item:
path,newname
file.txt,file2.txt
import-csv ren.csv | Rename-Item -whatif
What if: Performing the operation "Rename File" on target "Item: /Users/js/foo/file.txt Destination: /Users/js/foo/file2.txt".
Hi I have a folder Called "A" and folder "A" has files and sub folders within it. I also have another folder directory called "Exclusion" with some copied files and folders from "A" within it. I'm looking for a Powershell script or Command Line option that will COPY & MOVE all the objects from A that are NOT found in the Exclusion directory to a new folder directory called "Output".
Thanks,
-B
Use Get-ChildItem to get a list of files in your exclusion directory, then take only the names of the files and hold those in an array.
Optionally use New-Item with the -Force parameter to ensure that your output directory exists before sending files there.
Next use Get-ChildItem to iterate through all files in our source (A) directory, use Where-Object and the -notin operator to exclude any files which have the same names as those gathered from your exclusion directory, then use Move-Item to move the files to your destination (Output) directory.
[string[]]$filenamesToExclude = Get-ChildItem -Path 'c:\somewhere\exclusion' -Recurse | Select-Object -ExpandProperty Name
New-Item -Path 'c:\somewhere\output\' -ItemType 'Directory' -Force | Out-Null #ensure the target directory exists / don't output this command's return value to the pipeline
Get-ChildItem -Path 'c:\somewhere\A' -Recurse | Where-Object {$_.Name -notin $filenamesToExclude} | Move-Item -Destination 'c:\somewhere\output\'