I want to check .jpg file in the 2nd folder. 2nd folder has some subfolder. if .jpg exist in the subfolder of 2nd folder, I will copy a file from 1st folder to subfolder of 2nd folder based on the base name. I can do this part refer to this great answer How to copy file based on matching file name using PowerShell?
https://stackoverflow.com/a/58182359/11066255
But I want to do this process with infinity loop. Once I use infinity loop, I found that I have a lot of duplicate file. How do I make limitation, if I already copy the file, I will not copy again in the next loop.
Anyone can help me please. Thank you.
for(;;)
{
$Job_Path = "D:\Initial"
$JobError = "D:\Process"
Get-ChildItem -Path "$OpJob_Path\*\*.jpg" | ForEach-Object {
$basename = $_.BaseName.Substring(15)
$job = "$Job_Path\${basename}.png"
if (Test-Path $job) {
$timestamp = Get-Date -Format 'yyyyMMddhhmmss'
$dst = Join-Path $_.DirectoryName "${timestamp}_${basename}.gif"
$Get = (Get-ChildItem -Name "$OpJob_Path\*\*$basename.jpg*" | Measure-Object).Count
Copy-Item $job $dst -Force
}
}
}
File management 101 is that, Windows will not allow duplicate file names in the same location. You can only have duplicate files, if the name of the file is unique, but the content is the same. Just check for the filename, but they must be the same filename, and do stuff if it is not a match else do nothing.
Also, personally, I'd suggest using a PowerShell FileSystemWatcher instead of a infinite loop. Just saying...
This line …
$timestamp = Get-Date -Format 'yyyyMMddhhmmss'
… will always generate a unique filename by design, the content inside it is meaningless, unless you are using file hashing for the compare as part of this.
Either remove / change that line to something else, or use file hash (they ensure uniqueness regardless of name used) ...
Get-FileHash -Path 'D:\Temp\input.txt'
Algorithm Hash Path
--------- ---- ----
SHA256 1C5B508DED35A28B9CCD815D47ECF500ECF8DDC2EDD028FE72AB5505C0EC748B D:\Temp\input.txt
... for compare and prior to the copy if another if/then.
something like...
If ($job.Hash -ne $dst.Hash)
{Copy-Item $job.Path $dst.Path}
Else
{
#Do nothing
}
There are of course other ways to do this as well, this is just one idea.
Related
I'm still fairly new to powershell, so please bear with me.
I have 2 almost identical directories. Files and folders from the old directory were copied over to a new directory. However, during this transfer process, something happened to the last modified date. The files and folders in the new directory have incorrect last modified dates (ex: today).
Rather than re-doing the transfer process, which would take a long time, I'd like to write something in powershell that will compare the last modified dates of the two directories and correct the dates in the new directory.
I'd also like to check first if file/folder has been modified since the file transfer. There would be no reason to change the date on those files.
What I found from looking around and googling:
Link1 Link2 Link 3 Link 4
I know that I can get the last modified date of a file with:
(Get-Item $filename).LastWriteTime
where $filename is the file directory.
I also came across the following:
dir $directory | ? {$_.lastwritetime -gt "6/1/19" -AND $_.lastwritetime -lt "12/30/19"}
I know I can get information regarding files that were modified between 2 dates. This, I can tweak to make it so the "less than (-lt)" can be used to check files that were not modified past a certain date.
dir $directory | ? {$_.lastwritetime -lt `12/13/19'}
This accomplishes one of my goals. I have a means to check if a file has been modified past a certain or not.
I saw this for changing the value lastwritetime
$folder = Get-Item C:\folder1
$folder.LastWriteTime = (Get-Date)
and realized this was simply
(Get-Item $filename).LastWriteTime = (Get-Date)
Which I could modify to meet my goal of replacing the new file's last write time wit the old file's correct time:
(Get-Item $filename).LastWriteTime = (Get-Item $filename2).LastWriteTime
I suppose what I'm struggling with is kind of putting it all together. I know how to recurse through files/folders for copy-item or even Get-Childitem by adding the "recurse" parameter. But I'm having difficulties wrapping my head around recursively navigating through each directory to change the dates.
Thank you for your help.
You could do the following to compare the LastWriteTime property of the original files and folders to the copies, while keping in mind that files in the copy folder could have been updated since the last transfer date.
# set the date to the last transfer date to determine if the file was updated after that
$lastTransferDate = (Get-Date).AddDays(-10) # just for demo 10 days ago
# set the paths for the rootfolder of the originals and the rootfolder to where everything was copied to
$originalPath = 'D:\OriginalStuff'
$copyPath = 'E:\TransferredStuff'
# loop through the files and folders of the originals
Get-ChildItem -Path $originalPath -Recurse | ForEach-Object {
# create the full path where the copied file of folder is to be found
$copy = Join-Path -Path $copyPath -ChildPath $_.FullName.Substring($originalPath.Length)
# test if this object can be found
if (Test-Path -Path $copy) {
$item = Get-Item -Path $copy
# test if the item has not been updated since the last transfer date
if ($item.LastWriteTime -le $lastTransferDate) {
# set the timestamp the same as the original
$item.LastWriteTime = $_.LastWriteTime
}
}
}
Great job with what you've done so far.
Just put what you have into a foreach statement.
Foreach($item in (gci 'C:\Users\usernamehere\Desktop\folder123' -recurse)){
(Get-Item $item.FullName).LastWriteTime = (Get-Item "C:\Users\usernamehere\Desktop\folderabc\RandomFile.txt").LastWriteTime
}
We wrap the Get-Childitem command with the -recurse flag into parenthesis so that the command executes on it's own and becomes a collection for our foreach command to traverse. $item is the current item in the loop. We will want to use the .FullName property to know the full path to the file for the current item. With that said you will use $item.FullName together for the files you are going to set the date on.
We are copying a long list of files from their different directories into a single location (same server). Once there, I need to rename them.
I was able to move the files until I found out that there are duplicates in the list of file names to move (and rename). It would not allow me to copy the file multiple times into the same destination.
Here is the list of file names after the move:
"10.csv",
"11.csv",
"12.csv",
"13.csv",
"14.csv",
"15.csv",
"16.csv",
"17.csv",
"18.csv",
"19.csv",
"20.csv",
"Invoices_Export(16) - Copy.csv" (this one's name should be "Zebra.csv")
I wrote a couple of foreach loops, but it is not working exactly correctly.
The script moves the files just fine. It is the rename that is not working the way I want. The first file does not rename; the other files rename. However, they leave the moved file in place too.
This script requires a csv that has 3 columns:
Path of the file, including the file name (eg. c:\temp\smefile.txt)
Destination of the file, including the file name (eg. c:\temp\smefile.txt)
New name of the file. Just the name and extention.
# Variables
$Path = (import-csv C:\temp\Test-CSV.csv).Path
$Dest = (import-csv C:\temp\Test-CSV.csv).Destination
$NN = (import-csv C:\temp\Test-CSV.csv).NewName
#Script
foreach ($D in $Dest) {
$i -eq 0
Foreach ($P in $Path) {
Copy-Item $P -destination C:\Temp\TestDestination -force
}
rename-item -path "$D" -newname $NN[$i] -force
$i += 1
}
There were no error per se, just not the outcome that I expected.
Welcome to Stack Overflow!
There are a couple ways to approach the duplicate names situation:
Check if the file exists already in the destination with Test-Path. If it does, start a while loop that appends a number to the end of the name and check if that exists. Increment the number you append after each check with Test-Path. Keep looping until Test-Path comes back $false and then break out of the loop.
Write an error message and skip that row in the CSV.
I'm going to show a refactored version of your script with approach #2 above:
$csv = Import-Csv 'C:\temp\Test-CSV.csv'
foreach ($row in $csv)
{
$fullDestinationPath = Join-Path -Path $row.Destination -ChildPath $row.NewName
if (Test-Path $fullDestinationPath)
{
Write-Error ("The path '$fullDestinationPath' already exists. " +
"Skipping row for $($row.Path).")
continue
}
# You may also want to check if $row.Path exists before attempting to copy it
Copy-Item -Path $row.Path -Destination $fullDestinationPath
}
Now that your question is answered, here are some thoughts for improving your code:
Avoid using acronyms and abbreviations in identifiers (variable names, function names, etc.) when possible. Remember that code is written for humans and someone else has to be able to understand your code; make everything as obvious as possible. Someone else will have to read your code eventually, even if it's Future-You™!
Don't Repeat Yourself (called the "DRY" principle). As Lee_daily mentioned in the comments, you don't need to import the CSV file three times. Import it once into a variable and then use the variable to access the properties.
Try to be consistent. PowerShell is case-insensitive, but you should pick a style and stick to it (i.e. ForEach or foreach, Rename-Item or rename-item, etc.). I would recommend PascalCase as PowerShell cmdlets are all in PascalCase.
Wrap literal paths in single quotes (or double quotes if you need string interpolation). Paths can have spaces in them and without quotes, PowerShell interprets a space as you are passing another argument.
$i -eq 0 is not an assignment statement, it is a boolean expression. When you run $i -eq 0, PowerShell will return $true or $false because you are asking it if the value stored in $i is 0. To assign the value 0 to $i, you need to write it like this: $i = 0.
There's nothing wrong with $i += 1, but it could be shortened to $i++, if you want to.
When you can, try to check for common issues that may come up with your code. Always think about what can go wrong. "If I copy a file, what can go wrong? Does the source file or folder exist? Is the name pulled from the CSV a valid path name or does it contain characters that are invalid in a path (like :)?" This is called defensive programming and it will save you so so many headaches. As with anything in life, be careful not to go overboard. Only check for likely scenarios; rare edge-cases should just raise errors.
Write some decent logs so you can see what happened at runtime. PowerShell provides a pair of great cmdlets called Start-Transcript and Stop-Transcript. These cmdlets log all the output that was sent to the PowerShell console window, in addition to some system information like the version of PowerShell installed on the machine. Very handy!
I'm trying to teach myself enough powershell or batch programming to figure out to achieve the following (I've had a search and looked through a couple hours of Youtube tutorials but can't quite piece it all together to figure out what I need - I don't get Tokens, for example, but they seem necessary in the For loop). Also, not sure if the below is best achieved by robocopy or xcopy.
Task:
Define a list of files to retrieve in a csv (file name will be listed as a 13 digit number, extension will be UNKNOWN, but will usually be .jpg but might occasionally be .png - could this be achieved with a wildcard?)
list would read something like:
9780761189931
9780761189988
9781579657159
For each line in this text file, do:
Search a network folder and all subfolders
If exact filename is found, copy to an arbitrary target (say a new folder created on desktop)
(Not 100% necessary, but nice to have) Once the For loop has completed, output a list of files copied into a text file in the newly created destination folder
I gather that I'll maybe need to do a couple of things first, like define variables for the source and destination folders? I found the below elsewhere but couldn't quite get my head around it.
set src_folder=O:\2017\By_Month\Covers
set dst_folder=c:\Users\%USERNAME&\Desktop\GetCovers
for /f "tokens=*" %%i in (ISBN.txt) DO (
xcopy /K "%src_folder%\%%i" "%dst_folder%"
)
Thanks in advance!
This solution is in powershell, by the way.
To get all subfiles of a folder, use Get-ChildItem and the pipeline, and you can then compare the name to the insides of your CSV (which you can get using import-CSV, by the way).
Get-ChildItem -path $src_folder -recurse | foreach{$_.fullname}
I'd personally then use a function to edit the name as a string, but I know this probably isn't the best way to do it. Create a function outside of the pipeline, and have it return a modified path in such a way that you can continue the previous line like this:
Get-ChildItem -path $src_folder -recurse | foreach{$_.CopyTo (edit-path $_.fullname)}
Where "edit-directory" is your function that takes in the path, and modifies it to return your destination path. Also, you can alternatively use robocopy or xcopy instead of CopyTo, but Copy-Item is a powershell native and doesn't require much string manipulation (which in my experience, the less, the better).
Edit: Here's a function that could do the trick:
function edit-path{
Param([string] $path)
$modified_path = $dst_folder + "\"
$modified_path = $path.substring($src_folder.length)
return $modified_path
}
Edit: Here's how to integrate the importing from CSV, so that the copy only happens to files that are written in the CSV (which I had left out, oops):
$csv = import-csv $CSV_path
Get-ChildItem -path $src_folder -recurse | where-object{$csv -contains $_.name} | foreach{$_.CopyTo (edit-path $_.fullname)}
Note that you have to put the whole CSV path in the $CSV_path variable, and depending on how the contents of that file are written, you may have to use $_.fullname, or other parameters.
This seems like an average enough problem:
$Arr = Import-CSV -Path $CSVPath
Get-ChildItem -Path $Folder -Recurse |
Where-Object -FilterScript { $Arr -contains $PSItem.Name.Substring(0,($PSItem.Length - 4)) } |
ForEach-Object -Process {
Copy-Item -Destination $env:UserProfile\Desktop
$PSItem.Name | Out-File -FilePath $env:UserProfile\Desktop\Results.txt -Append
}
I'm not great with string manipulation so the string bit is a bit confusing, but here's everything spelled out.
I began to compare 2 folder structures to find files that did not match by date and size, but the requirment has been changed to 4 folders and I am stuck.
So here is what I am trying to do:
We upload several hundred folders\files to 4 different servers. The files must all match. Sometimes a file will not copy properly. So I need a script to read all four directories and compare all the files to make sure they match by size and date.
Output should only be a simple list that shows me the files that didn't match.
Any ideas?
Thanks.
I can do two folders but am lost on four. Also, this output is confusing. Not sure how to only list those that don't match.
$path1 = "\\path\folder
$path2 = "\\path\folder1
$dif = Compare-Object -ReferenceObject $path1 -DifferenceObject $path2 -Property FullName, Length, LastWriteTime
$dif | ft -AutoSize
I'd go about it with a hash based approach, and possibly use a database table somehwere to help yourself out. BTW, PSCX has the Get-Hash commandlet which will help you do this.
Basic approach
Traverse each server's desired folder-tree (you want to do this on the servers involved for performance reasons, not over a network share!) and generate a hash on each file you find. Store the hash and the full path and server name somewhere, preferrably a database table accessible from all four servers--it'll make processing much easier.
Then, if you've used a database table, write a few simple queries:
find any hash where there are fewer than 4 instances of the hash.
find any file path (you may have to process the path string to get it to the same relative root for each server ) where there are differing hashes (although this might be covered by 1. above).
All of this can be done from within PS, of course.
Why this way of doing things may be helpful
You don't have to run a four-way Compare-Object. The hashes serve as your point of comparison.
Your Powershell code to generate the hashes is one identical function that gets run on each server.
It scales. You could easily do this for 100 folders.
You end up with something easily manipulated and "distributed",i.e. accesible to the servers involved--the database table.
Downside
PSCX Get-Hash isn't very fast. This can easily be remedied by having PS fire some faster hash generating command, such as this one, md5sums.
How to do without using a database table
1. Write the hashes, file paths, severnames to files on each server as you are processing folders for hashes, and bring those files back when done.
2. Process the files into a hash table that keys on the hashcodes and counts each hash code.
3. You can have a parallel hash table (built at that same time as 2. while you pass throug the result files) that keys on each hash code to an array of paths/servers for that hash code.
4. Look for hash codes in hash table 1 with a count of less than 4. Use parallel hash table 2 to look up hash codes found with a count less 4, to find out what the file path(s) and server(s) were.
Try this:
Remember that the PrimaryPath has to be a masterlocation(contents are correct). Also, be consistent with how you write the paths(if you include the \ or not). Ex. Either use c:\folders\folder1\ for all paths or c:\folders\folder1.
Compare.ps1
Param(
[parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
[parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
)
#Get filelist with relativepath property
function Get-FilesWithRelativePath ($Path) {
Get-ChildItem $Path -Recurse | ? { !$_.PSIsContainer } | % {
Add-Member -InputObject $_ -MemberType NoteProperty -Name RelativePath -Value $_.FullName.Substring($Path.Length)
$_
}
}
#If path exists and is folder
if (Test-Path $PrimaryPath -PathType Container) {
#Get master fileslist
$Masterfiles = Get-FilesWithRelativePath (Resolve-Path $PrimaryPath).Path
#Compare folders
foreach ($Folder in $ComparePath) {
if (Test-Path $Folder -PathType Container) {
#Getting filelist and adding relative-path property to files
$ResolvedFolder = (Resolve-Path $Folder).Path
$Files = Get-FilesWithRelativePath $ResolvedFolder
#Compare and output filepath to missing or old file
Compare-Object -ReferenceObject $Masterfiles -DifferenceObject $Files -Property RelativePath, Length, LastWriteTime | ? { $_.SideIndicator -eq "<=" } | Select #{n="FilePath";e={Join-Path $ResolvedFolder $_.RelativePath}}
} else { Write-Error "$Folder is not a valid foldername. Foldertype: Compare" }
}
} else { Write-Error "$PrimaryPath is not a valid foldername. Foldertype: Master" }
My source files all reside in one folder whose path is contained in a variable named $template.
I need to specify the exact filename as each file goes to a different destination.
My goal is to merely concatenate the filename to the variable.
Example:
$template = "D:\source\templatefiles\"
Filename1 is: "graphic-183.jpg"
I have tried:
Join-Path $template graphic-183.jpg
Issuing this at the cli appears to do what I want.
But now, how do I reference this concatenated file path short of creating a new variable for each file? It isn't as simple as for-nexting my way through a list as depending on the filename that determines where the file goes.
I am toying with case else, elseIf, but surely it isn't this hard.
The bottom line is, I just want to prefix the folder path to each filename and hard code the destination as it will always be the same each time the script is run.
edit
I just edited this as I forgot to mention how I am trying to use this.
In my script I intend to have lines like:
Copy-Item -Path $template filename.ext -Destination $destfolder
It's the highlighted part above that I am trying to join $template to the filename.
Thanks for any advice.
-= Bruce D. Meyer
maybe this is what you want?
you can call cmdlets in place, using parentheses, like so:
Copy-Item -Path (Join-Path $template filename.ext) -Destination $destfolder
this causes PowerShell to go from "argument mode" to "expression mode" - i.e., it returns the output of the Join-Path cmdlet as an expression.
and yes, David's and Ansgar's suggestions are also helpful - try this to get full paths only:
(get-childitem $template) | select fullname
You could build the path like this:
$template = "D:\source\templatefiles\"
Copy-Item -Path "${template}filename.ext" ...
However, I think David's suggestion might be a better solution for your problem. You could map filenames to destination folders with a hash table and do something like this:
$locations = #{
"foo" = "C:\some",
"bar" = "C:\other",
...
}
Get-ChildItem $template | % { Copy-Item $_ $location[$_.Name] }