I have to "almost" mirror two folders.
One folder contains the output from a tool with the file format *_output.txt. Files are continuously added here as the tool runs and it will produce hundreds of thousands of files.
The other one should contain the same files as input for another tool, but that tool expects the format to be *_input.txt
My current solution is a powershell script that loops through the first folder, checks if the renamed file exists and if it doesn't, copies and renames it with Copy-Item. This, however, is proving very inefficient once the file number goes high enough. I would like to improve this.
Is it possible to somehow make use of robocopy's /MIR and also rename files in the second folder? I would like to prevent the original files being mirrored if a renamed file exists.
Is such a thing possible?
You could use FileSystemWatcher:
$watcher = New-Object -TypeName System.IO.FileSystemWatcher -Property #{Path='c:\temp\lib'; Filter='*_output.txt'; NotifyFilter=[IO.NotifyFilters]::FileName, [IO.NotifyFilters]::LastWrite};
Register-ObjectEvent -InputObject $watcher -EventName Created -Action {Copy-Item $event.SourceEventArgs.FullPath $($event.SourceEventArgs.FullPath.Replace("output","input"))};
$watcher.EnableRaisingEvents = $true
Related
So I have been trying to figure out how to create user-specific home folders and give the user FullAccess rights to his/her own (new) Homefolder. The location of the Homefolders is always the same, so I want to make it work by just changing the names to the correct usernames.
So far I have come up with the following code. I will include the first part which does work, because it might help to understand what I'm aiming at. Also, I'm fairly new at powershell, so apologies for any amateur coding you may find...
###Create a new Homefolder
##Set the user(s) that need a new Homefolder (Use the SamAccountName)
$UserList = "Guest1","Guest2","Guest3"
foreach ($User in $UserList)
{
##Set the properties of the folder
$File = [PSCustomObject]#{
Name = "$User.test"
Path = "C:\Users\ItsMe\Documents\Homefolder test\"
ItemType = "directory"
}
##Create the directory in the specified path
New-Item -Path $File.Path -Name $File.Name -ItemType $File.ItemType
#Test if the folder is successfully created, before moving on.
Test-Path "C:\Users\ItsMe\Documents\Homefolder test\Guest1.test"
#Get the path of the new directory
$DirPath = $("$File.Path\$File.Name")
###Set Acl to assign FullAccess rights
$NewAcl = Get-Acl -Path "$DirPath"
## Set properties
$identity = "$User"
$fileSystemRights = "FullControl"
$type = "Allow"
}
The code goes a little further, but this is where the main error lies.
The output gives me the folders Guest1.test, Guest2.test and Guest3.test in the correct location (the ".test" after the SamAccountName is necessary, as it will be replaced by the Company name in the real script).
After that however, I get the following error a couple times.
Get-Acl : Cannot find drive. A drive with the name '#{Name=Guest1.test; Path=C' does not exist.
The $DirPath variable does not take the right path of the newly created folders. The Test-Path command confirms that the folder is created before the error. By using Write-Host $DirPath I found that it saved the following value:
Write-Host $DirPath
#{Name=Guest3.test; Path=C:\Users\ItsMe\Documents\Homefolder test\; ItemType=directory}.Path\#{Name=Guest3.test; Path=C:\Users\ItsMe\Documents\Homefolder test\; ItemType=directory}.Name
When I run Get-Acl by manually setting the path (that I want the $DirPath variable to be), after the folders have been created, it works like intended:
Get-Acl -Path "C:\Users\ItsMe\Documents\Homefolder test\Guest1.test"
I have tried to take the whole Acl part out of this "foreach" section and create another "foreach" to assign the FullAccess rights for each user, but so far I have not been able to make that work either (It could be the best way, but I just have not figured it out yet).
Any tips on how to make this work will be appreciated. I feel like the current structure might be wrong.
You might try:
$DirPath = $($File.Path\$File.Name)
Also, if you run test-path inside the foreach loop after you create the folder, you’ll know whether it’s now there or not.
Create a separate variable for each value in $File; it’s a hashtable.
I actually don’t know that it’s buying you anything based on the use-case.
ItemType is unchanging; I’d define it above the loop.
I’d build the user’s folder path by concatenating the base to the Username with .test
I am having a hard time defining this issue, but basically what I would like to know is "what are the symbolic variable names connected to files' metadata (preferably on a Windows installation)".
For example, taking a .mp3 file, checking its properties yields a Title, Bit Rate, Folder Path etc. descriptions. What I want to know is the name of the fields seen by programs (i.e. Title->title, Bit Rate->bit_rate etc.) if it makes any sense, as I've been trying to index some files and I'd like to gather as much info on them as possible.
I'm not convinced that there is such a thing as "symbolic names" for the metadata, especially not in relation to PowerShell. I suspect that Windows maintains support for a certain number of popular formats, and offers functionality through Explorer to view and sometimes edit them. I haven't found a source to prove this theory, but research implicitly supports it: there's several dozen search results about how to retrieve a file's metadata in PowerShell, and they all seem to suggest roughly the same approach (for example this blog post): using a Shell object to gather the information.
Since you tagged this PowerShell, here's my take on boiling it down to the essentials:
$path = 'C:\temp\file.txt' # pick a path
$parent = Split-Path -Parent $path # get the directory
$shell = New-Object -ComObject Shell.Application # get ourselves a shell
$folder = $shell.NameSpace($parent) # get a "folder namespace"
$file = $Folder.Items() | where { $_.Path -eq $path } # get the file itself from the folder
$count = 0 # zero our iterator
$object = New-Object PSObject # make a fresh object to hold our output
While ($folder.GetDetailsOf($folder.Items, $count) -ne "") { # iterate over the available metadata tags for the folder, and for each one get the value from the file
$object | Add-Member -Force NoteProperty ($folder.GetDetailsOf($folder.Items, $count)) ($folder.GetDetailsOf($file, $count))
$count += 1
}
Write-Output $object
Note that the attributes available for a given file are obviously not all of the attributes that could possibly be supported for any file, and additionally are not necessarily "symbolic names". I suspect that the process of querying the shell object causes it to examine the files in a folder and extract metadata that Windows recognizes--it might even do this based on the view type selected for the folder (Photos, Music, Documents, etc.).
As for writing the information, this might be possible through the same shell object, but I haven't explored that option. It's likely dependent on the specific format: for mp3 you probably want a library for viewing/editing mp3-specific metadata.
I'm new to Powershell scripting, and I'm struggling with how to identify when multiple files have been created in order to initiate a process.
Ultimately, I need to wait until a handful of files have been created (this usually occurs within a finite period of time the same time each day). These files are created at separate times. Once all files have been created and are at their final location, I need to perform a separate action with these files. The trouble I'm having is:
Identifying when all files are available
Identifying how to initiate a separate process once these files are available
If necessary,unregistering the events (I plan to run this script each morning...I don't know how long these events are registered)
I've toyed with using the IO.FileSystemWatcher with some success in order to monitor when any individual directory has this file. While this logs appropriately, I don't know how to consolidate the collection of these files. Possibly a flag? Not sure how to implement. I've also considered using Test-Path as a way of checking to see if these files exist -- but a drawback with this is that I'd need to periodically run the script (or Sleep) for a pre-defined period of time.
Does anyone have experience with doing something like this? Or possibly provide guidance?
What I've tried (with respect to IO.FileSystemWatcher) using test data:
$i=0
$paths = "C:\","Z:\"
$fileName='Test'+$Datestr.Trim()+'.txt'
foreach ($path in $paths)
{
$fsw = New-Object IO.FileSystemWatcher $path, $fileName -Property #{IncludeSubdirectories = $tru;NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'}
Register-ObjectEvent $fsw Created -SourceIdentifier "$i+fileCreated" -Action {
$name = $Event.SourceEventArgs.Name
$changeType = $Event.SourceEventArgs.ChangeType
$fpath = $Event.SourceEventArgs.FullPath
$timeStamp = $Event.TimeGenerated
Write-Host "The folder "$Event.SourceEventArgs.FullPath" was $changeType at $timeStamp" -fore green
Out-File -FilePath Z:\log.txt -Append -InputObject "The folder $fpath was $changeType at $timeStamp"
}
I am guessing you do not have control over the process(es) that create the files, or else you would use the completion of that job to trigger the "post processing" you need to do. I have used IO.FileSystemWatcher on a loop/timer like you described, and then I Group-Object on the file names to get a distinct list of the files that have changed. I was using this to monitor for small files (about 100 files at ~100KB each) that did not change frequently, and it still generated a large number of events every time the files changed. If you want to take action/start a process every time a change is made, then IO.FileSystemWatcher is your friend.
If the files are larger/take longer to generate, and because you only care once they are all done (not when they are created/modified) you may be better off skipping the filewatcher and just check if all of the files are there. IE: the process(s) to generate the files usually finishes by 6am. So at 6am you run a script that checks to see if all the files exist. (and I would also check the file size/last modified dttm to help you ensure that the process which generates the file is done writing to it.) You still may want to build a loop into this, especially if you want the process to start as soon as the files are done.
$filesIWatch = #()
$filesIWatch += New-Object -TypeName psobject -Property #{"FullName"="C:\TestFile1.txt"
"AvgSize"=100}
$filesIWatch += New-Object -TypeName psobject -Property #{"FullName"="C:\TestFile2.txt"
"AvgSize"=100}
[bool]$filesDone = $true
foreach($file in $filesIWatch){
#Check if the file is there. Also, maybe check the file size/modified dttm to help be sure that some other process is still writing to the file.
if(-not (Test-Path ($file.FullName))){
$filesDone = $false
}
}
if($filesDone){
#do your processing
}
else{
#throw error/handle them not being done
}
I have a folder which contains thousands of PDF files. I need to filter through these files based on file name (which will group these into 2 or more PDF's) and then merge these 2 more more PDF's into 1 PDF.
I'm OK with group the files but not sure the best way of then merging these into 1 PDF. I have researched iTextSharp but have been unable to get this to work in PowerShell.
Is iTextSharp the best way of doing this? Any help with the code for this would be much appreciated.
Many thanks
Paul
Have seen a few of these PowerShell-tagged questions that are also tagged with itextsharp, and always wondered why answers are given in .NET, which can be very confusing unless the person asking the question is proficient in PowerShell to begin with. Anyway, here's a simple working PowerShell script to get you started:
$workingDirectory = Split-Path -Parent $MyInvocation.MyCommand.Path;
$pdfs = ls $workingDirectory -recurse | where {-not $_.PSIsContainer -and $_.Extension -imatch "^\.pdf$"};
[void] [System.Reflection.Assembly]::LoadFrom(
[System.IO.Path]::Combine($workingDirectory, 'itextsharp.dll')
);
$output = [System.IO.Path]::Combine($workingDirectory, 'output.pdf');
$fileStream = New-Object System.IO.FileStream($output, [System.IO.FileMode]::OpenOrCreate);
$document = New-Object iTextSharp.text.Document;
$pdfCopy = New-Object iTextSharp.text.pdf.PdfCopy($document, $fileStream);
$document.Open();
foreach ($pdf in $pdfs) {
$reader = New-Object iTextSharp.text.pdf.PdfReader($pdf.FullName);
$pdfCopy.AddDocument($reader);
$reader.Dispose();
}
$pdfCopy.Dispose();
$document.Dispose();
$fileStream.Dispose();
To test:
Create an empty directory.
Copy code above into a Powershell script file in the directory.
Copy the itextsharp.dll to the directory.
Put some PDF files in the directory.
Not sure how you intend to group filter the PDFs based on file name, or if that's your intention (couldn't tell if you meant just pick out PDFs by extension), but that shouldn't be too hard to add.
Good luck. :)
I'm trying to create a script to delete all files in a folder and it's subfolders that are older than 45 days - I know how to do this, the problem is the parent folder in question has several links to other folders within itself - how do I prevent the script from deleting the links (as a link is a file), but instead treats the links like folders and to look "inside" the links for files older than 45 days.
If that's not possible, then is it possible to create a dynamic variable or array so that the script looks inside each folder I need it to and delete any files older than 45 days? If so, how do I do that.
Currently my only other option would be to create a separate script for each folder (or create code for each script in one file) and either call them individually or use yet another script to call each script.
For reference, this is in a Windows Server 2008 R2 environment
I can't work out a full solution right now. If I get time I'll come back and edit with one. Essentially I would create a function that would call itself recursively for folders anf for links where the .TargetPath was a folder. The creation of the recursive function is pretty standard fair. The only slightly opaque part is getting the .TargetPath of a .lnk file:
$sh = New-Object -COM WScript.Shell
$sc = $sh.CreateShortcut('E:\SandBox\ScriptRepository.lnk')
$targetPath = $sc.TargetPath
That is the PS way. The VBScript version is pretty much the same with a different variable naming convention and a different method for COM object instantiation.
So here is a more complete solution. I have not set up test folders and files to test it completely, but it should be pretty much what you need:
function Remove-OldFile{
param(
$Folder
)
$sh = New-Object -COM WScript.Shell
foreach($item in Get-ChildItem $Folder){
if ($item.PSIsContainer){
Remove-OldFile $item.FullName
}elseif($item.Extension -eq '.lnk'){
Remove-OldFile $sh.CreateShortcut($item.FullName).TargetPath
}else{
if(((Get-Date) - $item.CreationTime).Days -gt 45){
$item.Delete()
}
}
}
}
Remove-OldFile C:\Scripts
Just for completeness, here is an untested off the cuff VBS solution. I warn you that it may have some syntax errors, but the logic should be fine.
RemoveOldFiles "C:\Scripts"
Sub RemoveOldFiles(strFolderPath)
Dim oWSH : Set oWSh = CreateObject("WScript.Shell")
Dim oFSO : Set oFSO = CreateObject("Scripting.FileSystemObject")
For Each oFolder in oFSO.GetFolder(strFolderPath).SubFolders
RemoveOldFiles oFolder.Path
Next
For Each oFile in oFSO.GetFolder(strFolderPath).Files
if LCase(oFSO.GetExtensionName(oFile.Name)) = "lnk" Then
RemoveOldFiles oWSH.CreateShortcut(oFile.Path).TargetPath
Else
If DateDiff("d", oFile.DateCreated, Date) > 45 Then
oFSO.DeleteFile(oFile)
End If
End If
Next
End Sub
Very high level answer:
Loop through all files in current folder.
If `file.name` ends with `.lnk` (we have a link/shortcut).
Get the path of the shortcut with `.TargetPath`
You can now pass .TargetPath the same way you would pass the name of a subdirectory when you find one to continue recursing through the directory tree.