Unable to delete or rename directories or Files with Multiple Square ([[...]] brackets in Path [duplicate] - powershell

This question already has answers here:
How can I make PowerShell handle [ or ] in file name well?
(2 answers)
Error "Could not find a part of the path" while setting attributes on an existing file
(2 answers)
Closed 4 months ago.
This post was edited and submitted for review 4 months ago and failed to reopen the post:
Original close reason(s) were not resolved
I have regular Jobs to clean up old User Windows-Profile directories on a central NAS.
Last time I had Directories containing double square Brackets in the path (Import from Macromedia or so). This looks like:
\server.ad.local\HOME1\Username\System\Anwendungsdaten\Macromedia\FlashPlayer\SharedObjects\G5EUZBX2\www.daserste.de\[[IMPORT]]\players.edgesuite.net\flash\plugins\osmf\advanced-streaming-plugin\v3.4\osmf2.0\AkamaiAdvancedStreamingPlugin.swf\HDCore.sol
As this Job should run automatically with Powershell I tired various things:
Tried to replace the brackets an rename the folder - no success
Tried LiteralPath for Remove-Item - no success
Tried to Deleted with Scripting.FileSystemObject - no success
I always get the following error Message:
The Element ... cannot be removed: A part of the Path "HDCore.sol" cannot be found.
Are there any ideas?
Tried to rename the Folder, tried remove-item with -LiteralPath, tried to use FileSystemObject.
All of the actions gave the same result: Error
Just to complete: here are the functions I used last:
Function RemoveChar
{
Param ($Path)
$Pattern = '#|&|%|\[{1,}|\]{1,}|\^|\s|\.{2,}'
if ($Path -imatch $Pattern){
Rename-Item -Path $Path -NewName ($Path -replace $Pattern,'') -ErrorAction SilentlyContinue
return ($Path -replace $Pattern,'')
} else {
return $Path
}
}
Function Truncate
{
Param ($Path)
$total_path_length_threshold = 248 # maximal erlaubte Zeichenzahl (248 für Verzeichnisse, 260 für Dateien)
$characters_to_truncate = 60 # Anzahl der zeichen, um die der name zu kürzen ist. Ein eindeutiger Index wird angehangen
$virtual_drive = "v:" # Für temp. Prozessing muss das Format "v:" haben
$collection = cmd /c dir $Path /s /b |? {$_.length -gt $total_path_length_threshold }
$count_paths = ($collection | measure).count - 1
foreach ($pathlong in $collection) {
$parent_path = Split-path -path $pathlong
subst $virtual_drive $parent_path
$leaf = split-path -leaf $pathlong
$short_virtual_path = join-path $virtual_drive $leaf
$item = Get-Item -LiteralPath $short_virtual_path
if (Test-Path -LiteralPath $item) {
$filename = $item.name
$filename_extension = $item.Extension
$basename = $item.BaseName
$basename_length = $basename.length
$new_index = "X" + $counter + "X"
$adjusted_characters_to_truncate = $characters_to_truncate + $new_index.length
if ( $basename_length -gt $adjusted_characters_to_truncate ) {
$length_to_use = $basename_length - $adjusted_characters_to_truncate
$new_filename = $basename.substring(0, $length_to_use ) + $new_index + $filename_extension
$new_path = $parent_path + $new_filename
$Path = $new_path
Rename-Item -LiteralPath $short_virtual_path -NewName $new_filename
}
}
subst v: /d
}
return $Path
}
Function removeRecursive
{
param([String] $Path)
$fso = new-object -com "Scripting.FileSystemObject"
function proc {
param($folder)
$folder.Files | foreach-object {
RemoveChar $_
}
$folder.Files | foreach-object {
$short = $fso.GetFile($_.FullName).ShortPath
LogWrite "$FullDate : Processing: $short"
$fso.DeleteFile($short,$true)
}
$folder.SubFolders | foreach-object {
proc $_
}
}
proc $fso.GetFolder($Path)
}
The function I call from main code is removeRecursive. And yes I tried Remove-Item -LiteralPath SomePath -Recursive -Force too but no success as well.

Related

Variable for "Media created"? (Powershell)

What is the variable for "Media created", using Powershell?
I'm trying to change many .mov files at one time, in a folder, from "IMG-3523" to their respective creation dates.
I was using the following code to do so, using LastWriteTime:
Get-ChildItem | Rename-Item -NewName {$_.LastWriteTime.ToString("yyyy-MM-dd hh.mm.ss ddd") + ($_.Extension)}
However for some reason there are many duplicate entries for that variable, and I run into an error stating so.
Is there any way I can edit this code to use the "Media created" variable? It's the only date without duplicates.
(I've tried editing the code to include microseconds and $_.CreationTime, however there are many duplicates with that variable too (actually all have the same CreationTime - the timestamp of when I copied the files over from an external disk).
For all files within a directory:
$fldPath = "D:\FolderName";
$flExt = ".mov";
$attrName = "media created"
(Get-ChildItem -Path "$fldPath\*" -Include "*$flExt").FullName | % {
$path = $_
$shell = New-Object -COMObject Shell.Application;
$folder = Split-Path $path;
$file = Split-Path $path -Leaf;
$shellfolder = $shell.Namespace($folder);
$shellfile = $shellfolder.ParseName($file);
$a = 0..500 | % { Process { $x = '{0} = {1}' -f $_, $shellfolder.GetDetailsOf($null, $_); If ( $x.split("=")[1].Trim() ) { $x } } };
[int]$num = $a | % { Process { If ($_ -like "*$attrName*") { $_.Split("=")[0].trim() } } };
$mCreated = $shellfolder.GetDetailsOf($shellfile, $num);
$mCreated;
};
For one specific File, you can use:
$flPath = "D:\FolderName\FileName.mov";
$attrName = "media created"
$path = $flPath;
$shell = New-Object -COMObject Shell.Application;
$folder = Split-Path $path;
$file = Split-Path $path -Leaf;
$shellfolder = $shell.Namespace($folder);
$shellfile = $shellfolder.ParseName($file);
$a = 0..500 | % { Process { $x = '{0} = {1}' -f $_, $shellfolder.GetDetailsOf($null, $_); If ( $x.split("=")[1].Trim() ) { $x } } };
[int]$num = $a | % { Process { If ($_ -like "*$attrName*") { $_.Split("=")[0].trim() } } };
$mCreated = $shellfolder.GetDetailsOf($shellfile, $num);
$mCreated;
This was answered in SuperUser by vomit-it-chunky-mess-style on Getting a Media Created via PS
Using the Shell.Application object you can get this value from the file, but unfortunately, the property names are all Localized, so if you are looking for a property named Media created on a non-english machine, you will not find it..
For .MOV files, the index of this property in numbered 4.
You can use below function to get that value for all the files in the path and then use that to rename them:
function Get-MetaData {
[CmdletBinding()]
[OutputType([Psobject[]])]
Param (
# Path can be the path to a folder or the full path and filename of a single file
[Parameter(Mandatory = $true, ValueFromPipeline = $true, Position = 0)]
[string]$Path,
# Filter is unused if Path is pointing to a single file
[Alias('Pattern')]
[string]$Filter = '*.*',
[Alias('Indices')]
[int[]]$Properties = 1..500,
# Recurse is unused if Path is pointing to a single file
[switch]$Recurse,
[switch]$IncludeEmptyProperties
)
$item = Get-Item -Path $Path -ErrorAction SilentlyContinue
if (!$item) { Write-Error "$Path could not be found."; return }
if (!$item.PSIsContainer) {
# it's a file
$files = #($item)
$Path = $item.DirectoryName
}
else {
# it's a folder
$files = Get-ChildItem -Path $Path -Filter $Filter -File -Recurse:$Recurse
}
$shell = New-Object -ComObject "Shell.Application"
$objDir = $shell.NameSpace($Path)
foreach($file in $files) {
$objFile = $objDir.ParseName($file.Name)
$mediaFile = $objDir.Items()
foreach($index in $Properties) {
$name = $objDir.GetDetailsOf($mediaFile, $index)
if (![string]::IsNullOrWhiteSpace($name)) {
# to be on the safe side, remove any control character (ASCII < 32) that may be in there
$value = $objDir.GetDetailsOf($objFile, $index) -replace '[\x00-\x1F]+'
if (![string]::IsNullOrWhiteSpace($value) -or $IncludeEmptyProperties) {
[PsCustomObject]#{
Path = $file.FullName
Index = $index
Property = $name
Value = $value
}
}
}
}
}
# clean-up Com objects
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($objFile)
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($objDir)
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($shell)
[System.GC]::Collect()
[System.GC]::WaitForPendingFinalizers()
}
# property index 3 --> Last Modified
# property index 4 --> Date Created # we'll use this one
# property index 5 --> Last Accessed
$SourceFolder = 'X:\Somewhere'
Get-MetaData -Path $SourceFolder -Filter '*.mov' -Properties 4 |
ForEach-Object {
# HH for 24-hour clock; hh for 12-hour clock
$dateCreated = '{0:yyyy-MM-dd HH.mm.ss ddd}' -f ([datetime]$_.Value)
$directory = [System.IO.Path]::GetDirectoryName($_.Path)
$extension = [System.IO.Path]::GetExtension($_.Path)
# test if the suggested new name is already in use, append a sequence
# number to the basename until we have a unique name
$newName = '{0}{1}' -f $dateCreated, $extension
$index = 1
while (Test-Path -Path (Join-Path -Path $directory -ChildPath $newName) -PathType Leaf) {
$newName = '{0}({1}){2}' -f $dateCreated, $index++, $extension
}
Rename-Item -Path $_.Path -NewName $newName -WhatIf
}
The -WhatIf is a safety measure. It will make the code only show in the console what would happen. No file is actually renamed.
If you are convinced that output is correct, then remove the -WhatIf switch and run the code again
As "Media created" property of a .mov file might also be multiple, or undefined in many cases, use it may cause also file name duplication, even error.
I'd propose another approach, we first check existence of both "Media created" and "Date taken", if nonexistent, we keep using $_.LastWriteTime as basic file name, adding a number to each file name if duplicated, like this:
#
# fnc001: get file media created, date taken or LastWriteTime:
#
function entGetMediaCreatedOrLastWriteTime($objFile) {
$idxMediaCreated = 208
$objShell = New-Object -COMObject Shell.Application
$objShellFolder = $objShell.NameSpace($objFile.DirectoryName)
$iState = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($objShell)
$objShellFile = $objShellFolder.ParseName($objFile.Name)
$mediaCreated = $objShellFolder.GetDetailsOf($objShellFile, $idxMediaCreated)
#
# if media created is empty, we check if we have Date taken:
#
if($mediaCreated -eq "") {
#
# canon cameras set Date taken for photos:
#
$idxDateTaken = 12
$dateTaken = $objShellFolder.GetDetailsOf($objShellFile, $idxDateTaken)
#
# return LastWriteTime if neither media created, nor Date taken:
#
if($dateTaken -eq "") {
return $objFile.LastWriteTime
}
#
# otherwise return Date taken, removing non-ascii before:
#
else
{
return [DateTime]($dateTaken -replace '\P{IsBasicLatin}')
}
}
#
# otherwise return valid media created, removing non-ascii before:
#
else {
return [DateTime]($mediaCreated -replace '\P{IsBasicLatin}')
}
}
#
# fnc001: increment filename if it already exists:
#
function entIncrementIfExistent($filename) {
$fnew = $filename
$ext = Split-Path $filename -Extension
#
# define prefix before file number:
#
$prefix = "-"
$i = 0
#
# save file base length:
#
$lngbase = $fnew.length - $ext.length
#
# here $filename is like
# 2023-02-05 08.33.00 Sun.mov,
#
# if it exists, we try to use first available of:
#
# 2023-02-05 08.33.00 Sun-1.mov
# 2023-02-05 08.33.00 Sun-2.mov
# ...
# 2023-02-05 08.33.00 Sun-9.mov
# ...
# 2023-02-05 08.33.00 Sun-10.mov
# 2023-02-05 08.33.00 Sun-11.mov
# ...
#
while (Test-Path $fnew)
{
$i++
$fnew = $fnew.Substring(0, $lngbase) + $prefix + $i + $ext
}
return $fnew
}
Get-ChildItem *.mov | Rename-Item -NewName {
$xDateTime = entGetMediaCreatedOrLastWriteTime $_
$fnew = $xDateTime.ToString("yyyy-MM-dd hh.mm.ss ddd") + $_.Extension
entIncrementIfExistent $fnew
}
I've tried this code, it gets something like this:

Bulk File Renaming - Extract parent foldername and use in file renaming

PowerShell novice here again with my proof of concept.
The code below successfully extracts attached files from .msg files located in folders and leaves the extracted filename without changing it. What I'm now looking for now is to extract part of the parent folder name, with standard format of...
nnnn+string (e.g. "8322 MyStudy") i.e. 4 digits followed by a space then string.
...to rename the extracted filename from...
ExtractedFilename.pdf to "0nnnn - ExtractedFilename.pdf". e.g. "08322 - ExtractedFilename.pdf"
My main problem is how to extract the numeric part of the parent folder name (from where my module will be run). I'm hoping that my poor PS formatting skills will allow me to do the rest.
Once again, any help appreciated.
##
## Source: https://chris.dziemborowicz.com/blog/2013/05/18/how-to-batch-extract-attachments-from-msg-files-using-powershell/
##
## Usage: Expand-MsgAttachment *
##
##
function Expand-MsgAttachment
{
[CmdletBinding()]
Param
(
[Parameter(ParameterSetName="Path", Position=0, Mandatory=$True)]
[String]$Path,
[Parameter(ParameterSetName="LiteralPath", Mandatory=$True)]
[String]$LiteralPath,
[Parameter(ParameterSetName="FileInfo", Mandatory=$True, ValueFromPipeline=$True)]
[System.IO.FileInfo]$Item
)
Begin
{
# Load application
Write-Verbose "Loading Microsoft Outlook..."
$outlook = New-Object -ComObject Outlook.Application
}
Process
{
switch ($PSCmdlet.ParameterSetName)
{
"Path" { $files = Get-ChildItem -Path $Path }
"LiteralPath" { $files = Get-ChildItem -LiteralPath $LiteralPath }
"FileInfo" { $files = $Item }
}
$files | % {
# Work out file names
$msgFn = $_.FullName
# extract path, e.g. 'c:\path\to\'
$msgPath = Split-Path -Path $msgFn
# Skip non-.msg files
if ($msgFn -notlike "*.msg") {
Write-Verbose "Skipping $_ (not an .msg file)..."
return
}
# Extract message body
Write-Verbose "Extracting attachments from $_..."
$msg = $outlook.CreateItemFromTemplate($msgFn)
$msg.Attachments | % {
# Work out attachment file name
#$attFn = $msgFn -replace '\.msg$', " - Attachment - $($_.FileName)"
$attFn = Join-Path -Path $msgPath -ChildPath ($_.FileName)
# Do not try to overwrite existing files
if (Test-Path -literalPath $attFn) {
Write-Verbose "Skipping $($_.FileName) (file already exists)..."
return
}
# Save attachment
Write-Verbose "Saving $($_.FileName)..."
$_.SaveAsFile($attFn)
# Output to pipeline
Get-ChildItem -LiteralPath $attFn
}
}
}
# This function to rename expanded attachment file to study renaming standards
Function RenameExpandedAttachments {
}
End
{
Write-Verbose "Done."
}
}
The currently running script is :
$script:MyInvocation.MyCommand.Path
Use Split-Path to get only the Path,
Split-Path $script:MyInvocation.MyCommand.Path
to get only the last element use again Split-Path with the -Leaf parameter
Split-Path -Leaf (Split-Path $script:MyInvocation.MyCommand.Path)
To extract leading numbers use a Regular Expression with a (capture group).
'^(\d+) (.*)$'
And wrap all this in an if:
If ((Split-Path -Leaf (Split-Path $script:MyInvocation.MyCommand.Path)) -match '^(\d+) (.*)$'){
$NewName = "{0:00000} - {1}" -f $Matches[1],$ExtractedFileName
} else {
"No numbers found in this path"
}

Powershell incorrect parameter

So I am trying to pass the file directory to the function convert at the bottom. When I run the script I receive the output:
Succes
C:\test 2\00000027627-00001\PROCESSING CHECKLIST
convert : Invalid Parameter - 2\00000027627-00001\PROCESSING
At C:\Users\pmanca\Desktop\msbxmlreader.ps1:35 char:13
+ convert([string]$hostdirectoryPlusLoanPlusDocType)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (Invalid Paramet...0001\PROCESSING:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
It appears to be cutting off the file path after C:\test. Could the file path be too long? even so i wouldn't imagine getting the error at the function call and instead somewhere in the function when it couldn't resolve the path.
#? is a shortcut for where object and % is a shortcut for foreach-object
cls
#count 1 = Loan Number
#count 4 = Cust Num
#count 5 = Document Type
$hostdirectory = "C:\test 2"
$count = 0
$file = (Select-xml -Path "$hostdirectory\index.xml" -XPath / ).Node
$test = $file.ExportedResult.Docs.ExportedDoc.Doc.UdiValues.UdiValue.Value# | Select-Object {$_.UdiValue.Value.InnerXML} -Unique #|? {$_.UdiValue.Name -eq "Loan Number"}
$test | ForEach-Object{
$count++
# Write-Host $_.innerxml "----" $count
if($count -eq 1){
[string]$xmlHold = $_.InnerXML
$hostdirectoryPlusLoan = "$hostdirectory\$xmlHold"
if(!(test-path $hostdirectoryPlusLoan)){
New-Item $hostdirectoryPlusLoan -ItemType directory
}
}
if($count -eq 5){
[string]$xmlHold = $_.InnerXML
$hostdirectoryPlusLoanPlusDocType = "$hostdirectoryPlusLoan\$xmlHold"
if(!(test-path $hostdirectoryPlusLoanPlusDocType)){
New-Item $hostdirectoryPlusLoanPlusDocType -ItemType directory
}
if(Test-Path "$hostdirectory\$xmlhold.pdf"){
$check = Copy-Item "$hostdirectory\$xmlHold.pdf" -Destination $hostdirectoryPlusLoanPlusDocType -ErrorAction SilentlyContinue
if(-not $?) {write-warning "Copy Failed"; Write-Host $Error[0].exception.message}
else {write-host "Succes"}
Write-Host $hostdirectoryPlusLoanPlusDocType
convert([string]$hostdirectoryPlusLoanPlusDocType)
}
}
if($count -ge 8){
$count = 0
# Write-Host "-----------------------------------"
}
}
function convert([string]$inputDirectory){
write-host $inputDirectory
#Variable to hold current input directory
$InputPathFilter = $InputDirectory + '\*.pdf'
#Variable to hold the list of PDFs from the current input directory
$PDFList = #(gci $InputPathFilter | foreach {write-output $_.name})
#Loop through list of PDF files to convert to TIF image files.
for ($j=0; $j -lt $PDFList.count; $j++) {
#Each PDF will go into its own directory/batch
#Create a variable with only the file name
$FileName = $PDFList[$j] -replace(".pdf",'')
#Variable of the full path to the current PDF
$InputPath = $InputDirectory + '\' + $PDFList[$j]
#Variable to hold output path of each TIF file. Filename format is that used by batches in ScerIS (i.e. 00010001.tif, 00010002.tif, etc...)
$OutputFullDirectory = $inputlDirectory + '\' + $FileName + "_" + "{0:D4}" -f + '1' + '%04d.tif'
#Calls Ghostscript command line executable to process each PDF file into a group of single page TIF image files
&'C:\Program Files\gs\gs9.14\bin\gswin64c.exe' -sDEVICE=tiffg4 -dBATCH -dNOPAUSE -q -r600 "-sOutputFile=$OutputFullDirectory" "$InputPath"
#Increment the counter for the loop
$DocCounter = $j + 1
#Rename the current pdf so that it isn't processed again.
$RenamePath = $PdfList[$j] -replace("pdf", "pd_")
Rename-Item $InputPath $RenamePath -Force
}
}
First : In PowerShell, when you call a function you must not use parenthesis :
convert $hostdirectoryPlusLoanPlusDocType
or as suggested in comment
convert -inputDirectory $hostdirectoryPlusLoanPlusDocType
but not :
convert([string]$hostdirectoryPlusLoanPlusDocType)
Second : Your function should be declarated first and use after :
function toto ($var)
{
Write-Host $var
}
toto "voila"
and not
toto "voila"
function toto ($var)
{
Write-Host $var
}

Comparing filehash and outputting files

I am new to PowerShell and am writing a script to get the hash of a directory and store it in a .txt file.
I then want to compare it to an earlier version and check for changes. If there are changes, I want a new .txt or .html file containing which line items have changed, with last modified dates.
So far, I've gotten the comparison to work, and the resulting steps based upon the pass/fail work fine.
What I need help with is outputting the results into a .txt file that lists only the files that have changed, with fields of Algorithm, Hash, Filename, Last edit time. I know I can use
(Get-Item $source).LastWriteTime
To fetch the write time, but I need to do it for every file in the directory, not just the .txt file that contains the hash.
# Variables
$Hashstore = "d:\baseline.txt"
$HashCompare = "d:\hashcompare.txt"
$HashTemp = "d:\hashtemp.txt"
$FileDir = "d:\New2"
$DateTime = Get-Date -format M.d.yyyy.hh.mm.ss
# Email Variables
$smtp_server = '<yourSMTPServer>'
$to_email = '<email>'
$from_email = '<email>'
$dns_server = "<yourExternalDNSServer>"
$domain = "<yourDomain>"
# Check if Baseline.txt Exists
If (Test-Path $Hashstore)
# // File exists
{}
Else {
# // File does not exist - Should never happen!
$RefreshHash = dir $FileDir | Get-FileHash -Algorithm MD5
$RefreshHash | Out-File $Hashstore
}
# Generate new Compare Hash.txt
$HashNew = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$HashNew | Out-File $HashCompare
# Get Hash of baseline.txt
$HashBaseline = Get-FileHash -Path d:\baseline.txt -Algorithm MD5
#Get Hash of hashcompare.txt
$HashDiff = Get-FileHash -Path d:\hashcompare.txt -Algorithm MD5
#If changed, output hash to storage, and flag changes
If ($HashBaseline.hash -eq $HashDiff.hash)
{
Add-Content -Path d:\success.$DateTime.txt -Value " Source Files ARE EQUAL </p>"
}
else
{
Add-Content -Path d:\failure.$DateTime.html -Value "Source Files NOT EQUAL </p>"
$HashNew | Out-File $HashTemp
}
# Compare two logs, send email if there is a change
If ($diff_results)
{
#$evt_message = Get-Content .\domain.new.txt | Out-String
#Write-EventLog -LogName Application -EventId 9000 -EntryType Error -Source "Maximo Validation Script" -Message $evt_message
#Send-MailMessage -To $to_email -From $from_email -SmtpServer $smtp_server -Attachments .\domain.new.txt -Subject "ALERT! Change in Records" -Body "A change has been detected in the Maximo system files.`n`n`tACTION REQUIRED!`n`nVerify that this change was authorized."
}
If ($HashNew.HashString -eq $Hashstore.HashString)
{
}
else
{
$HashTemp | Out-File $HashStore
}
I know the add-item may not be the best way to write to this log I'm creating. What would be the best way to add the last write time to every file that is read?
Here is a clean way to ouput the information you need (Algorithm, Hash, Filename, Last edit time) for each file that has changed :
$Hashstore = "d:\baseline.txt"
$HashCompare = "d:\hashcompare.txt"
$HashTemp = "d:\hashtemp.txt"
$FileDir = "d:\New2"
$DateTime = Get-Date -format M.d.yyyy.hh.mm.ss
# Check if Baseline.txt Exists
If (Test-Path $Hashstore)
# // File exists
{
}
Else {
# // File does not exist - Should never happen!
$RefreshHash = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$RefreshHash | Export-Csv -Path $Hashstore -NoTypeInformation -Force
}
# Generate new Compare Hash.txt
$HashNew = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$HashNew | Export-Csv -Path $HashCompare -NoTypeInformation -Force
# Get Hash of baseline.txt
$HashBaseline = Get-FileHash -Path $Hashstore -Algorithm MD5
#Get Hash of hashcompare.txt
$HashDiff = Get-FileHash -Path $HashCompare -Algorithm MD5
#If changed, output hash to storage, and flag changes
If ($HashBaseline.hash -eq $HashDiff.hash) {
Add-Content -Path D:\success.$DateTime.txt -Value " Source Files ARE EQUAL </p>"
}
Else {
Add-Content -Path D:\failure.$DateTime.txt -Value "Source Files NOT EQUAL </p>"
$HashNew | Export-Csv -Path $HashTemp -NoTypeInformation -Force
# Storing a collection of differences in $Diffs
$Diffs = Compare-Object -ReferenceObject (Import-Csv $Hashstore) -DifferenceObject (Import-Csv $HashCompare)
Foreach ($Diff in $Diffs) {
$DiffHashInfo = $Diff | Select-Object -ExpandProperty InputObject
$DiffFileInfo = Get-ChildItem -Path $DiffHashInfo.Path
# Creating a list of properties for the information you need
$DiffObjProperties = [ordered]#{'Algorithm'=$DiffHashInfo.Algorithm
'Hash'=$DiffHashInfo.Hash
'Filename'=$DiffFileInfo.Name
'Last edit time'=$DiffFileInfo.LastWriteTime
}
# Building a custom object from the list of properties in $DiffObjProperties
$DiffObj = New-Object -TypeName psobject -Property $DiffObjProperties
$DiffObj
}
}
Before creating the files $Hashstore and $HashCompare, I convert the information they contain to CSV format, rather than plain text.
It makes their content much easier to manipulate later , using Import-CSV.
This makes proper objects with properties I can use.
This also makes them easier to compare, and the result of this comparison ($Diffs) is a collection of these proper objects.
So $Diffs contains all the files that have changed and I loop through each of them in a Foreach statement.
This allows you to create a custom object ($DiffObj) with exactly the information you need ($DiffObjProperties) for each of the file that have changed.
PowerShell v3+ Recursive Directory Diff Using MD5 Hashing
I use this pure PowerShell (no dependencies) recursive file content diff. It calculates in-memory the MD5 hash (the algorithm is configurable) for each directories file contents and gives results in standard PowerShell Compare-Object format.
It can optionally export to CSV files along with a summary text file. It can either drop the rdiff.ps1 file into your path or copy the contents into your script.
USAGE: rdiff path/to/left,path/to/right [-s path/to/summary/dir]
Here is the gist. I copied below for reference but I recommend using the gist version as I will be adding new features to it over time.
#########################################################################
### USAGE: rdiff path/to/left,path/to/right [-s path/to/summary/dir] ###
### ADD LOCATION OF THIS SCRIPT TO PATH ###
#########################################################################
[CmdletBinding()]
param (
[parameter(HelpMessage="Stores the execution working directory.")]
[string]$ExecutionDirectory=$PWD,
[parameter(Position=0,HelpMessage="Compare two directories recursively for differences.")]
[alias("c")]
[string[]]$Compare,
[parameter(HelpMessage="Export a summary to path.")]
[alias("s")]
[string]$ExportSummary
)
### FUNCTION DEFINITIONS ###
# SETS WORKING DIRECTORY FOR .NET #
function SetWorkDir($PathName, $TestPath) {
$AbsPath = NormalizePath $PathName $TestPath
Set-Location $AbsPath
[System.IO.Directory]::SetCurrentDirectory($AbsPath)
}
# RESTORES THE EXECUTION WORKING DIRECTORY AND EXITS #
function SafeExit() {
SetWorkDir /path/to/execution/directory $ExecutionDirectory
Exit
}
function Print {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Message to print.")]
[string]$Message,
[parameter(HelpMessage="Specifies a success.")]
[alias("s")]
[switch]$SuccessFlag,
[parameter(HelpMessage="Specifies a warning.")]
[alias("w")]
[switch]$WarningFlag,
[parameter(HelpMessage="Specifies an error.")]
[alias("e")]
[switch]$ErrorFlag,
[parameter(HelpMessage="Specifies a fatal error.")]
[alias("f")]
[switch]$FatalFlag,
[parameter(HelpMessage="Specifies a info message.")]
[alias("i")]
[switch]$InfoFlag = !$SuccessFlag -and !$WarningFlag -and !$ErrorFlag -and !$FatalFlag,
[parameter(HelpMessage="Specifies blank lines to print before.")]
[alias("b")]
[int]$LinesBefore=0,
[parameter(HelpMessage="Specifies blank lines to print after.")]
[alias("a")]
[int]$LinesAfter=0,
[parameter(HelpMessage="Specifies if program should exit.")]
[alias("x")]
[switch]$ExitAfter
)
PROCESS {
if($LinesBefore -ne 0) {
foreach($i in 0..$LinesBefore) { Write-Host "" }
}
if($InfoFlag) { Write-Host "$Message" }
if($SuccessFlag) { Write-Host "$Message" -ForegroundColor "Green" }
if($WarningFlag) { Write-Host "$Message" -ForegroundColor "Orange" }
if($ErrorFlag) { Write-Host "$Message" -ForegroundColor "Red" }
if($FatalFlag) { Write-Host "$Message" -ForegroundColor "Red" -BackgroundColor "Black" }
if($LinesAfter -ne 0) {
foreach($i in 0..$LinesAfter) { Write-Host "" }
}
if($ExitAfter) { SafeExit }
}
}
# VALIDATES STRING MIGHT BE A PATH #
function ValidatePath($PathName, $TestPath) {
If([string]::IsNullOrWhiteSpace($TestPath)) {
Print -x -f "$PathName is not a path"
}
}
# NORMALIZES RELATIVE OR ABSOLUTE PATH TO ABSOLUTE PATH #
function NormalizePath($PathName, $TestPath) {
ValidatePath "$PathName" "$TestPath"
$TestPath = [System.IO.Path]::Combine((pwd).Path, $TestPath)
$NormalizedPath = [System.IO.Path]::GetFullPath($TestPath)
return $NormalizedPath
}
# VALIDATES STRING MIGHT BE A PATH AND RETURNS ABSOLUTE PATH #
function ResolvePath($PathName, $TestPath) {
ValidatePath "$PathName" "$TestPath"
$ResolvedPath = NormalizePath $PathName $TestPath
return $ResolvedPath
}
# VALIDATES STRING RESOLVES TO A PATH AND RETURNS ABSOLUTE PATH #
function RequirePath($PathName, $TestPath, $PathType) {
ValidatePath $PathName $TestPath
If(!(Test-Path $TestPath -PathType $PathType)) {
Print -x -f "$PathName ($TestPath) does not exist as a $PathType"
}
$ResolvedPath = Resolve-Path $TestPath
return $ResolvedPath
}
# Like mkdir -p -> creates a directory recursively if it doesn't exist #
function MakeDirP {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path create.")]
[string]$Path
)
PROCESS {
New-Item -path $Path -itemtype Directory -force | Out-Null
}
}
# GETS ALL FILES IN A PATH RECURSIVELY #
function GetFiles {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path to get files for.")]
[string]$Path
)
PROCESS {
ls $Path -r | where { !$_.PSIsContainer }
}
}
# GETS ALL FILES WITH CALCULATED HASH PROPERTY RELATIVE TO A ROOT DIRECTORY RECURSIVELY #
# RETURNS LIST OF #{RelativePath, Hash, FullName}
function GetFilesWithHash {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path to get directories for.")]
[string]$Path,
[parameter(HelpMessage="The hash algorithm to use.")]
[string]$Algorithm="MD5"
)
PROCESS {
$OriginalPath = $PWD
SetWorkDir path/to/diff $Path
GetFiles $Path | select #{N="RelativePath";E={$_.FullName | Resolve-Path -Relative}},
#{N="Hash";E={(Get-FileHash $_.FullName -Algorithm $Algorithm | select Hash).Hash}},
FullName
SetWorkDir path/to/original $OriginalPath
}
}
# COMPARE TWO DIRECTORIES RECURSIVELY #
# RETURNS LIST OF #{RelativePath, Hash, FullName}
function DiffDirectories {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Directory to compare left.")]
[alias("l")]
[string]$LeftPath,
[parameter(Mandatory=$TRUE,Position=1,HelpMessage="Directory to compare right.")]
[alias("r")]
[string]$RightPath
)
PROCESS {
$LeftHash = GetFilesWithHash $LeftPath
$RightHash = GetFilesWithHash $RightPath
diff -ReferenceObject $LeftHash -DifferenceObject $RightHash -Property RelativePath,Hash
}
}
### END FUNCTION DEFINITIONS ###
### PROGRAM LOGIC ###
if($Compare.length -ne 2) {
Print -x "Compare requires passing exactly 2 path parameters separated by comma, you passed $($Compare.length)." -f
}
Print "Comparing $($Compare[0]) to $($Compare[1])..." -a 1
$LeftPath = RequirePath path/to/left $Compare[0] container
$RightPath = RequirePath path/to/right $Compare[1] container
$Diff = DiffDirectories $LeftPath $RightPath
$LeftDiff = $Diff | where {$_.SideIndicator -eq "<="} | select RelativePath,Hash
$RightDiff = $Diff | where {$_.SideIndicator -eq "=>"} | select RelativePath,Hash
if($ExportSummary) {
$ExportSummary = ResolvePath path/to/summary/dir $ExportSummary
MakeDirP $ExportSummary
$SummaryPath = Join-Path $ExportSummary summary.txt
$LeftCsvPath = Join-Path $ExportSummary left.csv
$RightCsvPath = Join-Path $ExportSummary right.csv
$LeftMeasure = $LeftDiff | measure
$RightMeasure = $RightDiff | measure
"== DIFF SUMMARY ==" > $SummaryPath
"" >> $SummaryPath
"-- DIRECTORIES --" >> $SummaryPath
"`tLEFT -> $LeftPath" >> $SummaryPath
"`tRIGHT -> $RightPath" >> $SummaryPath
"" >> $SummaryPath
"-- DIFF COUNT --" >> $SummaryPath
"`tLEFT -> $($LeftMeasure.Count)" >> $SummaryPath
"`tRIGHT -> $($RightMeasure.Count)" >> $SummaryPath
"" >> $SummaryPath
$Diff | Format-Table >> $SummaryPath
$LeftDiff | Export-Csv $LeftCsvPath -f
$RightDiff | Export-Csv $RightCsvPath -f
}
$Diff
SafeExit
Another my version. But without date/time.
# Check images. Display if differ
#
$file_path = "C:\Files"
$last_state = "last_state.json"
# Check last_state.json. If false - create new empty file.
If (!(Test-Path $last_state)) {
New-Item $last_state -ItemType file | Out-Null
}
$last_state_obj = Get-Content $last_state | ConvertFrom-Json
# Get files list and hash. Also you can use -Recurse option
Get-ChildItem $file_path -Filter *.* |
Foreach-Object {
if (!$_.PSIsContainer) {
$current_state += #($_ | Get-FileHash -Algorithm MD5)
}
}
# Compare hash
ForEach ($current_file in $current_state) {
if (($last_state_obj | where {$current_file.Path -eq $_.Path}).Hash -ne $current_file.Hash) {
$changed += #($current_file)
}
}
# Display changed files
$changed
# Save new hash to last_state.json
$current_state | ConvertTo-JSON | Out-File $last_state

Get-Item fails with closed pipeline error

If I have an example function ...
function foo()
{
# get a list of files matched pattern and timestamp
$fs = Get-Item -Path "C:\Temp\*.txt"
| Where-Object {$_.lastwritetime -gt "11/01/2009"}
if ( $fs -ne $null ) # $fs may be empty, check it first
{
foreach ($o in $fs)
{
# new bak file
$fBack = "C:\Temp\test\" + $o.Name + ".bak"
# Exception here Get-Item! See following msg
# Exception thrown only Get-Item cannot find any files this time.
# If there is any matched file there, it is OK
$fs1 = Get-Item -Path $fBack
....
}
}
}
The exception message is ... The WriteObject and WriteError methods cannot be called after the pipeline has been closed. Please contact Microsoft Support Services.
Basically, I cannot use Get-Item again within the function or loop to get a list of files in a different folder.
Any explanation and what is the correct way to fix it?
By the way I am using PS 1.0.
This is just a minor variation of what has already been suggested, but it uses some techniques that make the code a bit simpler ...
function foo()
{
# Get a list of files matched pattern and timestamp
$fs = #(Get-Item C:\Temp\*.txt | Where {$_.lastwritetime -gt "11/01/2009"})
foreach ($o in $fs) {
# new bak file
$fBack = "C:\Temp\test\$($o.Name).bak"
if (!(Test-Path $fBack))
{
Copy-Item $fs.Fullname $fBack
}
$fs1 = Get-Item -Path $fBack
....
}
}
For more info on the issue with foreach and scalar null values check out this blog post.
I modified the above code slightly to create the backup file, but I am able to use the Get-Item within the loop successfully, with no exceptions being thrown. My code is:
function foo()
{
# get a list of files matched pattern and timestamp
$files = Get-Item -Path "C:\Temp\*.*" | Where-Object {$_.lastwritetime -gt "11/01/2009"}
foreach ($file in $files)
{
$fileBackup = [string]::Format("{0}{1}{2}", "C:\Temp\Test\", $file.Name , ".bak")
Copy-Item $file.FullName -destination $fileBackup
# Test that backup file exists
if (!(Test-Path $fileBackup))
{
Write-Host "$fileBackup does not exist!"
}
else
{
$fs1 = Get-Item -Path $fileBackup
...
}
}
}
I am also using PowerShell 1.0.