Comparing filehash and outputting files - powershell

Question

I am new to PowerShell and am writing a script to get the hash of a directory and store it in a .txt file.
I then want to compare it to an earlier version and check for changes. If there are changes, I want a new .txt or .html file containing which line items have changed, with last modified dates.
So far, I've gotten the comparison to work, and the resulting steps based upon the pass/fail work fine.
What I need help with is outputting the results into a .txt file that lists only the files that have changed, with fields of Algorithm, Hash, Filename, Last edit time. I know I can use
(Get-Item $source).LastWriteTime
To fetch the write time, but I need to do it for every file in the directory, not just the .txt file that contains the hash.
# Variables
$Hashstore = "d:\baseline.txt"
$HashCompare = "d:\hashcompare.txt"
$HashTemp = "d:\hashtemp.txt"
$FileDir = "d:\New2"
$DateTime = Get-Date -format M.d.yyyy.hh.mm.ss
# Email Variables
$smtp_server = '<yourSMTPServer>'
$to_email = '<email>'
$from_email = '<email>'
$dns_server = "<yourExternalDNSServer>"
$domain = "<yourDomain>"
# Check if Baseline.txt Exists
If (Test-Path $Hashstore)
# // File exists
{}
Else {
# // File does not exist - Should never happen!
$RefreshHash = dir $FileDir | Get-FileHash -Algorithm MD5
$RefreshHash | Out-File $Hashstore
}
# Generate new Compare Hash.txt
$HashNew = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$HashNew | Out-File $HashCompare
# Get Hash of baseline.txt
$HashBaseline = Get-FileHash -Path d:\baseline.txt -Algorithm MD5
#Get Hash of hashcompare.txt
$HashDiff = Get-FileHash -Path d:\hashcompare.txt -Algorithm MD5
#If changed, output hash to storage, and flag changes
If ($HashBaseline.hash -eq $HashDiff.hash)
{
Add-Content -Path d:\success.$DateTime.txt -Value " Source Files ARE EQUAL </p>"
}
else
{
Add-Content -Path d:\failure.$DateTime.html -Value "Source Files NOT EQUAL </p>"
$HashNew | Out-File $HashTemp
}
# Compare two logs, send email if there is a change
If ($diff_results)
{
#$evt_message = Get-Content .\domain.new.txt | Out-String
#Write-EventLog -LogName Application -EventId 9000 -EntryType Error -Source "Maximo Validation Script" -Message $evt_message
#Send-MailMessage -To $to_email -From $from_email -SmtpServer $smtp_server -Attachments .\domain.new.txt -Subject "ALERT! Change in Records" -Body "A change has been detected in the Maximo system files.`n`n`tACTION REQUIRED!`n`nVerify that this change was authorized."
}
If ($HashNew.HashString -eq $Hashstore.HashString)
{
}
else
{
$HashTemp | Out-File $HashStore
}
I know the add-item may not be the best way to write to this log I'm creating. What would be the best way to add the last write time to every file that is read?

Answer 1

Here is a clean way to ouput the information you need (Algorithm, Hash, Filename, Last edit time) for each file that has changed :
$Hashstore = "d:\baseline.txt"
$HashCompare = "d:\hashcompare.txt"
$HashTemp = "d:\hashtemp.txt"
$FileDir = "d:\New2"
$DateTime = Get-Date -format M.d.yyyy.hh.mm.ss
# Check if Baseline.txt Exists
If (Test-Path $Hashstore)
# // File exists
{
}
Else {
# // File does not exist - Should never happen!
$RefreshHash = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$RefreshHash | Export-Csv -Path $Hashstore -NoTypeInformation -Force
}
# Generate new Compare Hash.txt
$HashNew = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$HashNew | Export-Csv -Path $HashCompare -NoTypeInformation -Force
# Get Hash of baseline.txt
$HashBaseline = Get-FileHash -Path $Hashstore -Algorithm MD5
#Get Hash of hashcompare.txt
$HashDiff = Get-FileHash -Path $HashCompare -Algorithm MD5
#If changed, output hash to storage, and flag changes
If ($HashBaseline.hash -eq $HashDiff.hash) {
Add-Content -Path D:\success.$DateTime.txt -Value " Source Files ARE EQUAL </p>"
}
Else {
Add-Content -Path D:\failure.$DateTime.txt -Value "Source Files NOT EQUAL </p>"
$HashNew | Export-Csv -Path $HashTemp -NoTypeInformation -Force
# Storing a collection of differences in $Diffs
$Diffs = Compare-Object -ReferenceObject (Import-Csv $Hashstore) -DifferenceObject (Import-Csv $HashCompare)
Foreach ($Diff in $Diffs) {
$DiffHashInfo = $Diff | Select-Object -ExpandProperty InputObject
$DiffFileInfo = Get-ChildItem -Path $DiffHashInfo.Path
# Creating a list of properties for the information you need
$DiffObjProperties = [ordered]#{'Algorithm'=$DiffHashInfo.Algorithm
'Hash'=$DiffHashInfo.Hash
'Filename'=$DiffFileInfo.Name
'Last edit time'=$DiffFileInfo.LastWriteTime
}
# Building a custom object from the list of properties in $DiffObjProperties
$DiffObj = New-Object -TypeName psobject -Property $DiffObjProperties
$DiffObj
}
}
Before creating the files $Hashstore and $HashCompare, I convert the information they contain to CSV format, rather than plain text.
It makes their content much easier to manipulate later , using Import-CSV.
This makes proper objects with properties I can use.
This also makes them easier to compare, and the result of this comparison ($Diffs) is a collection of these proper objects.
So $Diffs contains all the files that have changed and I loop through each of them in a Foreach statement.
This allows you to create a custom object ($DiffObj) with exactly the information you need ($DiffObjProperties) for each of the file that have changed.

Answer 2

PowerShell v3+ Recursive Directory Diff Using MD5 Hashing
I use this pure PowerShell (no dependencies) recursive file content diff. It calculates in-memory the MD5 hash (the algorithm is configurable) for each directories file contents and gives results in standard PowerShell Compare-Object format.
It can optionally export to CSV files along with a summary text file. It can either drop the rdiff.ps1 file into your path or copy the contents into your script.
USAGE: rdiff path/to/left,path/to/right [-s path/to/summary/dir]
Here is the gist. I copied below for reference but I recommend using the gist version as I will be adding new features to it over time.
#########################################################################
### USAGE: rdiff path/to/left,path/to/right [-s path/to/summary/dir] ###
### ADD LOCATION OF THIS SCRIPT TO PATH ###
#########################################################################
[CmdletBinding()]
param (
[parameter(HelpMessage="Stores the execution working directory.")]
[string]$ExecutionDirectory=$PWD,
[parameter(Position=0,HelpMessage="Compare two directories recursively for differences.")]
[alias("c")]
[string[]]$Compare,
[parameter(HelpMessage="Export a summary to path.")]
[alias("s")]
[string]$ExportSummary
)
### FUNCTION DEFINITIONS ###
# SETS WORKING DIRECTORY FOR .NET #
function SetWorkDir($PathName, $TestPath) {
$AbsPath = NormalizePath $PathName $TestPath
Set-Location $AbsPath
[System.IO.Directory]::SetCurrentDirectory($AbsPath)
}
# RESTORES THE EXECUTION WORKING DIRECTORY AND EXITS #
function SafeExit() {
SetWorkDir /path/to/execution/directory $ExecutionDirectory
Exit
}
function Print {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Message to print.")]
[string]$Message,
[parameter(HelpMessage="Specifies a success.")]
[alias("s")]
[switch]$SuccessFlag,
[parameter(HelpMessage="Specifies a warning.")]
[alias("w")]
[switch]$WarningFlag,
[parameter(HelpMessage="Specifies an error.")]
[alias("e")]
[switch]$ErrorFlag,
[parameter(HelpMessage="Specifies a fatal error.")]
[alias("f")]
[switch]$FatalFlag,
[parameter(HelpMessage="Specifies a info message.")]
[alias("i")]
[switch]$InfoFlag = !$SuccessFlag -and !$WarningFlag -and !$ErrorFlag -and !$FatalFlag,
[parameter(HelpMessage="Specifies blank lines to print before.")]
[alias("b")]
[int]$LinesBefore=0,
[parameter(HelpMessage="Specifies blank lines to print after.")]
[alias("a")]
[int]$LinesAfter=0,
[parameter(HelpMessage="Specifies if program should exit.")]
[alias("x")]
[switch]$ExitAfter
)
PROCESS {
if($LinesBefore -ne 0) {
foreach($i in 0..$LinesBefore) { Write-Host "" }
}
if($InfoFlag) { Write-Host "$Message" }
if($SuccessFlag) { Write-Host "$Message" -ForegroundColor "Green" }
if($WarningFlag) { Write-Host "$Message" -ForegroundColor "Orange" }
if($ErrorFlag) { Write-Host "$Message" -ForegroundColor "Red" }
if($FatalFlag) { Write-Host "$Message" -ForegroundColor "Red" -BackgroundColor "Black" }
if($LinesAfter -ne 0) {
foreach($i in 0..$LinesAfter) { Write-Host "" }
}
if($ExitAfter) { SafeExit }
}
}
# VALIDATES STRING MIGHT BE A PATH #
function ValidatePath($PathName, $TestPath) {
If([string]::IsNullOrWhiteSpace($TestPath)) {
Print -x -f "$PathName is not a path"
}
}
# NORMALIZES RELATIVE OR ABSOLUTE PATH TO ABSOLUTE PATH #
function NormalizePath($PathName, $TestPath) {
ValidatePath "$PathName" "$TestPath"
$TestPath = [System.IO.Path]::Combine((pwd).Path, $TestPath)
$NormalizedPath = [System.IO.Path]::GetFullPath($TestPath)
return $NormalizedPath
}
# VALIDATES STRING MIGHT BE A PATH AND RETURNS ABSOLUTE PATH #
function ResolvePath($PathName, $TestPath) {
ValidatePath "$PathName" "$TestPath"
$ResolvedPath = NormalizePath $PathName $TestPath
return $ResolvedPath
}
# VALIDATES STRING RESOLVES TO A PATH AND RETURNS ABSOLUTE PATH #
function RequirePath($PathName, $TestPath, $PathType) {
ValidatePath $PathName $TestPath
If(!(Test-Path $TestPath -PathType $PathType)) {
Print -x -f "$PathName ($TestPath) does not exist as a $PathType"
}
$ResolvedPath = Resolve-Path $TestPath
return $ResolvedPath
}
# Like mkdir -p -> creates a directory recursively if it doesn't exist #
function MakeDirP {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path create.")]
[string]$Path
)
PROCESS {
New-Item -path $Path -itemtype Directory -force | Out-Null
}
}
# GETS ALL FILES IN A PATH RECURSIVELY #
function GetFiles {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path to get files for.")]
[string]$Path
)
PROCESS {
ls $Path -r | where { !$_.PSIsContainer }
}
}
# GETS ALL FILES WITH CALCULATED HASH PROPERTY RELATIVE TO A ROOT DIRECTORY RECURSIVELY #
# RETURNS LIST OF #{RelativePath, Hash, FullName}
function GetFilesWithHash {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path to get directories for.")]
[string]$Path,
[parameter(HelpMessage="The hash algorithm to use.")]
[string]$Algorithm="MD5"
)
PROCESS {
$OriginalPath = $PWD
SetWorkDir path/to/diff $Path
GetFiles $Path | select #{N="RelativePath";E={$_.FullName | Resolve-Path -Relative}},
#{N="Hash";E={(Get-FileHash $_.FullName -Algorithm $Algorithm | select Hash).Hash}},
FullName
SetWorkDir path/to/original $OriginalPath
}
}
# COMPARE TWO DIRECTORIES RECURSIVELY #
# RETURNS LIST OF #{RelativePath, Hash, FullName}
function DiffDirectories {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Directory to compare left.")]
[alias("l")]
[string]$LeftPath,
[parameter(Mandatory=$TRUE,Position=1,HelpMessage="Directory to compare right.")]
[alias("r")]
[string]$RightPath
)
PROCESS {
$LeftHash = GetFilesWithHash $LeftPath
$RightHash = GetFilesWithHash $RightPath
diff -ReferenceObject $LeftHash -DifferenceObject $RightHash -Property RelativePath,Hash
}
}
### END FUNCTION DEFINITIONS ###
### PROGRAM LOGIC ###
if($Compare.length -ne 2) {
Print -x "Compare requires passing exactly 2 path parameters separated by comma, you passed $($Compare.length)." -f
}
Print "Comparing $($Compare[0]) to $($Compare[1])..." -a 1
$LeftPath = RequirePath path/to/left $Compare[0] container
$RightPath = RequirePath path/to/right $Compare[1] container
$Diff = DiffDirectories $LeftPath $RightPath
$LeftDiff = $Diff | where {$_.SideIndicator -eq "<="} | select RelativePath,Hash
$RightDiff = $Diff | where {$_.SideIndicator -eq "=>"} | select RelativePath,Hash
if($ExportSummary) {
$ExportSummary = ResolvePath path/to/summary/dir $ExportSummary
MakeDirP $ExportSummary
$SummaryPath = Join-Path $ExportSummary summary.txt
$LeftCsvPath = Join-Path $ExportSummary left.csv
$RightCsvPath = Join-Path $ExportSummary right.csv
$LeftMeasure = $LeftDiff | measure
$RightMeasure = $RightDiff | measure
"== DIFF SUMMARY ==" > $SummaryPath
"" >> $SummaryPath
"-- DIRECTORIES --" >> $SummaryPath
"`tLEFT -> $LeftPath" >> $SummaryPath
"`tRIGHT -> $RightPath" >> $SummaryPath
"" >> $SummaryPath
"-- DIFF COUNT --" >> $SummaryPath
"`tLEFT -> $($LeftMeasure.Count)" >> $SummaryPath
"`tRIGHT -> $($RightMeasure.Count)" >> $SummaryPath
"" >> $SummaryPath
$Diff | Format-Table >> $SummaryPath
$LeftDiff | Export-Csv $LeftCsvPath -f
$RightDiff | Export-Csv $RightCsvPath -f
}
$Diff
SafeExit

Answer 3

Another my version. But without date/time.
# Check images. Display if differ
#
$file_path = "C:\Files"
$last_state = "last_state.json"
# Check last_state.json. If false - create new empty file.
If (!(Test-Path $last_state)) {
New-Item $last_state -ItemType file | Out-Null
}
$last_state_obj = Get-Content $last_state | ConvertFrom-Json
# Get files list and hash. Also you can use -Recurse option
Get-ChildItem $file_path -Filter *.* |
Foreach-Object {
if (!$_.PSIsContainer) {
$current_state += #($_ | Get-FileHash -Algorithm MD5)
}
}
# Compare hash
ForEach ($current_file in $current_state) {
if (($last_state_obj | where {$current_file.Path -eq $_.Path}).Hash -ne $current_file.Hash) {
$changed += #($current_file)
}
}
# Display changed files
$changed
# Save new hash to last_state.json
$current_state | ConvertTo-JSON | Out-File $last_state