Nested zip contents listing - powershell

I've been working on a little side project of listing files compressed in nested zip files.
I've cooked up a script that does just that, but only if the depth of zip files is known.
In in example below the zip file has additional zips in it and then anthoer in one of them.
Add-Type -AssemblyName System.IO.Compression.Filesystem
$path = "PATH"
$CSV_Path = "CSV_PATH"
$zipFile = Get-ChildItem $path -recurse -Filter "*.zip"
$rootArchive = [System.IO.Compression.zipfile]::OpenRead($zipFile.fullname)
$rootArchive.Entries | Select #{l = 'Source Zip'; e = {} }, #{l = "FullName"; e = { $_.FullName.Substring(0, $rootArchive.Fullname.Lastindexof('\')) } }, Name | Export-csv $CSV_Path -notypeinformation
$archivesLevel2 = $rootArchive.Entries | Where { $_.Name -like "*.zip" }
foreach ($archive in $archivesLevel2)
{
(New-object System.IO.Compression.ZipArchive ($archive.Open())).Entries | Select #{l = 'Source Zip'; e = { $archive.name } }, #{l = "FullName"; e = { $archive.FullName.Substring(0, $_.Fullname.Lastindexof('\')) } }, Name | Export-Csv $CSV_Path -NoTypeInformation -append;
New-object System.IO.Compression.ZipArchive($archive.Open()) -OutVariable +lastArchiveLevel2
}
$archivesLevel3 = $lastArchiveLevel2.entries | Where { $_.Name -like "*.zip" }
foreach ($archive in $archivesLevel3)
{
(New-Object System.IO.Compression.ZipArchive ($archive.Open())).Entries | Select #{l = 'Source Zip'; e = { $archive.name } }, #{l = "FullName"; e = { $archive.FullName.Substring(0, $_.Fullname.Lastindexof('\')) } }, Name | Export-Csv $CSV_Path -NoTypeInformation -append
}
What I ask of you is to help me modify this to accomodate an unknown depth of inner zip files. Is that even possible?

Here's an example on how to do it using a Queue object, which allow you to recursively go through all depths of your zip file in one go.
As requested, here are some comments to explain what is going on.
Add-Type -AssemblyName System.IO.Compression.Filesystem
$path = "PATH"
$CSV_Path = "CSV_PATH"
$Queue = [System.Collections.Queue]::New()
$zipFiles = Get-ChildItem $path -recurse -Filter "*.zip"
# All records will be stored here
$Output = [System.Collections.Generic.List[PSObject]]::new()
# Main logic. Used when looking at the root zip and any zip entries.
# ScriptBlock is used to prevent code duplication.
$ProcessEntries = {
Param($Entries)
$Entries | % {
# Put all zip in the queue for future processing
if ([System.IO.Path]::GetExtension($entry) -eq '.zip') { $Queue.Enqueue($_) }
# Add a Source Zip property with the parent zip since we want this informations in the csv export and it is not available otherwise.
$_ | Add-Member -MemberType NoteProperty -Name 'Source Zip' -Value $zip.name
# Every entries, zip or not, need to be part of the output
$output.Add($_)
}
}
# Your initial Get-ChildItem to find zip file implicate there could be multiple root zip files, so a loop is required.
Foreach ($zip in $zipFiles) {
$archive = [System.IO.Compression.zipfile]::OpenRead($zip.fullname)
# The $ProcessEntries scriptblock is invoked to fill the Queue and the output.
. $ProcessEntries $archive.Entries
# Should the Zip file have no zip entries, this loop will never be entered.
# Otherwise, the loop will continue as long as zip entries are detected while processing any child zip.
while ($Queue.Count -gt 0) {
# Removing item from the queue to avoid reprocessing it again.
$Item = $Queue.Dequeue()
$archive = New-object System.IO.Compression.ZipArchive ($Item.open())
# We call the main scriptblock again to fill the queue and the output.
. $ProcessEntries $archive.Entries
}
}
$Output | Select 'Source Zip', FullName, Name | Export-Csv $CSV_Path -NoTypeInformation
References
Queue

Here you have a little example of how recursion would look like, basically, you loop over the .Entries property of ZipFile and check if the extension of each item is .zip, if it is, then you pass that entry to your function.
EDIT: Un-deleting this answer mainly to show how this could be approached using a recursive function, my previous answer was inaccurate. I was using [ZipFile]::OpenRead(..) to read the nested .zip files which seemed to work correctly on Linux (.NET Core) however it clearly does not work when using Windows PowerShell. The correct approach would be to use [ZipArchive]::new($nestedZip.Open()) as Sage Pourpre's helpful answer shows.
using namespace System.IO
using namespace System.IO.Compression
function Get-ZipFile {
[cmdletbinding()]
param(
[parameter(ValueFromPipeline)]
[object]$Path,
[parameter(DontShow)]
[int]$Nesting = -1
)
begin { $Nesting++ }
process {
try
{
$zip = if(-not $Nesting) {
[ZipFile]::OpenRead($Path)
}
else {
[ZipArchive]::new($Path.Open())
}
foreach($entry in $zip.Entries) {
[pscustomobject]#{
Nesting = $Nesting
Parent = $Path.Name
Contents = $entry.FullName
}
if([Path]::GetExtension($entry) -eq '.zip') {
Get-ZipFile -Path $entry -Nesting $Nesting
}
}
}
catch
{
$PSCmdlet.WriteError($_)
}
finally
{
if($null -ne $zip) {
$zip.Dispose()
}
}
}
}
Get-ChildItem *.zip | Get-ZipFile

Related

File Name and Hash comparison

How to alter the script so that it checks for the file hash only when the condition of File name matches with each folder {Pre,Post}
$result = [System.Collections.Generic.List[object]]::new()
$sb = {
process {
if($_.Name -eq 'Thumbs.db') { return }
[PSCustomObject]#{
h = (Get-FileHash $_.FullName -Algorithm SHA1).Hash
n = $_.Name
s = $_.Length
fn = $_.fullname
}
}
}
$refFiles = Get-ChildItem 'C:\Users\HP\hello\pre' -Recurse -File | & $sb
$diffFiles = Get-ChildItem 'C:\Users\HP\hello\post' -Recurse -File | & $sb
foreach($file in $diffFiles) {
# this file exists on both folders, skip it
if($file.h -in $refFiles.h) { continue }
# this file exists on reference folder but has changed
if($file.n -in $refFiles.n) {
$file.PSObject.Properties.Add(
[psnoteproperty]::new('Status', 'Changed in Ref')
)
$result.Add($file)
continue
}
# this file does not exist on reference folder
# based on previous conditions
$file.PSObject.Properties.Add(
[psnoteproperty]::new('Status', 'Unique in Diff')
)
$result.Add($file)
}
foreach($file in $refFiles) {
# this file is unique in reference folder, rest of the files
# not meeting this condition can be ignored since we're
# interested only in files on reference folder that are unique
if($file.h -notin $diffFiles.h) {
$file.PSObject.Properties.Add(
[psnoteproperty]::new('Status', 'Unique in Ref')
)
$result.Add($file)
}
}
$result | Format-Table
This Code Produces output for every hash differ from the Reference folder regardless of File Name. Thank you

PowerShell - Extracting the Metadata of of files and grid viewing it

Please see the following code:
# import .NET 4.5 compression utilities
Add-Type -As System.IO.Compression.FileSystem;
$zipArchives = Get-ChildItem "*.zip";
foreach($zipArchive in $zipArchives)
{
$archivePath = $zipArchive.FullName;
$archive = [System.IO.Compression.ZipFile]::OpenRead($archivePath);
try
{
foreach($archiveEntry in $archive.Entries)
{
if($archiveEntry.FullName -notmatch '/$')
{
$tempFile = [System.IO.Path]::GetTempFileName();
try
{
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($archiveEntry, $tempFile, $true);
$windowsStyleArchiveEntryName = $archiveEntry.FullName.Replace('/', '\');
Select-String -pattern "<dc:title>.*</dc:title>" -path (Get-ChildItem $tempFile) | Select-Object #{Name="Path";Expression={Join-Path $archivePath (Split-Path $windowsStyleArchiveEntryName -Parent)}}
#Select-String -pattern "<dc:title>.*</dc:title>" -path (Get-ChildItem $tempFile) | Select-Object Matches
#Select-String -pattern "<dc:subject>.*</dc:subject>" -path (Get-ChildItem $tempFile) | Select-Object Matches
#Select-String -pattern "<dc:date>.*</dc:date>" -path (Get-ChildItem $tempFile) | Select-Object Matches
}
finally
{
Remove-Item $tempFile;
}
}
}
}
finally
{
$archive.Dispose();
}
}
It's a modified version of code that I found on the internet and helped me to find strings inside zip files.
My intention now is to extract metadata from zip files using this code.
I don't understand how can I display the two types of information in separate lines. If you run the script with only one Select-String... pipeline line active, the code works as expected. If you activate (uncomment) the second Select-String... pipeline line, the second type of information (the <dc:title> value) is not displayed and instead there is a blank line.
Please help me:
1) How can I also display the dc:title value using the Select-String | Select-Object mechanism that I used in the code.
2) How can I output all the data in a table format, so the table would look something like this:
* ZIP Filename * DC Title *
* zipfile01.zip * Bla Bla 01 *
* zipfile02.zip * Bla Bla 02 *
* zipfile03.zip * Bla Bla 03 *
This format of output would be the most usable for me.
The console "view" for pipeline-objcts is created based on the first object (which only have a Path-property). The second object is missing a Path-property which is why you see a blank line. If you had commented out the first Select-String ..-line (that shows Path), then the second line would work.
Objects sent through the pipeline should have the same set of properties so avoid using select-object with different property-sets. Ex:
.....
$tempFile = [System.IO.Path]::GetTempFileName();
try
{
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($archiveEntry, $tempFile, $true);
[System.IO.Compression.ZipFileExtensions]::
$windowsStyleArchiveEntryName = $archiveEntry.FullName.Replace('/', '\');
Select-String -pattern "<dc:title>(.*)</dc:title>" -path (Get-ChildItem $tempFile) | Select-Object #{n="Zip FileName";e={$zipArchive.Name}}, #{Name="DC Title";Expression={ $_.Matches.Groups[1].Value}}
}
finally
{
Remove-Item $tempFile;
}
.....
To output all the metadata, you should create an object that includes all the values. Ex:
$tempFile = [System.IO.Path]::GetTempFileName();
try
{
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($archiveEntry, $tempFile, $true);
[System.IO.Compression.ZipFileExtensions]::
$windowsStyleArchiveEntryName = $archiveEntry.FullName.Replace('/', '\');
#Avoid multiple reads
$content = Get-Content $tempFile
New-Object -TypeName psobject -Property #{
"Zip Filename" = $zipArchive.Name
"DC Title" = if($content -match '<dc:title>(.*)</dc:title>') { $Matches[1] } else { $null }
"DC Subject" = if($content -match '<dc:subject>(.*)</dc:subject>') { $Matches[1] } else { $null }
"DC Date" = if($content -match '<dc:date>(.*)</dc:date>') { $Matches[1] } else { $null }
}
}
finally
{
Remove-Item $tempFile;
}
....
Ex. output
Zip Filename DC Subject DC Title DC Date
------------ ---------- -------- -------
test.zip Subject O M G 5/18/2016
If you really want to force separate views (will get ugly), then you need to send he objects to | Out-Default to create a new view every time, ex:
Select-String -pattern "<dc:title>.*</dc:title>" -path (Get-ChildItem $tempFile) | Select-Object #{Name="Path";Expression={Join-Path $archivePath (Split-Path $windowsStyleArchiveEntryName -Parent)}} | Out-Default
i know it's not the answer you were looking for, but as a temporary workaround, you may be able to combine the two commands into one like this
Select-String -pattern "<dc:title>.*</dc:title>" -path (Get-ChildItem $tempFile) | Select-Object Matches, #{Name="Path";Expression={Join-Path $archivePath (Split-Path $windowsStyleArchiveEntryName -Parent)}}

Comparing filehash and outputting files

I am new to PowerShell and am writing a script to get the hash of a directory and store it in a .txt file.
I then want to compare it to an earlier version and check for changes. If there are changes, I want a new .txt or .html file containing which line items have changed, with last modified dates.
So far, I've gotten the comparison to work, and the resulting steps based upon the pass/fail work fine.
What I need help with is outputting the results into a .txt file that lists only the files that have changed, with fields of Algorithm, Hash, Filename, Last edit time. I know I can use
(Get-Item $source).LastWriteTime
To fetch the write time, but I need to do it for every file in the directory, not just the .txt file that contains the hash.
# Variables
$Hashstore = "d:\baseline.txt"
$HashCompare = "d:\hashcompare.txt"
$HashTemp = "d:\hashtemp.txt"
$FileDir = "d:\New2"
$DateTime = Get-Date -format M.d.yyyy.hh.mm.ss
# Email Variables
$smtp_server = '<yourSMTPServer>'
$to_email = '<email>'
$from_email = '<email>'
$dns_server = "<yourExternalDNSServer>"
$domain = "<yourDomain>"
# Check if Baseline.txt Exists
If (Test-Path $Hashstore)
# // File exists
{}
Else {
# // File does not exist - Should never happen!
$RefreshHash = dir $FileDir | Get-FileHash -Algorithm MD5
$RefreshHash | Out-File $Hashstore
}
# Generate new Compare Hash.txt
$HashNew = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$HashNew | Out-File $HashCompare
# Get Hash of baseline.txt
$HashBaseline = Get-FileHash -Path d:\baseline.txt -Algorithm MD5
#Get Hash of hashcompare.txt
$HashDiff = Get-FileHash -Path d:\hashcompare.txt -Algorithm MD5
#If changed, output hash to storage, and flag changes
If ($HashBaseline.hash -eq $HashDiff.hash)
{
Add-Content -Path d:\success.$DateTime.txt -Value " Source Files ARE EQUAL </p>"
}
else
{
Add-Content -Path d:\failure.$DateTime.html -Value "Source Files NOT EQUAL </p>"
$HashNew | Out-File $HashTemp
}
# Compare two logs, send email if there is a change
If ($diff_results)
{
#$evt_message = Get-Content .\domain.new.txt | Out-String
#Write-EventLog -LogName Application -EventId 9000 -EntryType Error -Source "Maximo Validation Script" -Message $evt_message
#Send-MailMessage -To $to_email -From $from_email -SmtpServer $smtp_server -Attachments .\domain.new.txt -Subject "ALERT! Change in Records" -Body "A change has been detected in the Maximo system files.`n`n`tACTION REQUIRED!`n`nVerify that this change was authorized."
}
If ($HashNew.HashString -eq $Hashstore.HashString)
{
}
else
{
$HashTemp | Out-File $HashStore
}
I know the add-item may not be the best way to write to this log I'm creating. What would be the best way to add the last write time to every file that is read?
Here is a clean way to ouput the information you need (Algorithm, Hash, Filename, Last edit time) for each file that has changed :
$Hashstore = "d:\baseline.txt"
$HashCompare = "d:\hashcompare.txt"
$HashTemp = "d:\hashtemp.txt"
$FileDir = "d:\New2"
$DateTime = Get-Date -format M.d.yyyy.hh.mm.ss
# Check if Baseline.txt Exists
If (Test-Path $Hashstore)
# // File exists
{
}
Else {
# // File does not exist - Should never happen!
$RefreshHash = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$RefreshHash | Export-Csv -Path $Hashstore -NoTypeInformation -Force
}
# Generate new Compare Hash.txt
$HashNew = dir $FileDir -Recurse | Get-FileHash -Algorithm MD5
$HashNew | Export-Csv -Path $HashCompare -NoTypeInformation -Force
# Get Hash of baseline.txt
$HashBaseline = Get-FileHash -Path $Hashstore -Algorithm MD5
#Get Hash of hashcompare.txt
$HashDiff = Get-FileHash -Path $HashCompare -Algorithm MD5
#If changed, output hash to storage, and flag changes
If ($HashBaseline.hash -eq $HashDiff.hash) {
Add-Content -Path D:\success.$DateTime.txt -Value " Source Files ARE EQUAL </p>"
}
Else {
Add-Content -Path D:\failure.$DateTime.txt -Value "Source Files NOT EQUAL </p>"
$HashNew | Export-Csv -Path $HashTemp -NoTypeInformation -Force
# Storing a collection of differences in $Diffs
$Diffs = Compare-Object -ReferenceObject (Import-Csv $Hashstore) -DifferenceObject (Import-Csv $HashCompare)
Foreach ($Diff in $Diffs) {
$DiffHashInfo = $Diff | Select-Object -ExpandProperty InputObject
$DiffFileInfo = Get-ChildItem -Path $DiffHashInfo.Path
# Creating a list of properties for the information you need
$DiffObjProperties = [ordered]#{'Algorithm'=$DiffHashInfo.Algorithm
'Hash'=$DiffHashInfo.Hash
'Filename'=$DiffFileInfo.Name
'Last edit time'=$DiffFileInfo.LastWriteTime
}
# Building a custom object from the list of properties in $DiffObjProperties
$DiffObj = New-Object -TypeName psobject -Property $DiffObjProperties
$DiffObj
}
}
Before creating the files $Hashstore and $HashCompare, I convert the information they contain to CSV format, rather than plain text.
It makes their content much easier to manipulate later , using Import-CSV.
This makes proper objects with properties I can use.
This also makes them easier to compare, and the result of this comparison ($Diffs) is a collection of these proper objects.
So $Diffs contains all the files that have changed and I loop through each of them in a Foreach statement.
This allows you to create a custom object ($DiffObj) with exactly the information you need ($DiffObjProperties) for each of the file that have changed.
PowerShell v3+ Recursive Directory Diff Using MD5 Hashing
I use this pure PowerShell (no dependencies) recursive file content diff. It calculates in-memory the MD5 hash (the algorithm is configurable) for each directories file contents and gives results in standard PowerShell Compare-Object format.
It can optionally export to CSV files along with a summary text file. It can either drop the rdiff.ps1 file into your path or copy the contents into your script.
USAGE: rdiff path/to/left,path/to/right [-s path/to/summary/dir]
Here is the gist. I copied below for reference but I recommend using the gist version as I will be adding new features to it over time.
#########################################################################
### USAGE: rdiff path/to/left,path/to/right [-s path/to/summary/dir] ###
### ADD LOCATION OF THIS SCRIPT TO PATH ###
#########################################################################
[CmdletBinding()]
param (
[parameter(HelpMessage="Stores the execution working directory.")]
[string]$ExecutionDirectory=$PWD,
[parameter(Position=0,HelpMessage="Compare two directories recursively for differences.")]
[alias("c")]
[string[]]$Compare,
[parameter(HelpMessage="Export a summary to path.")]
[alias("s")]
[string]$ExportSummary
)
### FUNCTION DEFINITIONS ###
# SETS WORKING DIRECTORY FOR .NET #
function SetWorkDir($PathName, $TestPath) {
$AbsPath = NormalizePath $PathName $TestPath
Set-Location $AbsPath
[System.IO.Directory]::SetCurrentDirectory($AbsPath)
}
# RESTORES THE EXECUTION WORKING DIRECTORY AND EXITS #
function SafeExit() {
SetWorkDir /path/to/execution/directory $ExecutionDirectory
Exit
}
function Print {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Message to print.")]
[string]$Message,
[parameter(HelpMessage="Specifies a success.")]
[alias("s")]
[switch]$SuccessFlag,
[parameter(HelpMessage="Specifies a warning.")]
[alias("w")]
[switch]$WarningFlag,
[parameter(HelpMessage="Specifies an error.")]
[alias("e")]
[switch]$ErrorFlag,
[parameter(HelpMessage="Specifies a fatal error.")]
[alias("f")]
[switch]$FatalFlag,
[parameter(HelpMessage="Specifies a info message.")]
[alias("i")]
[switch]$InfoFlag = !$SuccessFlag -and !$WarningFlag -and !$ErrorFlag -and !$FatalFlag,
[parameter(HelpMessage="Specifies blank lines to print before.")]
[alias("b")]
[int]$LinesBefore=0,
[parameter(HelpMessage="Specifies blank lines to print after.")]
[alias("a")]
[int]$LinesAfter=0,
[parameter(HelpMessage="Specifies if program should exit.")]
[alias("x")]
[switch]$ExitAfter
)
PROCESS {
if($LinesBefore -ne 0) {
foreach($i in 0..$LinesBefore) { Write-Host "" }
}
if($InfoFlag) { Write-Host "$Message" }
if($SuccessFlag) { Write-Host "$Message" -ForegroundColor "Green" }
if($WarningFlag) { Write-Host "$Message" -ForegroundColor "Orange" }
if($ErrorFlag) { Write-Host "$Message" -ForegroundColor "Red" }
if($FatalFlag) { Write-Host "$Message" -ForegroundColor "Red" -BackgroundColor "Black" }
if($LinesAfter -ne 0) {
foreach($i in 0..$LinesAfter) { Write-Host "" }
}
if($ExitAfter) { SafeExit }
}
}
# VALIDATES STRING MIGHT BE A PATH #
function ValidatePath($PathName, $TestPath) {
If([string]::IsNullOrWhiteSpace($TestPath)) {
Print -x -f "$PathName is not a path"
}
}
# NORMALIZES RELATIVE OR ABSOLUTE PATH TO ABSOLUTE PATH #
function NormalizePath($PathName, $TestPath) {
ValidatePath "$PathName" "$TestPath"
$TestPath = [System.IO.Path]::Combine((pwd).Path, $TestPath)
$NormalizedPath = [System.IO.Path]::GetFullPath($TestPath)
return $NormalizedPath
}
# VALIDATES STRING MIGHT BE A PATH AND RETURNS ABSOLUTE PATH #
function ResolvePath($PathName, $TestPath) {
ValidatePath "$PathName" "$TestPath"
$ResolvedPath = NormalizePath $PathName $TestPath
return $ResolvedPath
}
# VALIDATES STRING RESOLVES TO A PATH AND RETURNS ABSOLUTE PATH #
function RequirePath($PathName, $TestPath, $PathType) {
ValidatePath $PathName $TestPath
If(!(Test-Path $TestPath -PathType $PathType)) {
Print -x -f "$PathName ($TestPath) does not exist as a $PathType"
}
$ResolvedPath = Resolve-Path $TestPath
return $ResolvedPath
}
# Like mkdir -p -> creates a directory recursively if it doesn't exist #
function MakeDirP {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path create.")]
[string]$Path
)
PROCESS {
New-Item -path $Path -itemtype Directory -force | Out-Null
}
}
# GETS ALL FILES IN A PATH RECURSIVELY #
function GetFiles {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path to get files for.")]
[string]$Path
)
PROCESS {
ls $Path -r | where { !$_.PSIsContainer }
}
}
# GETS ALL FILES WITH CALCULATED HASH PROPERTY RELATIVE TO A ROOT DIRECTORY RECURSIVELY #
# RETURNS LIST OF #{RelativePath, Hash, FullName}
function GetFilesWithHash {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Path to get directories for.")]
[string]$Path,
[parameter(HelpMessage="The hash algorithm to use.")]
[string]$Algorithm="MD5"
)
PROCESS {
$OriginalPath = $PWD
SetWorkDir path/to/diff $Path
GetFiles $Path | select #{N="RelativePath";E={$_.FullName | Resolve-Path -Relative}},
#{N="Hash";E={(Get-FileHash $_.FullName -Algorithm $Algorithm | select Hash).Hash}},
FullName
SetWorkDir path/to/original $OriginalPath
}
}
# COMPARE TWO DIRECTORIES RECURSIVELY #
# RETURNS LIST OF #{RelativePath, Hash, FullName}
function DiffDirectories {
[CmdletBinding()]
param (
[parameter(Mandatory=$TRUE,Position=0,HelpMessage="Directory to compare left.")]
[alias("l")]
[string]$LeftPath,
[parameter(Mandatory=$TRUE,Position=1,HelpMessage="Directory to compare right.")]
[alias("r")]
[string]$RightPath
)
PROCESS {
$LeftHash = GetFilesWithHash $LeftPath
$RightHash = GetFilesWithHash $RightPath
diff -ReferenceObject $LeftHash -DifferenceObject $RightHash -Property RelativePath,Hash
}
}
### END FUNCTION DEFINITIONS ###
### PROGRAM LOGIC ###
if($Compare.length -ne 2) {
Print -x "Compare requires passing exactly 2 path parameters separated by comma, you passed $($Compare.length)." -f
}
Print "Comparing $($Compare[0]) to $($Compare[1])..." -a 1
$LeftPath = RequirePath path/to/left $Compare[0] container
$RightPath = RequirePath path/to/right $Compare[1] container
$Diff = DiffDirectories $LeftPath $RightPath
$LeftDiff = $Diff | where {$_.SideIndicator -eq "<="} | select RelativePath,Hash
$RightDiff = $Diff | where {$_.SideIndicator -eq "=>"} | select RelativePath,Hash
if($ExportSummary) {
$ExportSummary = ResolvePath path/to/summary/dir $ExportSummary
MakeDirP $ExportSummary
$SummaryPath = Join-Path $ExportSummary summary.txt
$LeftCsvPath = Join-Path $ExportSummary left.csv
$RightCsvPath = Join-Path $ExportSummary right.csv
$LeftMeasure = $LeftDiff | measure
$RightMeasure = $RightDiff | measure
"== DIFF SUMMARY ==" > $SummaryPath
"" >> $SummaryPath
"-- DIRECTORIES --" >> $SummaryPath
"`tLEFT -> $LeftPath" >> $SummaryPath
"`tRIGHT -> $RightPath" >> $SummaryPath
"" >> $SummaryPath
"-- DIFF COUNT --" >> $SummaryPath
"`tLEFT -> $($LeftMeasure.Count)" >> $SummaryPath
"`tRIGHT -> $($RightMeasure.Count)" >> $SummaryPath
"" >> $SummaryPath
$Diff | Format-Table >> $SummaryPath
$LeftDiff | Export-Csv $LeftCsvPath -f
$RightDiff | Export-Csv $RightCsvPath -f
}
$Diff
SafeExit
Another my version. But without date/time.
# Check images. Display if differ
#
$file_path = "C:\Files"
$last_state = "last_state.json"
# Check last_state.json. If false - create new empty file.
If (!(Test-Path $last_state)) {
New-Item $last_state -ItemType file | Out-Null
}
$last_state_obj = Get-Content $last_state | ConvertFrom-Json
# Get files list and hash. Also you can use -Recurse option
Get-ChildItem $file_path -Filter *.* |
Foreach-Object {
if (!$_.PSIsContainer) {
$current_state += #($_ | Get-FileHash -Algorithm MD5)
}
}
# Compare hash
ForEach ($current_file in $current_state) {
if (($last_state_obj | where {$current_file.Path -eq $_.Path}).Hash -ne $current_file.Hash) {
$changed += #($current_file)
}
}
# Display changed files
$changed
# Save new hash to last_state.json
$current_state | ConvertTo-JSON | Out-File $last_state

reconstructing path from outlined directory structure

I have a csv file in the form:
Address,L0,L1,L2,L3,L4
01,Species,,,,
01.01,,Mammals,,,
01.01.01,,,Threatened,,
...
I want to use it to create a matching directory structure. I'm new to scripting and PowerShell, and in this case I'm not sure if I'm on totally the wrong track. Should I use a separate array to store each level's Name/Address pairs and then use those arrays like a lookup table to build the path? If so, I guess I'm stuck on how to set up if-thens based on a row's Address. This is as far as I've got so suggestions on general strategy or links to similar kinds of problem would be really welcome:
$folders = Import-Csv "E:\path\to\file.csv"
$folders | foreach {
$row = new-object PSObject -Property #{
Address = $_.Address;
Level = ([regex]::Matches($_.Address, "\." )).count;
L0 = $_.L0
L1 = $_.L1
L2 = $_.L2
L3 = $_.L3
}
$array += $row
}
#top level directories
$0 = $array | ?{$_.Level -eq 0} |
Select-Object #{n="Address";e={$_.Address;}},#{n="Name";e={$_.L0}}
#2nd level directories
$1 = $array | ?{$_.Level -eq 1} |
Select-Object #{n="Number";e={$_.Address.split(".")[-1];}},#{n="Name";e={$_.L1}}
Not tested, but I think this might do what you want:
$root = 'E:\SomeDirName'
Switch -Regex (Get-Content "E:\path\to\file.csv")
{
'^01,(\w+),,,,$' { $L1,$L2,$L3,$L4 = $null; $L0=$matches[1];mkdir "$root\$L0" }
'^01\.01,,(\w+),,,$' { $L1=$matches[1];mkdir "$root\$L0\$L1" }
'^01\.01\.01,,,(\w+),,$' { $L2=$matches[1];mkdir "$root\$L0\$L1\$L2" }
'^01\.01\.01\.01,,,,(\w+),$' { $L3=$matches[1];mkdir "$root\$L0\$L1\$L2\$L3" }
'^01\.01\.01\.01\.01,,,,,(\w+)$' { $L4=$matches[1];mkdir "$root\$L0\$L1\$L2\$L3\$L4" }
}
To solve that kind of problem a programming concept called recursion is often used.
In short a recursive function is a function that call itself.
I successfully tested this code with you CSV input:
$csvPath = 'C:\Temp\test.csv'
$folderRoot = 'C:\Temp'
$csv = Import-Csv $csvPath -Delimiter ',' | Sort-Object -Property Address
# Recursive function
function Recurse-Folder( $folderAddress, $basePath )
{
# Getting the list of current folder subfolders
$childFolders = $null
if( $folderAddress -like '' )
{
$childFolders = $csv | Where-Object { $_.Address -like '??' }
}
else
{
$childFolders = $csv | Where-Object { $_.Address -like $( $folderAddress + '.??' ) }
}
# For each child folder
foreach( $childFolder in $childFolders )
{
# Get the folder name
$dotCount = $childFolder.Address.Split('.').Count - 1
$childFolderName = $childFolder.$('L'+$dotCount)
if( $childFolderName -ne '')
{
$childFolderPath = $basePath + '\' + $childFolderName
# Creating child folder and calling recursive function for it
New-Item -Path $childFolderPath -ItemType Directory
Recurse-Folder $childFolder.Address $childFolderPath
}
}
}
Recurse-Folder '' $folderRoot

Get-Item fails with closed pipeline error

If I have an example function ...
function foo()
{
# get a list of files matched pattern and timestamp
$fs = Get-Item -Path "C:\Temp\*.txt"
| Where-Object {$_.lastwritetime -gt "11/01/2009"}
if ( $fs -ne $null ) # $fs may be empty, check it first
{
foreach ($o in $fs)
{
# new bak file
$fBack = "C:\Temp\test\" + $o.Name + ".bak"
# Exception here Get-Item! See following msg
# Exception thrown only Get-Item cannot find any files this time.
# If there is any matched file there, it is OK
$fs1 = Get-Item -Path $fBack
....
}
}
}
The exception message is ... The WriteObject and WriteError methods cannot be called after the pipeline has been closed. Please contact Microsoft Support Services.
Basically, I cannot use Get-Item again within the function or loop to get a list of files in a different folder.
Any explanation and what is the correct way to fix it?
By the way I am using PS 1.0.
This is just a minor variation of what has already been suggested, but it uses some techniques that make the code a bit simpler ...
function foo()
{
# Get a list of files matched pattern and timestamp
$fs = #(Get-Item C:\Temp\*.txt | Where {$_.lastwritetime -gt "11/01/2009"})
foreach ($o in $fs) {
# new bak file
$fBack = "C:\Temp\test\$($o.Name).bak"
if (!(Test-Path $fBack))
{
Copy-Item $fs.Fullname $fBack
}
$fs1 = Get-Item -Path $fBack
....
}
}
For more info on the issue with foreach and scalar null values check out this blog post.
I modified the above code slightly to create the backup file, but I am able to use the Get-Item within the loop successfully, with no exceptions being thrown. My code is:
function foo()
{
# get a list of files matched pattern and timestamp
$files = Get-Item -Path "C:\Temp\*.*" | Where-Object {$_.lastwritetime -gt "11/01/2009"}
foreach ($file in $files)
{
$fileBackup = [string]::Format("{0}{1}{2}", "C:\Temp\Test\", $file.Name , ".bak")
Copy-Item $file.FullName -destination $fileBackup
# Test that backup file exists
if (!(Test-Path $fileBackup))
{
Write-Host "$fileBackup does not exist!"
}
else
{
$fs1 = Get-Item -Path $fileBackup
...
}
}
}
I am also using PowerShell 1.0.