Compare directory with sub-folders - powershell

I have been have a tough time with this but I am trying to do a compare between two servers and have them compare all of the files from one server to the other. Even inside the subfolders. So far I've gotten it to compare the folder names and files but can't get it to go inside the folders and compare the contents. Here's a few things I've tried.
$A = Get-ChildItem -Recurse -path $Path
$B = Get-ChildItem -Recurse -path $PAth1
Compare-Object -ReferenceObject $A -DifferenceObject $B -PassThru
This is what I started with and it works the best but still doesn't go inside the sub-folders. I also tried to use foreach statements with Arrays to store the content in an array and compare the arrays but this doesn't seem to be working at all.
$FileDirectoryA = "Path"
$FileDirectoryC = "path"
$A = foreach($folderA in Get-ChildItem $fileDirectoryA)
{
$FolderA
}
$B = foreach($FileA in Get-ChildItem $ArrayA)
{
$FileA
}
$C = foreach($folderC in Get-ChildItem $fileDirectoryC)
{
$FolderC
}
$D = foreach($FileC in Get-ChildItem $ArrayC)
{
$FileC
}
Compare-Object $B $D
When I try and just do a
Compare-Object $Path $Path2
It errors on all of the folders saying permission denied which doesn't make much sense to me.
Thanks,
Andrew

It should work if you specify the property to compare :
$a = ls c:\temp\ -rec
$b = icm -computername $servername -scriptblock{ls c:\temp\ -rec}
compare $a $b -Property fullname
A more reliable way might be to use robocopy
robocopy.exe \\serverA\Folder1 \\serverB\Folder2 /e /l /log:c:\temp\dif.txt

I think this powershell script https://github.com/oflisback/dirdiff addresses your problem, it scans two directories and reports differences:
PS C:\> dirdiff.ps1 C:\dir1 C:\dir2
One identical file:
.\identical.txt
One file only present in C:\dir1:
.\newdir1.txt
2 files only present in C:\dir2:
.\newdir2.txt
.\newdir\newfilesdir2.txt
One file present in both directories and has different content:
.\modified.bin
The script is based on this gist so credit to cchamberlein.

Get-ChildItem $Source -Recurse | ForEach-Object{
$Dest = Join-Path $Destination $_.FullName.Substring($Source.Length)
$bool = Test-Path $Dest
if(!$bool){
#Write of some thing here to indicate difference
}
}

This should give you a reasonable comparison I think. You can use regex on the return values to rebuild the original filepaths if you so desire.
$Dir1="C:\Test"
$Dir2="C:\Test2"
$A=(Get-ChildItem $Dir1 -Recurse).VersionInfo.Filename -replace [regex]::Escape($Dir1),""
$B=(Get-ChildItem $Dir2 -Recurse).VersionInfo.Filename -replace [regex]::Escape($Dir2),""
Compare-Object $A $B
This reuslts in an output like...
InputObject SideIndicator
----------- -------------
\New folder\temp.txt =>
\temp.txt <=
\New folder\New folder\temp.txt <=
Update
I think should do everything you are looking for. You'll need to change the $Dir1 and $Dir2 values. It's likely over-complicated but it does appear to work.
$Dir1="C:\Test"
$Dir2="C:\Test2"
$Items1=Get-ChildItem $Dir1 -Recurse
$Items2=Get-ChildItem $Dir2 -Recurse
$A=$Items1.VersionInfo.Filename -replace [regex]::Escape($Dir1),""
$B=$Items2.VersionInfo.Filename -replace [regex]::Escape($Dir2),""
$Exist_Diff=Compare-Object $A $B
$Exist_Diff | %{
$Current_Item=$_.InputObject
if($_.SideIdicator -eq "<="){
Write-Host "No file equivilent to $Dir1$Current_Item found in $Dir2"
}else{
Write-Host "No file equivilent to $Dir2$Current_Item found in $Dir1"
}
}
$Items1 | %{
if ($Exist_Diff.InputObject -notcontains ($_.VersionInfo.Filename -replace [regex]::Escape($Dir1),""))
{
$MatchingFile=[string]$_.VersionInfo.Filename -replace [regex]::Escape($Dir1),$Dir2
try{if(Test-Path $MatchingFile)
{
if((Get-Item $MatchingFile).length -gt $_.Length)
{
Write-host $_.fullname "is larger than" $MatchingFile
}
}}catch{}
}
}
$Items2 | %{
if ($Exist_Diff.InputObject -notcontains ($_.VersionInfo.Filename -replace [regex]::Escape($Dir1),""))
{
$MatchingFile=[string]$_.VersionInfo.Filename -replace [regex]::Escape($Dir2),$Dir1
try{if(Test-Path $MatchingFile)
{
if((Get-Item $MatchingFile).length -gt $_.Length)
{
Write-host $_.fullname "is larger than" $MatchingFile
}
}}catch{}
}
}

Related

How to parse through folders and files using PowerShell?

I am trying to construct a script that moves through specific folders and the log files in it, and filters the error codes. After that it passes them into a new file.
I'm not really sure how to do that with for loops so I'll leave my code bellow.
If someone could tell me what I'm doing wrong, that would be greatly appreciated.
$file_name = Read-Host -Prompt 'Name of the new file: '
$path = 'C:\Users\user\Power\log_script\logs'
Add-Type -AssemblyName System.IO.Compression.FileSystem
function Unzip
{
param([string]$zipfile, [string]$outpath)
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipfile, $outpath)
}
if ([System.IO.File]::Exists($path)) {
Remove-Item $path
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
} else {
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
}
$folder = Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles'
$files = foreach($logfolder in $folder) {
$content = foreach($line in $files) {
if ($line -match '([ ][4-5][0-5][0-9][ ])') {
echo $line
}
}
}
$content | Out-File $file_name -Force -Encoding ascii
Inside the LogFiles folder are three more folders each containing log files.
Thanks
Expanding on a comment above about recursing the folder structure, and then actually retrieving the content of the files, you could try something line this:
$allFiles = Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' -Recurse
# iterate the files
$allFiles | ForEach-Object {
# iterate the content of each file, line by line
Get-Content $_ | ForEach-Object {
if ($_ -match '([ ][4-5][0-5][0-9][ ])') {
echo $_
}
}
}
It looks like your inner loop is of a collection ($files) that doesn't yet exist. You assign $files to the output of a ForEach(...) loop then try to nest another loop of $files inside it. Of course at this point $files isn't available to be looped.
Regardless, the issue is you are never reading the content of your log files. Even if you managed to loop through the output of Get-ChildItem, you need to look at each line to perform the match.
Obviously I cannot completely test this, but I see a few issues and have rewritten as below:
$file_name = Read-Host -Prompt 'Name of the new file'
$path = 'C:\Users\user\Power\log_script\logs'
$Pattern = '([ ][4-5][0-5][0-9][ ])'
if ( [System.IO.File]::Exists( $path ) ) { Remove-Item $path }
Expand-Archive 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
Select-String -Path 'C:\Users\user\Power\log_script\logs\LogFiles\*' -Pattern $Pattern |
Select-Object -ExpandProperty line |
Out-File $file_name -Force -Encoding ascii
Note: Select-String cannot recurse on its own.
I'm not sure you need to write your own UnZip function. PowerShell has the Expand-Archive cmdlet which can at least match the functionality thus far:
Expand-Archive -Path <SourceZipPath> -DestinationPath <DestinationFolder>
Note: The -Force parameter allows it to over write the destination files if they are already present. which may be a substitute for testing if the file exists and deleting if it does.
If you are going to test for the file that section of code can be simplified as:
if ( [System.IO.File]::Exists( $path ) ) { Remove-Item $path }
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
This is because you were going to run the UnZip command regardless...
Note: You could also use Test-Path for this.
Also there are enumerable ways to get the matching lines, here are a couple of extra samples:
Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' |
ForEach-Object{
( Get-Content $_.FullName ) -match $Pattern
# Using match in this way will echo the lines that matched from each run of
# Get-Content. If nothing matched nothing will output on that iteration.
} |
Out-File $file_name -Force -Encoding ascii
This approach will read the entire file into an array before running the match on it. For large files it may pose a memory issue, however it enabled the clever use of -match.
OR:
Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' |
Get-Content |
ForEach-Object{ If( $_ -match $Pattern ) { $_ } } |
Out-File $file_name -Force -Encoding ascii
Note: You don't need the alias echo or its real cmdlet Write-Output
UPDATE: After fuzzing around a bit and trying different things I finally got it to work.
I'll include the code below just for demonstration purposes.
Thanks everyone
$start = Get-Date
"`n$start`n"
$file_name = Read-Host -Prompt 'Name of the new file: '
Out-File $file_name -Force -Encoding ascii
Expand-Archive -Path 'C:\Users\User\Power\log_script\logs.zip' -Force
$i = 1
$folders = Get-ChildItem -Path 'C:\Users\User\Power\log_script\logs\logs\LogFiles' -Name -Recurse -Include *.log
foreach($item in $folders) {
$files = 'C:\Users\User\Power\log_script\logs\logs\LogFiles\' + $item
foreach($file in $files){
$content = Get-Content $file
Write-Progress -Activity "Filtering..." -Status "File $i of $($folders.Count)" -PercentComplete (($i / $folders.Count) * 100)
$i++
$output = foreach($line in $content) {
if ($line -match '([ ][4-5][0-5][0-9][ ])') {
Add-Content -Path $file_name -Value $line
}
}
}
}
$end = Get-Date
$time = [int]($end - $start).TotalSeconds
Write-Output ("Runtime: " + $time + " Seconds" -join ' ')

Test-Path returning true on empty folders

Test-Path is returning true, or at least appears to be on folders that are empty if they are located in the same directory as folders returning true that are NOT empty.
I've tried adjusting wildcard locations, adding additional \ to change paths to see if I could reduce the number of returned folders, but I cannot. I assumed the foreach portion and Test-Path would individually check each folder and return a result, but it appears once it sees contents, all folders thereafter are returned as true.
[int]$subDay = Read-Host "Enter days to subtract"
$date0 = (Get-Date).AddDays(-$subDay).ToString("yy") +
((Get-Date).AddDays(-$subDay).DayOfYear).ToString("D3")
$date1 = (Get-Date).AddDays(-$subDay).ToString("yyyy") +
((Get-Date).AddDays(-$subDay).DayOfYear).ToString()
$path0 = ".\11"
$path1 = ".\12"
$path2 = ".\13"
$a = $path0, $path1, $path2
function Split {
Param ($split)
Split-Path -Parent $split |
Get-Item |
Select-Object -ExpandProperty Name |
Add-Content $archLog
foreach ($element in $a) {
if (Test-Path $element\$date1\"$date0*"\*) {
Split $element\$date1
Split $element\$date1\"$date0*"\*
} else {
Split $element\$date1
Write-Output "Folders do not exist or are empty." |
Add-Content $archlog
}
}
I expected the code to return folder name if it had contents, which it does. However, if an empty folder exists where a folder with contents does, both are returned. If you take away the contents of the folder, none are returned.
To be able to see every subfolder that contains a file use the parameters -file and -recurse of the Get-ChildItem cmdlet:
$a = $path0, $path1, $path2
foreach ($element in $a) { Get-ChildItem -Path $element\$date1 -file -Recurse | foreach {$_.Directory.Name} | select -uniq }

Powershell - Exclude folders in Get-ChildItem

How to exclude folders ? Now I hardcode the folder names but i want it to be more flexible.
foreach($file in Get-ChildItem $fileDirectory -Exclude folderA,folderb)
"How to exclude folders ?" , if you mean all folders :
get-childitem "$fileDirectory\\*" -file
but it works only for the first level of $fileDirectory .
This works recursevly :
Get-ChildItem "$fileDirectory\\*" -Recurse | ForEach-Object { if (!($_.PSIsContainer)) { $_}}
or
Get-ChildItem "$fileDirectory\\*" -Recurse | where { !$_.PSisContainer }
You can do this by using the pipeline and a Where-Object filter.
First of all, the idiomatic way to iterate over a group of files in PowerShell is to pipe Get-Childitem to Foreach-Object. So rewriting your command gets:
Get-ChildItem $fileDirectory | foreach {
$file = $_
...
}
The advantage of using the pipeline is that now you can insert other cmdlets in between. Specifically, we use Where-Object to filter the list of files. The filter will pass on a file only if it isn't contained in a given array.
$excludelist = 'folderA', 'folderB'
Get-Childitem $fileDirectory |
where { $excludeList -notcontains $_ } |
foreach {
$file = $_
...
}
If you're going to use this a lot, you can even write a custom filter function to modify the list of files in an arbitrary way before passing to foreach.
filter except($except, $unless = #()) {
if ($except -notcontains $_ -or $unless -contains $_ ){
$_
}
}
$excludelist = 'folderA', 'folderB'
$alwaysInclude = 'folderC', 'folderD'
Get-ChildItem $fileDirectory |
except $excludeList -unless $alwaysInclude |
foreach {
...
}
#dvjz said that -file works only in the first level of a folder, but not recursively. But it seems to work for me.
get-childitem "$fileDirectory\\*" -file -recurse
For future googlers, I have found that files have a property called PSIsContainer which is $true when they are a directory.
A command listing all files in $fileDirectory would be:
foreach ($file in Get-ChildItem $fileDirectory | Where-Object -Property PSIsContainer -eq $false)
{
Write-Host $file.Name
}
Note that -Property is optional for the cmdlet Where-Object.
The simplest way to exclude your folders recursively:
foreach($file in Get-ChildItem $fileDirectory -Exclude {Get-ChildItem folderA},{Get-ChildItem folderB})
Where:
$fileDirectory - search folder
folderA, folderB - excluded folders

powershell iterate through multiple subfolders for differences and move to multiple targets

Relatively new to PS. I am trying to write a PS that will iterate through multiple source and target subfolders with the same folder names. Do a gci against each folder in both source and target, perform a diff and copy the differences from sources to target. I have a script working for a single sub-folder.
############# source #############
$s = "C:\LogBackup\ConversionExpress\"
$src_conexp = [string[]](gci $s -recurse -include *.safe, *.done) #test location
############ target #############
$t = "C:\LogBackup\ConversionExpress2\" #TST
$trg_conexp = [string[]](gci $t -recurse -include *.safe, *.done) #TST
diff -ReferenceObject ($src_conexp) -DifferenceObject ($trg_conexp) -Passthru |
% {
cpi -path $_ -Destination $t -Force
}
My issue is, I have 6 sub-folders in source and target that I need to do a diff against and copy the resulting diff's to the corresponding target file location.
Here is where my ugly R&D has got me:
#$pfolder = "C:\LogBackup\"
$sfolders = $(gci "C:\LogBackup\");
$tfolders = $(gci "C:\LogBackup2\");
foreach ($s in $sfolders)
{
$sfile = (gci "C:\LogBackup\$_" -r -include *.safe, *.done);
}
foreach ($t in $tfolders)
{
$tfile = (gci "C:\LogBackup2\$_" -r -include *.safe, *.done);
diff -ReferenceObject ($sfile) -DifferenceObject ($tfile) -Passthru |
% {
cpi -path $_ -Destination $t -Force
}
}
I wanted to maybe see if I could get this function I found to work as well.
Function GetFiles($folder)
{
Write-Host "+"$folder.Name
foreach($file in $folder.Files)
{
Write-Host "`t" $file.Name
}
# Use recursion to loop through all subfolders.
foreach ($subFolder in $folder.SubFolders)
{
Write-Host "`t" -NoNewline
GetFiles($Subfolder)
}
}
any direction or help to get me past this would be greatly appreciated.
Thanks.

Using PowerShell to compare and copy files of a certain type recursively

I am trying to copy a particular file type such as foo.pbo. These pbo's are scatted in a lot of sub-directories below the source folder. In the destination it is a folder with one sub-directory below it.
I was to use powershell to compare the source folder and sub-directories with files of a .pbo extension to the destination folder, but here is the twist only copy the files that are newer.
I have tried breaking it down, so here was my attempt.
$s = Get-ChildItem .\mods -filter *pbo -Recurse
$d = Get-ChildItem .\updates\addons -filter *pbo
Compare-Object -ReferenceObject $s -DifferenceObject $d -Property Name, LastWriteTime | Where-Object { $_.SideIndicator -eq "=>" }
How can I copy the files that are found in the compare that are modified in the multiple source directories to a single directory?
All I get in the compare is just the filenames and date modified, I have no path to reference?
I think Compare-Object is the wrong tool for this. It compares the existence of properties - you have no way of specifying that it should compare the dates.
I suggest iterating over each source object with a foreach loop and checking if it needs to be copied.
$s = Get-ChildItem .\source -Filter *.txt -Recurse
$d = Get-ChildItem .\dest -Filter *.txt
foreach ($file in $s) {
$targetFile = $d | where Name -eq $file.Name
# this copies files which do not exist in the target
if ($file.LastWriteTime -gt $targetFile.LastWriteTime) {
Copy-Item $file.FullName .\dest
}
# this copies only files which exist in the target
if ($targetFile -and $file.LastWriteTime -gt $targetFile.LastWriteTime) {
Copy-Item $file.FullName .\dest
}
}
Assuming the Compare-Object is working properly the data you seek is in InputObject note property. At the end of your Where-Object pipe into Select-Object
Where-Object { $_.SideIndicator -eq "=>" } | Select-Object -ExpandProperty InputObject
That will return System.IO.FileInfo objects just like Get-ChildItem that you can do with what you please.
$old_files = Get-ChildItem "C:\Users\USERNAME\Documents\old_files"
$new_files = Get-ChildItem "C:\Users\USERNAME\Documents\new_files"
foreach($file in $new_files){
if($old_files.Name.Contains($file.Name)){}
else {
Copy-Item $file.FullName "C:\Users\USERNAME\Documents\new_addition"
}
}