Powershell - Compare multiple files against single csv files - powershell

I am trying to compare multiple files against a single document. I have managed to make that part work however where my issue is, is that i want to be able to check if the files exist before a comparison is run.
i.e. check if file A exists, if so compare against master csv file, if not continue on and check if file b exists, if so compare against master csv and so on.
my script so far goes:
$files = get-content -path "H:\Compare\File Location\servername Files.txt"
$prod = "H:\compare\Results\master_SystemInfo.csv"
foreach ($file in $files) {
If((Test-Path -path $file))
{
Write-Host "File exists, comparing against production"
$content1 = Get-Content "H:\Compare\Results\$file"
$content2 = Get-Content $prod
$comparedLines = Compare-Object $content1 $content2 -IncludeEqual |
Sort-Object { $_.InputObject.ReadCount }
$lineNumber = 0
$comparedLines | foreach {
$pattern = ".*"
if($_.SideIndicator -eq "==" -or $_.SideIndicator -eq "=>")
{
$lineNumber = $_.InputObject.ReadCount
}
if($_.InputObject -match $pattern)
{
if($_.SideIndicator -ne "==")
{
if($_.SideIndicator -eq "=>")
{
$lineOperation = "prod"
}
elseif($_.SideIndicator -eq "<=")
{
$lineOperation = "test"
}
[PSCustomObject] #{
Line = $lineNumber
File = $lineOperation
Text = $_.InputObject
}
}
}
} | Export-Csv "h:\compare\Comparison Reports\Prod.vs.$file" - NoTypeInformation
}
Else
{ "File does not exist, aborting" ; return}
}
The comparison is working just need to add the check for file before running comparison as it is still spitting out results for files that don't exist.
Thank you very much,

I have found the answer by altering the code, this time im just creating a txt file from the files in the folder first that way i don't need to test-path. This now generates a file list from the folder, then compares each file against the master file and outputs multiple files, one for each comparison saving it as the original filename i.e. "Prod.vs._SystemInfor.csv"
FYI - In the first line the abc123* is a variable i put in to look for specific server names within the folder and generate a file list based on those only. We have a number of servers all with similar naming conventions just the last 4 digits are different depending on where they are located.
Thanks
Working Powershell script:
Get-ChildItem -file abc123* H:\Compare\Results -Name | Out-File "H:\Compare\Results\Office Files.txt"
$officefiles = get-content -path "H:\Compare\results\Office Files.txt"
$officeprod = "H:\compare\Results\master_SystemInfo.csv"
foreach ($officefile in $officefiles) {
$content1 = Get-Content "H:\Compare\Results\$officefile"
$content2 = Get-Content $officeprod
$comparedLines = Compare-Object $content1 $content2 -IncludeEqual |
Sort-Object { $_.InputObject.ReadCount }
$lineNumber = 0
$comparedLines | foreach {
$pattern = ".*"
if($_.SideIndicator -eq "==" -or $_.SideIndicator -eq "=>")
{
$lineNumber = $_.InputObject.ReadCount
}
if($_.InputObject -match $pattern)
{
if($_.SideIndicator -ne "==")
{
if($_.SideIndicator -eq "=>")
{
$lineOperation = "prod"
}
elseif($_.SideIndicator -eq "<=")
{
$lineOperation = "test"
}
[PSCustomObject] #{
Line = $lineNumber
File = $lineOperation
Text = $_.InputObject
}
}
}
} | Export-Csv "h:\compare\Comparison Reports\Prod.vs.$officefile" -NoTypeInformation
}

Related

How to sort array based on naming structure

I've built a small report which essentially just does a row counts for Excel files within a share. However, there is now a requirement for the report to display the directory count in a specific order.
I cannot fathom how I'd go about that.
#Searching location
$searchinfolder = '\\Report\testing\'
#Creation of Array.
$data = #()
#Get Child items where not folder object and directory not "Postions"
$Files = Get-ChildItem -Path $searchinfolder -Recurse | Where { ! $_.Directory.Name -ne "Positions" }
Foreach ($File in $Files) {
#Main section. Get csv files, does a row count after removing top 2 and last 3 lines.
$fileStats = Get-Content $File.FullName | Select-Object -Skip 2 | Select-Object -SkipLast 3 | Measure-Object -line
$linesInFile = $fileStats.Lines - 1
#Added a counter because arrays start at 0.. need to start at 1.
$linesInFile++
#Only gets files with data in them.
if ($linesInFile -gt 0) {
$data += #(
[pscustomobject]#{
Filename = $File.fullname;
Rowcount = $linesInFile;
Directory = $File.Directory.Name
})
}
}
#Group by directory and get total sum of each file.
$data = $data | Group-Object Directory | ForEach-Object {
[PSCustomObject]#{
Directory = $_.Group.Directory | Get-Unique
Rowcount = ($_.Group.Rowcount | Measure-Object -sum).Sum
}
}
So for example, let's say the folder structure we're scraping is Cat, Dog, Goat, Programmer, Lama, Mouse.
Let's say all the folders but 1 contain files. How would I go about having the $data array arranged in a specific order of choosing? Furthermore, how would you go about setting the order and just skipping to the next assigned directory if the current directory is empty?
See below my attempt at pseudo-code trying to explain this.
Foreach ($item in $data){
if ($item.directory -eq "cat") { $item = $array[0] }
if ($item.directory -eq "dog") { $item = $array[1] }
if ($item.directory -eq "goat") { $item = $array[2] }
if ($item.directory -eq "Programmer") { $item = $array[3] }
if ($item.directory -eq "Lama") { $item = $array[4] }
if ($item.directory -eq "Mouse") { $item = $array[5] }
}

Comparison of two directories

I have a script which will compare two very similar directories to see which has newer, updated files. The two paths have the same files but the path names are locations that are slightly different. The two folders have about the same amount of files, maybe two or three less than the other.
$path1 = "E:\docs\training\files"
$path2 = "D:\docs\training - Copy\files"
$outdatedFiles = #()
foreach($file in $Folder1)
{
foreach($file2 in $Folder2)
{
if($file2.BaseName -match $file.BaseName)
{
if($file.LastWriteTime -gt $file2.LastWriteTime)
{
$Result = "" | Select OutDatedFile,LastWriteTime
$Result.OutDatedFile = $file2.FullName
$Result.LastWriteTime = $file2.LastWriteTime
$outdatedFiles += $Result
}
}
}
}
In the $outdatedFiles array, I get files that are not newer than their counterpart in the other directory. I think it might be due to my comparison in the if statement, I tried -match, -contains, and -ccontains to see if any of these would give me what I wanted. Neither worked. It might be that the foreach doesn't work due to the slightly different amount of files in each folder. Any suggestions?
EDIT
I tried building a hash but this did not find all the updated files:
$outdatedFiles = #()
foreach($file in $Folder1)
{
foreach($file2 in $Folder2)
{
if($file2.Name -like $file.Name)
{
#compare hash here
$Hash2 = Get-FileHash $file2.FullName -Algorithm SHA256
$Hash1 = Get-FileHash $file.FullName -Algorithm SHA256
if($Hash2.Hash -ne $Hash1.Hash)
{
$Result = "" | Select OutDatedFile,LastWriteTime
$Result.OutDatedFile = $file2.FullName
$Result.LastWriteTime = $file2.LastWriteTime
$outdatedFiles += $Result
}
}
}
}
EDIT
This was my solution
$Differences1 = #()
foreach($file in $Folder1)
{
foreach($file2 in $Folder2)
{
<#Trim path then compare#>
#File1
$file1part1 = ($file.FullName).Split("\")[-2]
$file1part2 = ($file.FullName).Split("\")[-1]
$newPath1 = $file1part1 + "\" + $file1part2
#File2
$file2part1 = ($file2.FullName).Split("\")[-2]
$file2part2 = ($file2.FullName).Split("\")[-1]
$newPath2 = $file2part1 + "\" + $file2part2
if($newPath1 -like $newPath2)
{
$Differences1 += Compare-Object (gci $file2.FullName) -DifferenceObject (gci $file.FullName) -Property LastWriteTime -PassThru | Select Name,FullName,LastWriteTime | Sort-Object -Property Name
}
}
}
if($Differences1 -ne $null)
{
$Differences1 | Out-File $textFile -Append
}
else
{
"No folders have different modified dates" | Out-File $textFile -Append
}
In the end, the solution was more complex than I wanted, or maybe I just made it that way.
The problem was I had multiple files with the same name and similar paths, as in the subfolder was named the same. I had to trim the path to be able to get a better comparison:
$Differences1 = #()
foreach($file in $Folder1)
{
foreach($file2 in $Folder2)
{
<#Trim path then compare#>
#File1
$file1part1 = ($file.FullName).Split("\")[-2]
$file1part2 = ($file.FullName).Split("\")[-1]
$newPath1 = $file1part1 + "\" + $file1part2
#File2
$file2part1 = ($file2.FullName).Split("\")[-2]
$file2part2 = ($file2.FullName).Split("\")[-1]
$newPath2 = $file2part1 + "\" + $file2part2
if($newPath1 -like $newPath2)
{
$Differences1 += Compare-Object (gci $file2.FullName) -DifferenceObject (gci $file.FullName) -Property LastWriteTime -PassThru | Select Name,FullName,LastWriteTime | Sort-Object -Property Name
}
}
}
if($Differences1 -ne $null)
{
$Differences1 | Out-File $textFile -Append
}
else
{
"No folders have different modified dates" | Out-File $textFile -Append
}

Matching Lines in a text file based on values in CSV

Hi Everyone,
I am having trouble with the below script. Here is the requirement:
1) Each text file needs to be compared with a single CSV file. The CSV file contains the data to that if present in the text file should match.
2) If the data in the text file matches, output the matches only and run jobs etc..
3) If the text file has no matches to the CSV file, exit with 0 as no matches are found.
I have tried to do this, but what I end up with is matches, and also non matches. What I really need is to match the lines, run the jobs,exit, if text file has no matches, then return 0
$CSVFIL = Import-Csv -Path $DRIVE\test\csvfile.csv
$TEXTFIL = Get-Content -Path "$TEXTFILFOL\*.txt" |
Select-String -Pattern 'PAT1' |
Select-String -Pattern 'PAT2' |
Select-String -Pattern 'TEST'
ForEach ($line in $CSVFIL) {
If ($TEXTFIL -match $line.COL1) {
Write-Host 'RUNNING:' ($line.JOB01)
} else {
write-host "No Matches Found Exiting"
I would handle this a different way. First you need to find matches, if there are matches then process else output 0.
$matches = #()
foreach ($line in $CSVFIL)
{
if ($TEXTFIL -contains $line.COL1)
{ $matches += $line }
}
if ($matches.Count -gt 0)
{
$matches | Foreach-Object {
Write-Output "Running: $($_.JOB01)"
}
}
else
{
Write-Output "No matches found, exiting"
}
$CSVFIL = Import-Csv -Path "$DRIVE\test\csvfile.csv"
Get-Content -Path "$TEXTFILFOL\*.txt" |
where {$_ -like "*PAT1*" -and $_ -like "*PAT2*" -and $_ -like "*TEST*" } |
%{
$TEXTFOUNDED=$_; $CSVFIL | where {$TEXTFOUNDED -match $_.COL1} |
%{ [pscustomobject]#{Job=$_.JOB01;TextFounded=$TEXTFOUNDED;Col=$_.COL1 } }
}

parsing script never ends

i have the following script but it does never end executing.
what could be the issue ? I tried to debug it but apparently it works correctly with a single file, but when I throw it against a folder full of content fails.
$path = split-path -parent $MyInvocation.MyCommand.Definition
$files = Get-ChildItem "$path\CodeForCertification\5_SourceCode\*" -Include *.c,*.h -Recurse | where{
! $_.PSIsContainer
}#$PSScriptRoot
ForEach ($file in $files){
$data = Get-Content -Path $file.FullName
$feature = Get-Content "$path\Disabled_Features.txt"
#[System.ArrayList]$Modifier
$nl=[Environment]::NewLine
$Modifier=#()
$flag=0
$data = $data | ForEach-Object -Begin {
$ignore = $false; $levels = 0
} -Process {
for($counter=0; $counter -lt $feature.Count; $counter++){
$parse = $feature[$counter]
if($_ -match "^#ifdef $parse" -And $flag -eq '0') {
$ignore = $true
$flag = 1;
}
}
if($ignore) {
if ($_ -match "^#ifdef") {
$levels++
}elseif ($_ -match "#endif") {
if($levels -ge 1) {
$levels--
if($levels -eq '0'){
$ignore = $false
}
}
}
}else {
$flag=0
$temp=$_
$_
$Modifier+="$temp"
}
}
$data | Out-File $file.FullName
}
OK, Jackson, let's solve your problem before you enter some kind of question spam filter ;-)
Consider this (just put it somewhere at the start of your script):
function RemoveUndesiredFeatures([string[]]$lines,[string[]]$undesiredFeatures)
{
$inIgnoreBlock = $false
$nestingLevel = 0
foreach ($line in $lines)
{
if ($inIgnoreBlock)
{
# Only search for nested blocks and end of block
if ($line -like "#ifdef*")
{
$nestingLevel++
}
elseif ($line -like "#endif*")
{
$nestingLevel--
}
if ($nestingLevel -eq 0)
{
$inIgnoreBlock = $false
}
}
else
{
# Search for undesired feature
$isIfdefMatch = $line -match "#ifdef (?<feature>\w+)"
if ($isIfdefMatch -and ($Matches.feature -in $undesiredFeatures))
{
# Ignore Feature
$inIgnoreBlock = $true
$nestingLevel++
}
else
{
# Output line
$line
}
}
}
}
Here is my example to use it:
$undesiredFeatures = #("F1","F2") # Just as example. Get-Content on a file with features is also fine
$files = Get-ChildItem *.c,*.h -Recurse # Again, just as example
foreach ($file in $files)
{
$lines = Get-Content $file.FullName
$changedLines = RemoveUndesiredFeatures $lines $undesiredFeatures
if ($changedLines.Count -ne $lines.Count)
{
# Features were removed. Write out changed file (to a different file to preserve my test files)
Set-Content -Value $changedLines -Path "$($file.FullName).changed"
}
}

Retrieve matching strings from two text files

I have a text file file_paths.txt that contains full paths on each line:
C:\MyFolder1\app1.exe
C:\MyFolder2\l1.dll
C:\MyFolder3\app2.exe
C:\MyFolder1\l2.dll
C:\MyFolder5\app3.exe
C:\MyFolder3\app4.exe
C:\MyFolder6\app5.exe
I also have file folders.txt that contains list of folders:
C:\MyFolder1
C:\MyFolder2
C:\MyFolder3
C:\MyFolder4
C:\MyFolder8
I need to iterate through the list of folders in folders.txt, match it with files in file_paths.txt and write the results to a file result.txt like this:
In C:\MyFolder1 more than one files has been found:
C:\MyFolder1\app1.exe
C:\MyFolder1\l2.dll
In C:\MyFolder2 one file has been:
C:\MyFolder2\l1.dll
In C:\MyFolder3 more than one files has been found:
C:\MyFolder3\app2.exe
C:\MyFolder3\app4.exe
In C:\MyFolder4 no files has been found.
In C:\MyFolder8 no files has been found.
My attempt that doesn't work:
$paths = [System.IO.File]::OpenText("file_paths.txt")
$folders = [System.IO.File]::OpenText("folders.txt")
$result = "result.txt"
try {
for(;;) {
$folder = $folders.ReadLine()
if ($folder -eq $null) { break }
"In ">> $folder >> ": `n" >> $result
for(;;) {
$path = $paths.ReadLine()
if ($path -eq $null) { break }
if ($path -contains $folder) {" ">>$path>>"`n">>$result }
}
}
} finally {
$paths.Close()
$folders.Close()
}
I would separate processing from reporting. First build a hashtable from the contents of folders.txt and add the lines from file_paths.txt to the matching keys:
$folders = #{}
Get-Content 'folders.txt' | ForEach-Object { $folders[$_] = #() }
Get-Content 'file_paths.txt' | ForEach-Object {
$line = $_
$($folders.Keys) | Where-Object {
$line -like "$_*"
} | ForEach-Object {
$folders[$_] += $line
}
}
Then you can output the resulting data structure like this:
$folders.Keys | ForEach-Object {
'In {0} {1} files have been found' -f $_, $folders[$_].Count
if ($folders[$_].Count -gt 0) {
$folders[$_] | ForEach-Object { "`t$_" }
}
} | Out-File 'result.txt'
Below is a script you can use to do exactly what you need.
Note the $folderPath and $filePath variables. Replace with absolute or relative (to where you execute the script) path of the file_paths.txt and folders.txt files.
$folderPath = 'folders.txt'
$filePath = 'file_paths.txt'
(Get-Content $folderPath).Split('`r`n') | ForEach-Object {
$folder = $_
$count = 0
$fileArray = #()
(Get-Content $filePath).Split('`r`n') | ForEach-Object {
$file = $_
if( $file | Select-String $folder -Quiet ) {
$count++
$fileArray += $file
}
}
if($count -ne 0) {
Write-Output "In $folder, $count files has been found."
$fileArray | ForEach-Object {
Write-Output "`t$_"
}
} else {
Write-Output "In $folder, no files has been found."
}
}