way to determine if file is PDF faster - powershell

Question

Looking for some pointers / tips to increase the speed and/or efficacy of below. Would be open to other methods, but have only dabbled in powershell,cmd and python.
Also credit where credit is due: This is a hack-job on the following: https://stackoverflow.com/a/44183234/12834479
Rather than working local, I'm hitting a Network Share over VPN with abysmal connection speeds.
Roughly, it's working at 8 secs / PDF.
Issues I've tried to take care of, goal is to ensure each PDF is readable by Adobe. Images saved as PDF (but not pdfs) will open in some PDF software, but Adobe hates them. I have the method to convert, but my rate limiter is identifying them.
Adobe PDFs -start with %PDF
Some Bank PDFs - start with "blank space" then %PDF
3rd party software - Junk Headers, but %PDF is within document
$items = Get-ChildItem | Where-Object {$_.Extension -eq ".pdf"}
$arrary = #()
$logFile = "RESULTS_$(get-date -Format yyyymmdd).log"
$badCounter = 0
$goodCounter = 0
$msg = "`n`nProcessing " + $items.count + " files... "
Write-Host -nonewline -foregroundcolor Yellow $msg
foreach ($item in $items)
{
trap { Write-Output "Error trapped: $_"; continue; }
try {
$pdfText = Get-Content $item -raw
$ptr3 = '%PDF'
if ('%PDF' -ne $pdfText.SubString(([System.Math]::Max(0,$pdfText.IndexOf($ptr3))),4)) { $arrary+= "$item |-failed" >>$logfile;$badCounter += 1; $badCounter} else { $goodCounter += 1; $goodCounter}
continue;}
catch [System.Exception]{write-output "$item $_";}}
$totalCounter = $badCounter + $goodCounter
Write-Output $arrary >> $logFile
1..3 | %{ Write-Output "" >> $logFile }
Write-Output "Total: $totalCounter / BAD: $badCounter / GOOD: $goodCounter" >> $logFile
Write-Output "DONE!`n`n"
If any difference currently running in PS Version 7.1.3 / but also have 5.1.18 on local.

Answer 1

Actually, PDF files aren't plaintext files at all, but binary files, so you should not read them in as string.
What you are looking for is called a FourCC magic number in the file. This four-character code can be seen as Magic number to identify the file type.
For PDF files, these 4 bytes are 0x25, 0x50, 0x44, 0x46 ("%PDF") and the file should start with those bytes.
For those true PDF files, you could test with:
[byte[]]$fourCC = Get-Content -Encoding Byte -ReadCount 4 -TotalCount 4 -Path 'X:\TheFile.pdf'
if ([System.Text.Encoding]::ASCII.GetString($fourCC) -ceq '%PDF') {
Write-Host "This is a true PDF file"
}
However, as you say "Bank pdf's usually start with a blank space", to also consider those files "good", you can do:
[byte[]]$sixCC = Get-Content -Encoding Byte -ReadCount 6 -TotalCount 6 -Path 'X:\TheFile.pdf'
if ([System.Text.Encoding]::ASCII.GetString($sixCC) -cmatch '%PDF') {
Write-Host "This is a PDF file"
}
If you also want to treat files where "%PDF" is found anyhere in the file as "good", you will need to read the whole file as string, but with a one-to-one byte mapping of the bytes.
For that you can use below helper function:
function ConvertTo-BinaryString {
# converts the bytes of a file to a string that has a
# 1-to-1 mapping back to the file's original bytes.
# Useful for performing binary regular expressions.
Param (
[Parameter(Mandatory = $True, ValueFromPipeline = $True, Position = 0)]
[ValidateScript( { Test-Path $_ -PathType Leaf } )]
[String]$Path
)
# Note: Codepage 28591 returns a 1-to-1 char to byte mapping
$Encoding = [Text.Encoding]::GetEncoding(28591)
$Stream = [System.IO.FileStream]::new($Path, 'Open', 'Read')
$StreamReader = [System.IO.StreamReader]::new($Stream, $Encoding)
$BinaryText = $StreamReader.ReadToEnd()
$StreamReader.Close()
$Stream.Close()
return $BinaryText
}
Next, you can use that function as:
$binString = ConvertTo-BinaryString -Path 'X:\TheFile.pdf'
if ($binString.IndexOf("%PDF") -ge 0) {
Write-Host "This is a PDF file"
}
Putting it all together and assuming you want all files marked as .PDF files where the magic number '%PDF' (case-sensitive) can be found anywhere in the file:
function ConvertTo-BinaryString {
# converts the bytes of a file to a string that has a
# 1-to-1 mapping back to the file's original bytes.
# Useful for performing binary regular expressions.
Param (
[Parameter(Mandatory = $True, ValueFromPipeline = $True, Position = 0)]
[ValidateScript( { Test-Path $_ -PathType Leaf } )]
[String]$Path
)
# Note: Codepage 28591 returns a 1-to-1 char to byte mapping
$Encoding = [Text.Encoding]::GetEncoding(28591)
$Stream = [System.IO.FileStream]::new($Path, 'Open', 'Read')
$StreamReader = [System.IO.StreamReader]::new($Stream, $Encoding)
$BinaryText = $StreamReader.ReadToEnd()
$StreamReader.Close()
$Stream.Close()
return $BinaryText
}
$badCounter = 0
$goodCounter = 0
$logFile = "RESULTS_{0:yyyyMMdd}.log" -f (Get-Date)
# get an array of pdf file FullNames
$files = #(Get-ChildItem -File -Filter '*.pdf').FullName
Write-Host "Processing $($files.Count) files... " -ForegroundColor Yellow
# loop through the array, test if '%PDF' is found and output strings for the log file
$result = foreach ($item in $files) {
$pdfText = ConvertTo-BinaryString -Path $item
if ($pdfText.IndexOf("%PDF") -ge 0) {
$goodCounter++
"Success - $item"
}
else {
$badCounter++
"Fail - $item"
}
}
# write the output to the log file
$result | Set-Content -Path $logFile
"=" * 25 | Add-Content -Path $logFile
"BAD: $badCounter" | Add-Content -Path $logFile
"GOOD: $goodCounter" | Add-Content -Path $logFile
"Total: $($files.Count)" | Add-Content -Path $logFile
Write-Host "DONE!" -ForegroundColor Green