Efficiently loop over array, appending result to string - powershell

I'm very much a Powershell beginner (if that). I've been stringing together code from everywhere to make something work, but now I want to make it a bit more manageable to edit.
I previously had this script which I had written out all, but it is very repetitive. It was a copy from source to destination, and was easy to control with 1-5 to do. I'm now up to 14, and it's a lot of repetitive code.
I'm trying to consolidate it into an easier to maintain script.
Previously, I was copying the $copy and $result lines multiple times for each share couple.
I've now moved that into an array which I want to loop over:
# shares to copy source and destination
# blank 1st entry for easier numbering
$shares = #(
#(),
#("\\nas01\share01\", "\\nas02\share01\"),
...
#("\\nas01\share99\", "\\nas02\share99\"),
)
# loop
for($parentLoop=1; $parentLoop -lt 99; $parentLoop++) {
for($childLoop=0; $childLoop -lt 2 ; $childLoop++) {
$args = '/cmd=sync /open_window /force_close "'+$shares[$parentLoop][0]+'" /to="'+$shares[$parentLoop][1]+'"'
$copy = Start-Process -FilePath "C:\Program Files\FastCopy\FastCopy.exe" -ArgumentList $args -Wait -NoNewWindow -PassThru
$result = if( $copy.ExitCode -eq -1 ) { "FAILED" } else { "SUCCESS" }
$source = [Math]::Round(("{0:N2}" -f ((Get-ChildItem $shares[$parentLoop][0] -recurse | Measure-Object -property length -sum).Sum / 1GB) / 1024 ), 2)
$destination = [Math]::Round(("{0:N2}" -f ((Get-ChildItem $shares[$parentLoop][1] -recurse | Measure-Object -property length -sum).Sum / 1GB) / 1024 ), 2)
}
$response = "$result ( Source: $source / Destination: $destination )"
}
# get the date time
$dateTime = Get-Date -Format "dd-MM-yyyy HH:mm:ss"
# compile email
$emailBody = "Completed copy\n---\n\nShare 1: $response"
$emailArgs = "-host:192.168.0.1:25 -from:email#domain.ltd `"-to:me#domain.ltd`" `"-subject:Copy report - $dateTime`" `"-body:$emailBody`""
$email = Start-Process -FilePath "C:\Program Files\cMail\CMail.exe" -ArgumentList $emailArgs -Wait -NoNewWindow -PassThru
What I don't know how to do is to add in the output of the $response for each loop into the $emailBody.
I also don't know if there is an easier way to get the share size. I thought about mounting the share (source and destination) then capturing it locally (Get-WmiObject Win32_LogicalDisk -Filter "DeviceID='A:'" | Select-Object Size) and then unmounting it for the next loop, but wasn't sure it was efficient or if I'd run into issues of "failure to dismount".
Ideally, I'd like to be able to scale the $shares array up or down, and the email output to scale too. This was I can have a plaintext email (quicker to read than attachments) that says:
Completed copy
---
Share 1: SUCCESS (Source: 256 / Destination: 256)
...
Share 99: FAILURE (Source: 25 / Destination: 985)

Related

Print pdf files on different printers depending on their content

I want to print .pdf-files on different printers - depending on their content.
How can I check whether a specific single word is present in a file?
To queue through a folder's content I've build the following so far:
Unblock-File -Path S:\test\itextsharp.dll
Add-Type -Path S:\test\itextsharp.dll
$files = Get-ChildItem S:\test\*.pdf
$adobe='C:\Program Files (x86)\Adobe\Acrobat DC\Acrobat\Acrobat.exe'
foreach ($file in $files) {
$reader = [iTextSharp.text.pdf.parser.PdfTextExtractor]
$Extract = $reader::GetTextFromPage($File.FullName,1)
if ($Extract -Contains 'Lieferschein') {
Write-Host -ForegroundColor Yellow "Lieferschein"
$printername='XX1'
$drivername='XX1'
$portname='192.168.X.41'
} else {
Write-Host -ForegroundColor Yellow "Etikett"
$printername='XX2'
$drivername='XX2'
$portname='192.168.X.42'
}
$arglist = '/S /T "' + $file.FullName + '" "' + $printername + '" "' + $drivername + " " + $portname
start-process $adobe -argumentlist $arglist -wait
Start-Sleep -Seconds 15
Remove-Item $file.FullName
}
And for now I got 2 problems with it:
1st: Add-Type -Path itextsharp.dll gives me an error.
Add-Type: One or more types in the assembly cannot be loaded. Get the LoaderExceptions property for more information. In line: 2 character: 1
I've read that it might be due to the file being blocked. There is no information about that in the properties though. And the Unblock-File comand and the start doesn't change/solve anything.
After using $error[0].exception.loaderexceptions[0] I get the information that BouncyCastle.Crypto, Version=1.8.6.0 is missing. Unfortunatelly I can't find any sources for that yet.
2nd: Will if ($Extract -Contains 'Lieferschein') work as I intend? Will it check for the phrase after the Add-Type gets loaded successfully?
Alternatively: There's also the possibility to make it depend from the content's format. One type of the files has the size of DIN A4 for example. The other one is smaller than that. If there's an easier way to check for that, you'd make me happy aswell.
Thank you in advance!
Searching for a keyword in a pdf using Powershell and iTextSharp.dll. It's a very common thing. You then just use your conditional logic to send to whatever printer you choose.
SO, something like this should do.
Add-Type -Path 'C:\path_to_dll\itextsharp.dll'
$pdfs = Get-ChildItem 'C:\path_to_pdfs' -Filter '*.pdf'
$export = 'D:\Temp\PdfExport.csv'
$results = #()
$keywords = #('Keyword1')
foreach ($pdf in $pdfs)
{
"processing - $($pdf.FullName)"
$reader = New-Object iTextSharp.text.pdf.pdfreader -ArgumentList $pdf.FullName
for ($page = 1; $page -le $reader.NumberOfPages; $page++)
{
$pageText = [iTextSharp.text.pdf.parser.PdfTextExtractor]::GetTextFromPage($reader, $page).Split([char]0x000A)
foreach ($keyword in $keywords)
{
if ($pageText -match $keyword)
{
$response = #{
keyword = $keyword
file = $pdf.FullName
page = $page
}
$results += New-Object PSObject -Property $response
}
}
}
$reader.Close()
}
"`ndone"
$results |
Export-Csv $export -NoTypeInformation
Update
As per your comment, regarding your error.
Again, iTextSharp is a legacy, and you really need to move to iText7.
Nonetheless, that is not a PowerShell code issue. It is an iTextSharp.dll missing dependency. Even with iText7, you need to ensure you have all the dependencies on your machine and properly loaded.
As noted in this SO Q&A:
How to use Itext7 in powershell V5, Exception when loading pdfWriter
1st:
After finding the correct version (1.8.6) on nuget.org the Add-Type commands work perfectly. As expected I didn't even need the unblock command as it was not marked as a blocked file in the properties. Now the script starts with:
Add-Type -Path 'c:\BouncyCastle.Crypto.dll'
Add-Type -Path 'c:\itextsharp.dll'
2nd
Regarding the check-queue: I just had to replace -contains with -match in my if clause.
if ($Extract -Contains 'Lieferschein')

How do I correctly pass parameters to ImageMagick from PowerShell?

The following code works perfectly from the command line to combine two TIFF files.
magick -quiet file1.tif file2.tif -compress JPEG filecombined.tif
However, when I try to use it in PowerShell, I am getting many errors from Magick that indicate that it is not getting the correct parameters. My PowerShell code looks something like the following.
$InputFiles = 'files1.tif file2.tif'
$DestinationFile = 'filecombined.tif'
$magick -quiet $InputFiles -compress JPEG $DestinationFile
This gives me errors stating that it cannot find the input files and the message indicates that it thinks it is one filename instead of two. In PowerShell v4, I was able to get it to work by quoting each of the names. Not sure why this helped, but the names did not have spaces. However, I had to upgrade to v5 and this method broke.
I tried using a temporary file to store the input filenames, but this just caused a different error.
$InputFiles = 'files1.tif file2.tif'
$InputFiles | Out-File list.tmp
$DestinationFile = 'filecombined.tif'
$magick -quiet '#list.tmp' -compress JPEG $DestinationFile
magick.exe: unable to open image '#z:ÿþz
Put all the parameters for Magick into an array and use the call (&) operator to execute the command.
$MagickParameters = #( '-quiet' )
$MagickParameters += 'file1.tif'
$MagickParameters += 'file2.tif'
$MagickParameters += #( '-compress', 'JPEG' )
$MagickParameters += 'filecombined.tif'
&'magick' $MagickParameters
This may not be the most efficient use of arrays, but similar methods are possible if performance is a concern.
I had a large collection of EPS images in several folders that I had to convert to PNG. I tested many Image Conversion programs, but most could not handle recursive conversion of Vector to Raster without choking (most displayed errors after processing a limited number of files. Some could not convert recursively through many subfolders). I created the following Powershell script from various sources that solved my problem and made recursive conversion of many files and folders easier. You can modify the file to perform any ImageMagick functions you need.
Have Fun.
# Recursive-Convert-EPS-to-PNG.ps1
$srcfolder = "C:\Temp"
$destfolder = "C:\Temp"
$im_convert_exe = "convert.exe"
$src_filter = "*.eps"
$dest_ext = "png"
$options = "-depth 8 -colorspace gray -threshold 40% -alpha off"
$logfile = "C:\temp\convert.log"
$fp = New-Item -ItemType file $logfile -force
$count=0
foreach ($srcitem in $(Get-ChildItem $srcfolder -include $src_filter -recurse))
{
$srcname = $srcitem.fullname
# Construct the filename and filepath for the output
$partial = $srcitem.FullName.Substring( $srcfolder.Length )
$destname = $destfolder + $partial
$destname= [System.IO.Path]::ChangeExtension( $destname , $dest_ext )
$destpath = [System.IO.Path]::GetDirectoryName( $destname )
# Create the destination path if it does not exist
if (-not (test-path $destpath))
{
New-Item $destpath -type directory | Out-Null
}
# Perform the conversion by calling an external tool
$cmdline = $im_convert_exe + " `"" + $srcname + " `"" + $options + " `"" + $destname + " `""
#echo $cmdline
invoke-expression -command $cmdline
# Get information about the output file
$destitem = Get-item $destname
# Show and record information comparing the input and output files
$info = [string]::Format( "{0} `t {1} `t {2} `t {3} `t {4} `t {5}", $count, $partial, $srcname, $destname, $srcitem.Length , $destitem.Length)
echo $info
Add-Content $fp $info
$count=$count+1
}

Pipe all Write-Output to the same Out-File in PowerShell

As the title suggests, how do you make it so all of the Write-Outputs - no matter where they appear - automatically append to your defined log file? That way the script will be nicer to read and it removes a tiny bit of work!
Little example below, id like to see none of the "| Out-File" if possible, yet have them still output to that file!
$Author = 'Max'
$Time = Get-Date -Format "HH:mm:ss.fff"
$Title = "Illegal Software Removal"
$LogName = "Illegal_Remove_$($env:COMPUTERNAME).log"
$Log = "C:\Windows\Logs\Software" + "\" + $LogName
$RemoteLog = "\\Server\Adobe Illegal Software Removal"
Set-PSBreakpoint -Variable Time -Mode Read -Action { $global:Time = Get-Date -format "HH:mm:ss.fff" } | Out-Null
If((Test-Path $Log) -eq $False){ New-Item $Log -ItemType "File" -Force | Out-Null }
Else { $Null }
"[$Time][Startup] $Title : Created by $Author" | Out-File $Log -Append
"[$Time][Startup] Configuring initial variables required before run..." | Out-File $Log -Append
EDIT: This needs to work on PS v2.0, I don't want the output to appear on screen at all only in the log. So I have the same functionality, but the script would look like so...
"[$Time][Startup] $Title : Created by $Author"
"[$Time][Startup] Configuring initial variables required before run..."
You have two options, one is to do the redirection at the point the script is invoked e.g.:
PowerShell.exe -Command "& {c:\myscript.ps1}" > c:\myscript.log
Or you can use the Start-Transcript command to record everything (except exe output) the shell sees. After the script is done call Stop-Transcript.

'System.OutOfMemoryException' while looping through array in powershell

I was trying to write a function that to look for pool tags in .sys files. I created an array of all the directories that had .sys files then looped through them using the sysinternals Strings utility.
This is the array:
$paths = Get-ChildItem \\$server\c$ *.sys -Recurse -ErrorAction SilentlyContinue |
Select-Object Directory -unique
This was my first attempt at a loop:
foreach ($path in $paths) {
#convert object IO fileobject to string and strip out extraneous characters
[string]$path1 = $path
$path2 = $path1.replace("#{Directory=","")
$path3 = $path2.replace("}","")
$path4 = "$path3\*.sys"
Invoke-Command -ScriptBlock {strings -s $path4 | findstr $string}
}
I found some references to the error indicating that in foreach loops, all of the information is stored in memory until it completes its processing.
So I tried this:
for ($i = 0; $i -lt $paths.count; $i++){
[string]$path1 = $paths[$i]
$path2 = $path1.replace("#{Directory=","")
$path3 = $path2.replace("}","")
$path4 = "$path3\*.sys"
Invoke-Command -ScriptBlock {strings -s $path4 | findstr $string}
}
But it had the same result. I've read that sending an item at a time across the pipeline will prevent this error/issue, but I'm at a loss on how to proceed. Any thoughts?
Yeah, it is usually better to approach this problem using streaming so you don't have to buffer up a bunch of objects e.g.:
Get-ChildItem \\server\c$ -r *.sys -ea 0 | Foreach {
"Processing $_"; strings $_.Fullname | findstr $string}
Also, I'm not sure why you're using Invoke-Command when you can invoke strings and findstr directly. You typically use Invoke-Command to run a command on a remote computer.

How to speed up Powershell Get-Childitem over UNC

DIR or GCI is slow in Powershell, but fast in CMD. Is there any way to speed this up?
In CMD.exe, after a sub-second delay, this responds as fast as the CMD window can keep up
dir \\remote-server.domain.com\share\folder\file*.*
In Powershell (v2), after a 40+ second delay, this responds with a noticable slowness (maybe 3-4 lines per second)
gci \\remote-server.domain.com\share\folder\file*.*
I'm trying to scan logs on a remote server, so maybe there's a faster approach.
get-childitem \\$s\logs -include $filemask -recurse | select-string -pattern $regex
Okay, this is how I'm doing it, and it seems to work.
$files = cmd /c "$GETFILESBAT \\$server\logs\$filemask"
foreach( $f in $files ) {
if( $f.length -gt 0 ) {
select-string -Path $f -pattern $regex | foreach-object { $_ }
}
}
Then $GETFILESBAT points to this:
#dir /a-d /b /s %1
#exit
I'm writing and deleting this BAT file from the PowerShell script, so I guess it's a PowerShell-only solution, but it doesn't use only PowerShell.
My preliminary performance metrics show this to be eleventy-thousand times faster.
I tested gci vs. cmd dir vs. FileIO.FileSystem.GetFiles from #Shawn Melton's referenced link.
The bottom line is that, for daily use on local drives, GetFiles is the fastest. By far. CMD DIR is respectable. Once you introduce a slower network connection with many files, CMD DIR is slightly faster than GetFiles. Then Get-ChildItem... wow, this ranges from not too bad to horrible, depending on the number of files involved and the speed of the connection.
Some test runs. I've moved GCI around in the tests to make sure the results were consistent.
10 iterations of scanning c:\windows\temp for *.tmp files
.\test.ps1 "c:\windows\temp" "*.tmp" 10
GetFiles ... 00:00:00.0570057
CMD dir ... 00:00:00.5360536
GCI ... 00:00:01.1391139
GetFiles is 10x faster than CMD dir, which itself is more than 2x faster than GCI.
10 iterations of scanning c:\windows\temp for *.tmp files with recursion
.\test.ps1 "c:\windows\temp" "*.tmp" 10 -recurse
GetFiles ... 00:00:00.7020180
CMD dir ... 00:00:00.7644196
GCI ... 00:00:04.7737224
GetFiles is a little faster than CMD dir, and both are almost 7x faster than GCI.
10 iterations of scanning an on-site server on another domain for application log files
.\test.ps1 "\\closeserver\logs\subdir" "appname*.*" 10
GetFiles ... 00:00:00.3590359
CMD dir ... 00:00:00.6270627
GCI ... 00:00:06.0796079
GetFiles is about 2x faster than CMD dir, itself 10x faster than GCI.
One iteration of scanning a distant server on another domain for application log files, with many files involved
.\test.ps1 "\\distantserver.company.com\logs\subdir" "appname.2011082*.*"
CMD dir ... 00:00:00.3340334
GetFiles ... 00:00:00.4360436
GCI ... 00:11:09.5525579
CMD dir is fastest going to the distant server with many files, but GetFiles is respectably close. GCI on the other hand is a couple of thousand times slower.
Two iterations of scanning a distant server on another domain for application log files, with many files
.\test.ps1 "\\distantserver.company.com\logs\subdir" "appname.20110822*.*" 2
CMD dir ... 00:00:00.9360240
GetFiles ... 00:00:01.4976384
GCI ... 00:22:17.3068616
More or less linear increase as test iterations increase.
One iteration of scanning a distant server on another domain for application log files, with fewer files
.\test.ps1 "\\distantserver.company.com\logs\othersubdir" "appname.2011082*.*" 10
GetFiles ... 00:00:00.5304170
CMD dir ... 00:00:00.6240200
GCI ... 00:00:01.9656630
Here GCI is not too bad, GetFiles is 3x faster, and CMD dir is close behind.
Conclusion
GCI needs a -raw or -fast option that does not try to do so much. In the meantime, GetFiles is a healthy alternative that is only occasionally a little slower than CMD dir, and usually faster (due to spawning CMD.exe?).
For reference, here's the test.ps1 code.
param ( [string]$path, [string]$filemask, [switch]$recurse=$false, [int]$n=1 )
[reflection.assembly]::loadwithpartialname("Microsoft.VisualBasic") | Out-Null
write-host "GetFiles... " -nonewline
$dt = get-date;
for($i=0;$i -lt $n;$i++){
if( $recurse ){ [Microsoft.VisualBasic.FileIO.FileSystem]::GetFiles( $path,
[Microsoft.VisualBasic.FileIO.SearchOption]::SearchAllSubDirectories,$filemask
) | out-file ".\testfiles1.txt"}
else{ [Microsoft.VisualBasic.FileIO.FileSystem]::GetFiles( $path,
[Microsoft.VisualBasic.FileIO.SearchOption]::SearchTopLevelOnly,$filemask
) | out-file ".\testfiles1.txt" }}
$dt2=get-date;
write-host $dt2.subtract($dt)
write-host "CMD dir... " -nonewline
$dt = get-date;
for($i=0;$i -lt $n;$i++){
if($recurse){
cmd /c "dir /a-d /b /s $path\$filemask" | out-file ".\testfiles2.txt"}
else{ cmd /c "dir /a-d /b $path\$filemask" | out-file ".\testfiles2.txt"}}
$dt2=get-date;
write-host $dt2.subtract($dt)
write-host "GCI... " -nonewline
$dt = get-date;
for($i=0;$i -lt $n;$i++){
if( $recurse ) {
get-childitem "$path\*" -include $filemask -recurse | out-file ".\testfiles0.txt"}
else {get-childitem "$path\*" -include $filemask | out-file ".\testfiles0.txt"}}
$dt2=get-date;
write-host $dt2.subtract($dt)
Here is a good explanation on why Get-ChildItem is slow by Lee Holmes. If you take note of the comment from "Anon 11 Mar 2010 11:11 AM" at the bottom of the page his solution might work for you.
Anon's Code:
# SCOPE: SEARCH A DIRECTORY FOR FILES (W/WILDCARDS IF NECESSARY)
# Usage:
# $directory = "\\SERVER\SHARE"
# $searchterms = "filname[*].ext"
# PS> $Results = Search $directory $searchterms
[reflection.assembly]::loadwithpartialname("Microsoft.VisualBasic") | Out-Null
Function Search {
# Parameters $Path and $SearchString
param ([Parameter(Mandatory=$true, ValueFromPipeline = $true)][string]$Path,
[Parameter(Mandatory=$true)][string]$SearchString
)
try {
#.NET FindInFiles Method to Look for file
# BENEFITS : Possibly running as background job (haven't looked into it yet)
[Microsoft.VisualBasic.FileIO.FileSystem]::GetFiles(
$Path,
[Microsoft.VisualBasic.FileIO.SearchOption]::SearchAllSubDirectories,
$SearchString
)
} catch { $_ }
}
I tried some of the suggested methods with a large amount of files (~190.000). As mentioned in Kyle's comment, GetFiles isn't very useful here, because it needs nearly forever.
cmd dir was better than Get-ChildItems at my first tests, but it seems, GCI speeds up a lot if you use the -Force parameter. With this the needed time was about the same as for cmd dir.
P.S.: In my case I had to exclude most of the files because of their extension. This was made with -Exclude in gci and with a | where in the other commands. So the results for just searching files might slightly differ.
Here's an interactive reader that parses cmd /c dir (which can handle unc paths), and will collect the 3 most important properties for most people: full path, size, timestamp
usage would be something like $files_with_details = $faster_get_files.GetFileList($unc_compatible_folder)
and there's a helper function to check combined size $faster_get_files.GetSize($files_with_details)
$faster_get_files = New-Module -AsCustomObject -ScriptBlock {
#$DebugPreference = 'Continue' #verbose, this will take figuratively forever
#$DebugPreference = 'SilentlyContinue'
$directory_filter = "Directory of (.+)"
$file_filter = "(\d+/\d+/\d+)\s+(\d+:\d+ \w{2})\s+([\d,]+)\s+(.+)" # [1] is day, [2] is time (AM/PM), [3] is size, [4] is filename
$extension_filter = "(.+)[\.](\w{3,4})" # [1] is leaf, [2] is extension
$directory = ""
function GetFileList ($directory = $this.directory) {
if ([System.IO.Directory]::Exists($directory)) {
# Gather raw file list
write-Information "Gathering files..."
$files_raw = cmd /c dir $directory \*.* /s/a-d
# Parse file list
Write-Information "Parsing file list..."
$files_with_details = foreach ($line in $files_raw) {
Write-Debug "starting line {$($line)}"
Switch -regex ($line) {
$this.directory_filter{
$directory = $matches[1]
break
}
$this.file_filter {
Write-Debug "parsing matches {$($matches.value -join ";")}"
$date = $matches[1]
$time = $matches[2] # am/pm style
$size = $matches[3]
$filename = $matches[4]
# we do a second match here so as to not append a fake period to files without an extension, otherwise we could do a single match up above
Write-Debug "parsing extension from {$($filename)}"
if ($filename -match $this.extension_filter) {
$file_leaf = $matches[1]
$file_extension = $matches[2]
} else {
$file_leaf = $filename
$file_extension = ""
}
[pscustomobject][ordered]#{
"fullname" = [string]"$($directory)\$($filename)"
"filename" = [string]$filename
"folder" = [string]$directory
"file_leaf" = [string]$file_leaf
"extension" = [string]$file_extension
"date" = get-date "$($date) $($time)"
"size" = [int]$size
}
break
}
} # finish directory/file test
} # finish all files
return $files_with_details
} #finish directory exists test
else #directory doesn't exist {throw("Directory not found")}
}
function GetSize($files_with_details) {
$combined_size = ($files_with_details|measure -Property size -sum).sum
$pretty_size_gb = "$([math]::Round($combined_size / 1GB, 4)) GB"
return $pretty_size_gb
}
Export-ModuleMember -Function * -Variable *
}