powershell truncates file fullnames ONLY as automated task - powershell

We have a file server that processes files that are received. When a file fails to process for whatever reason, it is moved into a failure folder. I've written a script to iterate through every possible one of these folders and spit out the FullName of the file into an e-mail which it sends to me.
Now when I run it manually, it works fine. However, when I set it as a scheduled task (running as Local System), the script still runs successfully, but the e-mail contains paths like \\blahblah\blah\blahblahblah\bl.....
I've tweaked the script a bunch of different ways and every time the output ends up the same. When I run it manually, it works as intended, when it runs as an automated script, it truncates the FullNames. I've found other people with this issue, but not as an automated task.
This is the relevant code of the script.
$emailFileList = ""
$filelist = #()
try {
GCI $topLevelPath -Recurse |
? { $_.PSIsContainer } |
ForEach-Object {
dir $_.FullName |
Where-Object {$_.FullName -like $unableToProcess} | ForEach-Object {
$filelist += dir $_.FullName
}
}
$emailFileList = Out-String -InputObject $($filelist | Select-Object FullName | Format-Table -AutoSize)
$emailBody = $emailBody + $emailFileList
}
EDIT:
I used the HTML method below but it added a bunch of junk markup. I added 4 lines to replace the markup with quotes. The inside of the try block now looks like this, and it works even as scheduled tasks.
GCI $topLevelPath -Recurse |
? { $_.PSIsContainer } |
ForEach-Object {
dir $_.FullName |
Where-Object {$_.FullName -like $unableToProcess} | ForEach-Object {
$filelist += dir $_.FullName
}
}
$emailFileList = $filelist | Select-Object FullName | ConvertTo-Html -fragment
$emailFileList = [regex]::Replace($emailFileList, "<table>.+</th></tr>", "")
$emailFileList = $emailFileList -replace '<tr><td>', '"'
$emailFileList = $emailFileList -replace '</td></tr>', """`r`n"
$emailFileList = $emailFileList -replace '</table>', ''
$emailBody = $emailBody + $emailFileList
I guess I also technically used regex on html what have I done noooooooo
Edit: Regardling answer "duplication" the problem above is SPECIFICALLY an interaction between powershell and the windows scheduled tasks.

This gives the kind of output you would probably expect
[command] | Format-Table -AutoSize | Out-String -Width 10000 #| clip.exe

Since you were using Format-Table -Autosize it was probably truncating due to the the amount of characters per line in the powershell instance. You can use the ConvertTo-Html function with the -Fragment command to create an HTML table.
Try something like this:
$emailFileList = ""
$filelist = #()
try {
Get-ChildItem $topLevelPath -Recurse `
| Where-Object -Property PSIsContainer -EQ -Value $True `
| ForEach-Object {
Get-ChildItem $_.FullName |
Where-Object -Property FullName -Like -Value $UnableToProcess `
| ForEach-Object {
$filelist += Get-ChildItem $_.FullName
}
}
$emailFileList = $filelist | Select-Object FullName | ConvertTo-Html -Fragment
$emailBody = $emailBody + $emailFileList
}
catch
{
}

Your problem is the formatter truncating because the console host can't render the full strings. Here's a solution where you'll get a txt with a list of names that can be used however you want
General rule of thumb: filter left, format right.
#Requires -Version 3
Try
{
Get-ChildItem -Path $TopLevelPath -Recurse -Folder |
## If you're on version 2, replace the -Folder switch with the following:
#Where-Object { $_.PSIsContainer }
ForEach-Object {
## If version 2, remove #().FullName and replace with
# | Select-Object -ExpandProperty FullName
$FileList = #(Get-ChildItem -Path $_.FullName -Filter "*$UnableToProcess*").FullName
}
$EmailBody += #($FileList)
}
Catch
{
}

Related

Powershell - Combine CSV files and append a column

I'm trying (badly) to work through combining CSV files into one file and prepending a column that contains the file name. I'm new to PowerShell, so hopefully someone can help here.
I tried initially to do the well documented approach of using Import-Csv / Export-Csv, but I don't see any options to add columns.
Get-ChildItem -Filter *.csv | Select-Object -ExpandProperty FullName | Import-Csv | Export-Csv CombinedFile.txt -UseQuotes Never -NoTypeInformation -Append
Next I'm trying to loop through the files and append the name, which kind of works, but for some reason this stops after the first row is generated. Since it's not a CSV process, I have to use the switch to skip the first title row of each file.
$getFirstLine = $true
Get-ChildItem -Filter *.csv | Where-Object {$_.Name -NotMatch "Combined.csv"} | foreach {
$filePath = $_
$collection = Get-Content $filePath
foreach($lines in $collection) {
$lines = ($_.Basename + ";" + $lines)
}
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 1}
}
$getFirstLine = $false
Add-Content "Combined.csv" $linesToWrite
}
This is where the -PipelineVariable parameter comes in real handy. You can set a variable to represent the current iteration in the pipeline, so you can do things like this:
Get-ChildItem -Filter *.csv -PipelineVariable File | Where-Object {$_.Name -NotMatch "Combined.csv"} | ForEach-Object { Import-Csv $File.FullName } | Select *,#{l='OriginalFile';e={$File.Name}} | Export-Csv Combined.csv -Notypeinfo
Merging your CSVs into one and adding a column for the file's name can be done as follows, using a calculated property on Select-Object:
Get-ChildItem -Filter *.csv | ForEach-Object {
$fileName = $_.Name
Import-Csv $_.FullName | Select-Object #{
Name = 'FileName'
Expression = { $fileName }
}, *
} | Export-Csv path/to/merged.csv -NoTypeInformation

Powershell - Find all extensions on network shares

I'm new to PS scripting (really, I started today) and, for a project, I need to create a .txt file with all the extensions from all shared folders on the local machine (a Windows file server).
I think I'm on the right path with this :
get-childitem -Path C:\test -Recurse | select extension -unique > $PSScriptRoot\ExtensionList.txt
It's doing exactly what I want for a given path and all subfolders but now I need to apply this to all shared folders on the machine.
I was able to list all the shared folder's path with this command :
$Shares= #(Get-WmiObject Win32_Share |
Select Name,Path,Type |
Where-Object { $_.Type -match '0|2147483648' } |
Select -ExpandProperty Path |
Select -Unique)
Write-Host $Shares
Now I'm stuck, I suppose I need to use the foreach command but I can't find the way to make it work.
Can someone help me put this together ?
Thanks,
You can try Get-SMBShare cmdLet:
Get-SMBShare | Foreach {
Get-ChildItem "\\$($_.name)" | Select-Object Extension -Unique
}
You're probably looking for something similar to this:
$Shares = #( Get-CimInstance Win32_Share | Where-Object { $_.Type -match '0|2147483648' } | Select -Unique )
ForEach ( $Share In $Shares ) { Get-ChildItem -Path $Share.Path -File -Recurse -ErrorAction Ignore | Select -Unique -ExpandProperty Extension }
I'll leave you to split the lines to match your particular style and to output to a file, (I'd advise that you consider using Out-File instead of > for that).
Thank you guys for your help! I was able to figure it out.
The following script will gather all extensions on shared folders, sort them, eliminate duplicates and empty lines, add "*' before the extension and create a file list.txt with the result.
#get shares
$Shares = #( Get-CimInstance Win32_Share |
Where-Object { $_.Type -match '0|2147483648' } |
Select -Unique )
#list all extensions
ForEach ( $Share In $Shares ) { Get-ChildItem -Path $Share.Path -File -Recurse -ErrorAction Ignore | Select -Unique -ExpandProperty Extension | out-file C:\extensions\List1.txt -append }
#remove empty lines
#(gc C:\extensions\List1.txt) -match '\S' | out-file C:\extensions\List2.txt
#Add * before extention type
gc C:\extensions\List2.txt | %{"*$_"} | out-file C:\extensions\List3.txt
#Sort by name
gc C:\extensions\List3.txt | sort | get-unique > C:\extensions\List4.txt
#Remove duplicates
$hash = #{}
gc C:\extensions\List4.txt |
%{if($hash.$_ -eq $null) { $_ }; $hash.$_ = 1} > C:\extensions\List.txt
#Delete list1-4
Remove-Item C:\extensions\List1.txt, C:\extensions\List2.txt, C:\extensions\List3.txt, C:\extensions\List4.txt

Write all running processes to a text file in PowerShell

The purpose of this code is to get a list of all used executables from a specific folder. After a month we will delete any exe's not on this list.
I currently get the correct results using this:
while ($true) {
foreach ($process in Get-Process | where {$_.Path -imatch 'ksv'} | select -Unique) {
$dir = $process | Get-ChildItem;
New-Object -TypeName PSObject -Property #{
'Path' = $process.Path;
} | Out-String | Add-Content -LiteralPath Z:\processList.txt
}
Get-Content Z:\processList.txt | sort | Get-Unique > Z:\uniqueprocesslist.txt
}
I'm going to get rid of the while loop as this will be eventually running as a service.
The problem with this is that it creates a huge list in processlist.txt that I would like to eliminate to save space.
I tried to come up with a better solution that scans the text file to see if the path is written already before adding the new process path. I am not sure what I am doing wrong but nothing is ever written to the text file
while ($true) {
foreach ($process in Get-Process | where {$_.Path -imatch 'ksv'} | select -Unique) {
$dir = $process | Get-ChildItem;
$progPath = New-Object -TypeName PSObject -Property #{
'Path' = $process.Path
}
$file = Get-Content "Z:\processList.txt"
$containsLine = $file | %{$_ -match $progPath}
if ($containsLine -contains $false) {
Add-Content -LiteralPath Z:\processList.txt
}
}
}
If I understand your question correctly you want to build a "recently used" list of executables in a specific directory in a file, and update that (unique) list with each run of your script.
Something like this should do that:
$listfile = 'Z:\processlist.txt'
# Build a dictionary from known paths, so that we can check for already known
# paths with an index lookup instead of a linear search over an array.
$list = #{}
if (Test-Path -LiteralPath $listfile) {
Get-Content $listfile | ForEach-Object {
$list[$_] = $true
}
}
# List processes, expand their path, then check if the path contains the
# string "ksv" and isn't already known. Append the results to the list file.
Get-Process |
Select-Object -Expand Path |
Sort-Object -Unique |
Where-Object {$_ -like '*ksv*' -and -not $list.ContainsKey($_)} |
Add-Content $listfile
Hashtable lookup and wildcard match are used for performance reasons, because they're significantly faster than linear searches in arrays and regular expression matches.
while ($true) {
$file = Get-Content "Z:\processList.txt"
$KSVPaths = Get-Process |
Where-Object {$_.Path -imatch 'ksv'} |
Select-Object -ExpandProperty Path |
Select-Object -Unique
ForEach ($KSVPath in $KSVPaths) {
if ($KSVPath -notin $file) {
Add-Content -Path $file -Value $KSVPath
}
}
}

Combining CSV files in Powershell - different headings

I need to take a slew of csv files from a directory and get them into an array in Powershell (to eventually manipulate and write back to a CSV).
The problem is there are 5 file types. I need around 8 columns from each. The columns are essentially the same, but have different headings.
Is there an easy way to do this? I started creating a custom object with my 8 fields, looping through the files importing each one, looking at the filename (which tells me the column names I need) and then a bunch of ifs to add it to my custom object array.
I was wondering if there is a simpler way...like with a template saying which columns from each file.
wound up doing this. It may have not been the most efficient, but works. I wound up writing out each file separately and combining at the end as PS really got bogged down (over a million rows combined).
$Newcsv = #()
$path = "c:\scrap\BWFILES\"
$files = gci -path $path -recurse -filter *.csv | Where-Object { ! ($_.psiscontainer) }
$counter=1
foreach($file in $files)
{
$csv = Import-Csv $file.FullName
if ($file.Name -like '*SAV*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"SV"}},DMBRCH,DMACCT,DMSHRT
}
if ($file.Name -like '*TIME*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"TM"}},TMBRCH,TMACCT,TMSHRT
}
if ($file.Name -like '*TRAN*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"TR"}},DMBRCH,DMACCT,DMSHRT
}
if ($file.Name -like '*LN*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"LN"}},LNBRCH,LNNOTE,LNSHRT
}
$Newcsv | Export-Csv "C:\scrap\$file.name$counter.csv" -force -notypeinformation
$counter++
}
get-childItem "c:\scrap\*.csv" | foreach {
$filePath = $_
$lines = $lines = Get-Content $filePath
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 1}
}
$getFirstLine = $false
Add-Content "c:\scrap\combined.csv" $linesToWrite
}
With a hashtable for reference, a little RegEx matching, and using the automatic variable $Matches in a ForEach-Object loop (alias % used) that could all be shortened to:
$path = "c:\scrap\BWFILES\"
$Reference = #{
'SAV' = 'SV'
'TIME' = 'TM'
'TRAN' = 'TR'
'LN'='LN'
}
Set-Content -Value "PRODUCT,BRCH,ACCT,SHRT" -Path 'c:\scrap\combined.csv'
gci -path $path -recurse -filter *.csv | Where-Object { !($_.psiscontainer) -and $_.Name -match ".*(SAV|TIME|TRAN|LN).*"}|%{
$Product = $Reference[($Matches[1])]
Import-CSV $_.FullName | Select-Object #{Name="PRODUCT";Expression={$Product}},*BRCH,#{l='Acct';e={$_.LNNOTE, $_.DMACCT, $_.TMACCT|?{$_}}},*SHRT | ConvertTo-Csv -NoTypeInformation | Select -Skip 1 | Add-Content 'c:\scrap\combined.csv'
}
That should produce the exact same file. Only kind of tricky part was the LNNOTE/TMACCT/DMACCT field since obviously you can't just do the same as like *SHRT.

Write parts from filename including the first line in merged txt / csv -file with powershell

I'm currently working on a PowerShell script that reads out the default printer on several workstations and write the information in a textfile to a network drive. My last question regarding some text replacements inside the script was successfully solved. But now I work on the second part.
Get-WmiObject -Class Win32_Printer -Filter "Default = $true" | % {
$_.Name -replace '(?:.*)\\NDPS-([^\.]+)(?:.*)', 'PS-$1'
} | Out-File -FilePath "H:\daten\printer\$($env:COMPUTERNAME)_defaultprinter.txt"
Get-WmiObject Win32_Printer -Filter "Default = $true" `
| Select-Object -expandProperty Name `
| Out-File -FilePath "P:\batch\migration\Printer\$($env:COMPUTERNAME)_$($env:USERNAME)_defaultprinter.txt"
The last line of the provided code writes the default printer to the network drive. Now I have there nearly 1500 single txt-files. For better analysis I use the following PowerShell script to merge all the single txt files to one big file.
Get-ChildItem -path \\samplepath\prgs\prgs\batch\migration\Printer -recurse | ? {
! $_.PSIsContainer
} | ? {
($_.name).contains(".txt")
} | % {
Out-File -filepath \\samplepath\prgs\prgs\batch\migration\Printer\gesamt_printer.txt -inputobject (get-content $_.fullname) -Append
}
I receive a file wich contains the default printer information from every txt-file but I need the $($env:USERNAME)-part from the filename as a separate value in addition to the printer information in on line to use the data in Excel. Can someone please provide me a tip how to insert the part from filename in the merged file?
You could extract the username part from the file name like this:
$_.Name -match '^.*?_(.*)_.*?\.txt$'
$username = $matches[1]
The group in the regular expression (accsisible via $matches[1]) contains the text between the first and the last underscore in the filename.
You could use it like this:
$root = "\\samplepath\prgs\prgs\batch\migration\Printer"
$outfile = "$root\gesamt_printer.txt"
Get-ChildItem $root -Recurse | ? {
-not $_.PSIsContainer -and $_.Extension -eq ".txt"
} | % {
$_.Name -match '^.*?_(.*)_.*?\.txt$'
$username = $matches[1]
$content = Get-Content $_.FullName
"{0},{1}" -f ($content, $username) | Out-File $outfile -Append
}
You could also directly create a CSV:
$root = "\\samplepath\prgs\prgs\batch\migration\Printer"
$outfile = "$root\gesamt_printer.txt"
$getPrinter = { Get-Content $_.FullName }
$getUser = { $_.Name -match '^.*?_(.*)_.*?\.txt$' | Out-Null; $matches[1] }
Get-ChildItem $root -Recurse `
| ? { -not $_.PSIsContainer -and $_.Extension -eq ".txt" } `
| select #{n="Username";e=$getUser},#{n="Printer";e=$getPrinter} `
| Export-Csv $outfile -NoTypeInformation
Note that these code sample don't contain any checks to exclude file names that don't have at least two underscores in them.