OutOfMemoryException when running my PowerShell script - powershell

Here's the PowerShell script I am using to add "segment99" to the beginning of all the text files (one by one) within a folder:
Set Environmental Variables:
$PathData = '<<ESB_Data_Share_HSH>>\RwdPnP'
Go to each text file in the specified folder and add header to the file:
Get-ChildItem $PathData -filter 'test_export.txt'|%{
$content = '"segment99" ' + [io.file]::ReadAllText($_.FullName)
[io.file]::WriteAllText(($_.FullName -replace '\.txt$','_99.txt'),$content)
}
This is giving me the following error:
Error: Exception calling "ReadAllText" with "1" argument(s): "Exception of type 'Syste
Error: m.OutOfMemoryException' was thrown."
Error: At D:\apps\MVPSI\JAMS\Agent\Temp\JAMSTemp13142.ps1:17 char:51
Error: + $content = '"segment99" ' + [io.file]::ReadAllText <<<< ($_.FullName)
Error: + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
Error: + FullyQualifiedErrorId : DotNetMethodException
Error:
I am running this code on a folder that has 20 files, each over 2 GB.
How can I fix this?

Copying a header file + a large file to a new file will be less prone to outofmemory exceptions (for files of that size):
$header = '"segment99"'
$header | out-file header.txt -encoding ASCII
$pathdata = "."
Get-ChildItem $PathData -filter 'test_export.txt' | %{
$newName = "{0}{1}{2}" -f $_.basename,"_99",$_.extension
$newPath = join-path (split-path $_.fullname) $newname
cmd /c copy /b "header.txt"+"$($_.fullname)" "$newpath"
}

This is not optimal code but it solves the task without reading all text to memory: it adds the header to the first line and then outputs other lines. Also, note that it does nothing if the input file is empty.
Get-ChildItem $PathData -Filter 'test_export.txt' | %{
$header = $true
Get-Content $_.FullName | .{process{
if ($header) {
'"segment99" ' + $_
$header = $false
}
else {
$_
}
}} | Set-Content ($_.FullName -replace '\.txt$', '_99.txt')
}

Related

how to append output to a CSV file

foreach ( $newfile in $file )
{
$b = Get-CMDeploymentStatus -PackageId $newfile -StatusType Any | select PackageID
Write-Output $b | Export-Csv -Path "C:\Users\PSM-6A1A000000000000\Documents\list.csv"
}
I am giving input to this with an input file which has number of package names listed and then I want to process it in such a way that the output comes one after the other right now I am getting an error as
Export-Csv : Cannot bind argument to parameter 'InputObject' because it is null. At line:16 char:20 + Write-Output $b | Export-Csv -Path "C:\Users\PSM-6A1A000000000000\Documents\lis ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidData: (:) [Export-Csv], ParameterBindingValidationException + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.ExportCsvCommand
Your code is assuming that you will have a result coming back from $b, if it does not though, you'll get an error because you're piping $b, which is null, into Export-CSV.
$null |export-csv c:\temp\1.csv
Export-Csv : Cannot bind argument to parameter 'InputObject' because it is null.
At line:1 char:8
+ $null |export-csv c:\temp\1.csv
+ ~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Export-Csv], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.ExportCs
You should add a 'Guard Clause' before you try to export.
if ($null -ne $b){
Export-csv -Append -InputObject $b
}
At least this will continue executing. Now your next problem is to determine why $b would be empty...from my experience with CM, I bet you need to specify which property in your $file you need. Maybe that line should read:
$b = Get-CMDeploymentStatus -PackageId $newfile.PackageId -StatusType Any | select PackageID
Since you say "I am giving input to this with an input file which has number of package names listed", but your code uses PackageId..
It looks to me that this file contains a packageId, each on a single line.
Anyway, I don't see the code ever reading this file..
If my assumption about the text file is correct, try:
# read the content of the text file and loop through the lines
# collect the output from Get-CMDeploymentStatus in variable $result
$result = Get-Content -Path 'X:\TheFileWithPackageIds.txt' | ForEach-Object {
# inside the ForEach-Object, the $_ automatic variable represents a single line from the text file
Get-CMDeploymentStatus -PackageId $_ -StatusType Any | select PackageID
}
# output on screen
$result
# write to new CSV file
$result | Export-Csv -Path "C:\Users\PSM-6A1A000000000000\Documents\list.csv" -NoTypeInformation

Powershell excluding only files

I have the following error when running a script in powershell in version 5.1
Below my simple script to delete files over 180 days, I've tried some solutions but I haven't found what may be the error.
(Below error translated from Portuguese to English)
"Out-File: It is not possible to validate the argument in the 'Encoding' parameter. The" files \ DeleteLogFile_21_01_2020.log "argument does not belong to the set" unknown; string; unicode; bigendianunicode; utf8; utf7; utf32; ascii; default; oem "specified by the ValidateSet attribute. Provide an argument that is in the set and try the command again. On the line: 1 character: 36 + $ Log | Out-File -Append D: \ Program files \ DeleteLogFile_ $ LogDate.log + ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ CategoryInfo: InvalidData: (:) [Out-File], ParameterBindingValidationException + FullyQualifiedErrorId: ParameterArgumentValidationError, Microsoft .PowerShell.Commands.OutFileCommand "
$logPath = "D:\Program files\NICE Systems\Logs"
$nDays = 10
$startTime = Get-Date
$Extensions = "*.*"
$LogDate = (Get-Date).ToString("dd_MM_yyyy")
$Files = Get-Childitem $LogPath -Include $Extensions -Recurse | Where-Object {$_.LastWriteTime -le (Get-Date).AddDays(-$nDays)}
foreach ($File in $Files)
{
if ($NULL -ne $File )
{
$Log = "`nData e Hora da Deleção: " + $startTime + ". O arquivo " + $File + " foi deletado do sistema."
$Log | Out-File -Append D:\Program files\DeleteLogFile_$LogDate.log
Remove-Item $File.FullName | out-null
}
}
You have no quotes around the path to your log file so it's cutting it off at the first space. You can use double quotes and the variable will still be expanded. Single will not be.
$Log | Out-File -Append "D:\Program files\DeleteLogFile_$LogDate.log"
By default windows will also not allow you to write to the program files folder and this is generally not good practice.
On your Out-File line, your path has a space in it and is not surrounded with quotation marks. In PowerShell, and most scripting languages for that matter, you have to take into account Positional Parameters. Any space that is not in a string or is not followed by/preceded by a parameter (eg. -FileName C:\FileName.txt), will be assumed to precede a positional parameter.
With that said, it is trying to use "D:\Program" as the path, and "files\DeleteLogFile_$LogDate.log" as the encoding type, which is obviously not valid. To fix this, simply make the path a string by putting it in quotation marks like so:
$Log | Out-File -Append "D:\Program files\DeleteLogFile_$LogDate.log"

Issue with foreach + get-childitem

I'm doing a script to organize my media. I download files in one directory to accomodate them before adding to my media center.
If for example I have a file called Breaking.Bad.S01E01.DVDRip.XviD-ORPHEUS.avi I would like the script to get the name of the show, check for season on S01 and move that file to a folder in another disk, for example e:\series\breaking bad\season 01
So far it checks if the file is call s01e01 or S01E01 or s01.e01 or S01.E01 and returns Breaking Bad\Season 01, creates the path to move to and the moving action itself
I have part of that script but I cannot make get-childitem to work with foreach.
This is what I have so far and the error I'm getting:
CODE
$FilesList = Get-ChildItem -name -recurse -include *.mkv,*.mp4,*.srt,*.avi
$FilesList
foreach ($FL_Item in $FilesList)
{
$SeriesName = ($FL_Item.BaseName -split '\.s\d')[0].Replace('.', ' ')
$SE_Info = $FL_Item.BaseName.Split('.')[-3] -split 'e'`
$Season = $SE_Info[0] -replace 's', 'Season '
#$Episode = 'Episode{0}' -f $SE_Info[1]
$SeriesName
$Season
#$Episode
$SeriesDirectory = Join-Path -Path "$SeriesName" -ChildPath "$Season"
$SeriesDirectory
#$MoverArchivo = move-item -path $FileName -destination e:\series\$SeriesDirectory
#$MoverArchivo
''
}
OUTPUT I'm getting
Breaking.Bad.S01E01.DVDRip.XviD-ORPHEUS.avi
Breaking.Bad.S01E01.DVDRip.XviD-ORPHEUS.spa.srt
Breaking.Bad.S04E01.Box.Cutter.720p.hdtv.x264-orenji.mkv
Breaking.Bad.S04E01.Box.Cutter.720p.hdtv.x264-orenji.spa.srt
Breaking.Bad.S05E15.720p.HDTV.x264-EVOLVE.mkv
Breaking.Bad.S05E15.720p.HDTV.x264-EVOLVE.spa.srt
Path Of Blood (2018) [WEBRip] [1080p] [YTS.AM]\Path.Of.Blood.2018.1080p.WEBRip.x264-[YTS.AM].mp4
They Shall Not Grow Old (2018) [BluRay] [1080p] [YTS.AM]\They.Shall.Not.Grow.Old.2018.1080p.BluRay.x264-[YTS.AM].mp4
ERROR
You cannot call a method on a null-valued expression.
At D:\shared\temp\test3.ps1:8 char:5
+ $SE_Info = $FL_Item.BaseName.Split('.')[-3] -split 'e'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
Cannot index into a null array.
At D:\shared\temp\test3.ps1:10 char:5
+ $Season = $SE_Info[0] -replace 's', 'Season '
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray
Join-Path : Cannot bind argument to parameter 'Path' because it is an empty string.
At D:\shared\temp\test3.ps1:17 char:37
+ $SeriesDirectory = Join-Path -Path "$SeriesName" -ChildPath "$Sea ...
+ ~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Join-Path], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Microsoft.PowerShell.Commands.Join
PathCommand
Any ideas what could be wrong?
You are using the -name parameter of Get-ChildItem in your first line:
$FilesList = Get-ChildItem -name -recurse -include *.mkv,*.mp4,*.srt,*.avi
which means it will return just the names of the files as strings.
Later, in your loop, you access each element using the BaseName property, which is a property of FileInfo objects, not strings. So, $FL_Item.BaseName returns an empty string and you get the errors as shown.
Just remove the -name and it should work (or at least you won't get those errors).
I'd use a RegEx with (named) capture groups to grep the Series,Season and Episode number.
See the RegEx working live on regex101.com
## Q:\Test\2018\12\20\SO_53875674.ps1
$DstBase = "E:\series"
Get-ChildItem -Include *.mkv,*.mp4,*.srt,*.avi -Recurse -File|
Where-Object BaseName -match "^(?<Series>.*?)\.?S(?<Season>\d{1,2})\.?E(?<Episode>\d{2})"|
ForEach-Object {
$Destination = "{0}\{1}\Season {2:00}\" -f $DstBase,$Matches.Series.replace('.',' ').Trim(),[int]$Matches.Season
if (!(Test-Path $Destination)){MD $Destination -Force | Out-Null}
"Moving file [{0}] to [{1}]" -f $_.FullName,$Destination
$_ | Move-Item -Destination $Destination -Force
}
Sample tree (with above data) after running the script:
> tree /F
└───series
└───Breaking Bad
├───Season 01
│ Breaking.Bad.S01E01.DVDRip.XviD-ORPHEUS.avi
│ Breaking.Bad.S01E01.DVDRip.XviD-ORPHEUS.spa.srt
│
├───Season 04
│ Breaking.Bad.S04E01.Box.Cutter.720p.hdtv.x264-orenji.mkv
│ Breaking.Bad.S04E01.Box.Cutter.720p.hdtv.x264-orenji.spa.srt
│
└───Season 05
Breaking.Bad.S05E15.720p.HDTV.x264-EVOLVE.mkv
Breaking.Bad.S05E15.720p.HDTV.x264-EVOLVE.spa.srt

unable to fetch system.int64 registry datavalue in powershell

Below powershell code is for exporting the complete registry hive to CSV file. But it gives error for exporting REG_Binary type data whose value is big as shown in the attached registry screenshot. Though I tried to convert it to string but no luck. Please suggest on how to fetch the REG_Binary data or how to convert it to string.
Function Get_Reg_Keys ($reg_key) {
$actualKey = Get-Item -Path Registry::$reg_key
#Write-Host $actualKey
ForEach ($name in $actualKey.GetValueNames()) {
# Write-Host $name
$name.Property | foreach {
$type = $actualKey.GetValue($name).GetType()
#Write-Host $type
Write-Host $actualKey.Name " | " $name " | " $actualKey.GetValue($name) " | " $type
If ( $type -eq [System.Byte[]] -Or $type -eq [System.Int64[]]) {
$a = [System.BitConverter]::ToString($actualKey.GetValue($name))
( '"{0:0}","{1:0}", "{2:0}"' -f $actualKey.Name, $name, $a) |
Out-File "C:\Temp\Automation\HKCU.csv" -append -Encoding ascii -erroraction SilentlyContinue
}
Else {
( '"{0:0}","{1:0}", "{2:0}"' -f $actualKey.Name, $name, $actualKey.GetValue($name)) |
Out-File "C:\Temp\Automation\HKCU.csv" -append -Encoding ascii -erroraction SilentlyContinue
}
}
}
$keyinfo = Get-ChildItem Registry::$reg_key
if ($keyinfo.count -gt 0) {
ForEach ($keyitem in $keyinfo) { Get_Reg_Keys ($keyitem) }
}
}
Then running: Get_Reg_Keys ("HKEY_CURRENT_USER\")
Gives the error:
Cannot convert argument "index", with value: "MRUListEx", for "GetValue" to type "System.Int64": "Cannot convert value "MRUListEx" to type "System.Int64". Error: "Input string was
not in a correct format.""
At C:\Temp\Automation\HKCUregistrynew.ps1:27 char:21
+ ( '"{0:0}","{1:0}", "{2:0}"' -f $actualKey.Name, $name, $act ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodException
+ FullyQualifiedErrorId : MethodArgumentConversionInvalidCastArgument
Registry Key details:
To be honest, I'm having a tough time following your function. I'm not even sure how you're iterating over values in a key with that $name.properties | foreach... loop.
But, generally, this is pretty straight forward. For example, I've got a registry key HKCU:\Environment. I've created a property in that key named "thing", and I've stuffed it with some random binary data.
I'll take the guts of that property and output it to a file:
$reg_key = 'HKCU:\Environment'
$key_properties = Get-Item -Path $reg_key
[System.BitConverter]::ToString($key_properties.GetValue('thing')) | Out-File -FilePath .\Test.txt
cat .\Test.txt
Just to show there is nothing up my sleeve:
Good luck.
After making below changes I am able to export a registry hive to CSV file:
$actualKey = Get-Item -LiteralPath Registry::$reg_key
$keyinfo = Get-ChildItem -LiteralPath Registry::$reg_key

Search for a string in a number of files, and then rename the files with that string

I'm trying to create a PowerShell script that will search through a series of .txt files (.hl7 files to be exact, but they are just txt files) and search within those files to see if they contain a four digit number. If the file does contain that four digit number, it should then rename the file with the string added to the front of the original file name. So test.hl7 should become 8000_test.hl7 if that file includes those 4 digits within it.
After a day of ferocious googling and digging through this website, this is the best I could muster:
$AccountIDs = ("8155", "8156", "8428")
$Path = "C:\Users\ThatsMe\Downloads\messages"
$Files = (Get-ChildItem $Path -Filter "*.hl7")
for ($i = 0; $i -le $Files.Length; $i++) {
if (Get-Content $Files[$i].FullName | Select-String -Pattern $AccountIDs[$i]) {
Rename-Item $Files[$i].FullName -NewName (($AccountIDs[$i]) + "_" + $Files[$i].Name)
}
}
I am getting some interesting results. I currently have four test files in that messages folder, test, test2, test3, and skibbidybop. The very first one, test gets correctly changed to 8156_test. However, the other files aren't touched. Now, when I change the filename of test to ttest, the script completely skips over that file and then renames test2 and test3 to 8156_test2 (which is incorrect) and 8428_test3 respectively. skibbidybop is never touched.
And, of course, the error message from PowerShell:
Select-String : Cannot bind argument to parameter 'Pattern' because it is null.
At line:6 char:61
+ if (Get-Content $Files[$i].FullName | Select-String -Pattern <<<< $AccountIDs[$i]) {
+ CategoryInfo : InvalidData: (:) [Select-String], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.SelectStringCommand
Get-Content : Cannot bind argument to parameter 'Path' because it is null.
At line:6 char:16
+ if (Get-Content <<<< $Files[$i].FullName | Select-String -Pattern $AccountIDs[$i]) {
+ CategoryInfo : InvalidData: (:) [Get-Content], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.GetContentCommand
Updated Code
$Path = "C:\Users\ThatsMe\Downloads\messages"
$pattern = '\b(8155|8156|8428)\b'
Get-ChildItem $Path -Filter '*.hl7' |
Select-String -Pattern $pattern |
Group-Object Path |
ForEach-Object {
$id = $_.Group.Matches[0].Groups[0].Value
$filename = $_.Group.Filename | Select-Object -First 1
Rename-Item -Path $_.Name -NewName "${id}_${filename}" -WhatIf
}
This is the error that I receive now:
C:\> C:\Users\ThatsMe\Downloads\messages\changename.ps1
Cannot index into a null array.
At C:\Users\ThatsMe\Downloads\messages\changename.ps1:8 char:38
+ $id = $_.Group.Matches[ <<<< 0].Groups[0].Value
+ CategoryInfo : InvalidOperation: (0:Int32) [], RuntimeException
+ FullyQualifiedErrorId : NullArray
What if: Performing operation "Rename File" on Target "Item:
C:\Users\ThatsMe\Downloads\messages\test.hl7 Destination:
C:\Users\ThatsMe\Downloads\messages\_".
Cannot index into a null array.
At C:\Users\ThatsMe\Downloads\messages\changename.ps1:8 char:38
+ $id = $_.Group.Matches[ <<<< 0].Groups[0].Value
+ CategoryInfo : InvalidOperation: (0:Int32) [], RuntimeException
+ FullyQualifiedErrorId : NullArray
What if: Performing operation "Rename File" on Target "Item:
C:\Users\ThatsMe\Downloads\messages\test3.hl7 Destination:
C:\Users\ThatsMe\Downloads\messages\_".
The errors you get are caused by two mistakes, one of them a classic off-by-one error. PowerShell arrays are zero-based, meaning that the last index of the array is one less than the number of its elements:
[ 'a', 'b', 'c' ] → count == 3
0 1 2 → last index == 2 == 3-1
Thus your for loop may run while $i is less than $Files.Length (-lt), not less or equal (-le):
for ($i = 0; $i -lt $Files.Length; $i++) {
Also, you cannot use the same index variable for two different arrays ($Files and $AccountIDs) unless you made sure both arrays have the same length or at least that the second array ($AccountIDs) has more elements than the one used to determine the maximum index ($Files). If $AccountIDs has less elements than $Files your code will eventually attempt to access an index beyond the upper boundary of $AccountIDs. Besides, you probably want to check each file for all of the numbers from $AccountIDs anyway. Doing that requires a nested loop with a second index variable.
With that said, you're making this more complicated than it needs to be. You can simply put your IDs in a single regular expression and pipe the list of files into Select-String to check them against that regular expression:
$pattern = '\b(8155|8156|8428)\b'
Get-ChildItem $Path -Filter '*.hl7' |
Select-String -Pattern $pattern |
Group-Object Path |
ForEach-Object {
$id = $_.Group.Matches[0].Groups[0].Value
$filename = $_.Group.Filename | Select-Object -First 1
Rename-Item -Path $_.Name -NewName "${id}_${filename}" -WhatIf
}
The regular expression \b(8155|8156|8428)\b matches any of the given numbers. The \b restrict the match to word boundaries to avoid matching numbers like 81552 or 842893 as well.
The Group-Object statement ensures the uniqueness of the renamed files (so that you don't attempt to rename a file more than once if more than one match is found in it).
.Matches[0].Groups[0].Value extracts the value of the first capturing group of the first match for each file.
The Select-Object -First 1 ensures that even if multiple matches were found in a file you have just one string with the filename, not an array of them.
Remove the -WhatIf switch once you verified that the rename operation would work correctly and re-run the whole statement to actually rename the files.
Edit: For PowerShell v2 you need to adjust the group handling a little bit, because that version doesn't support member enumeration.
Get-ChildItem $Path -Filter '*.hl7' |
Select-String -Pattern $pattern |
Group-Object Path |
ForEach-Object {
$id = $_.Group | Select-Object -Expand Matches -First 1 |
ForEach-Object { $_.Groups[0].Value }
$filename = $_.Group | Select-Object -Expand Filename -First 1
Rename-Item -Path $_.Name -NewName "${id}_${filename}" -WhatIf
}