PowerShell to Split multiple text file in the folder to an output folder according 10000 lines with Added Text Each Files - powershell

Question

#Path // CHANGE FILE NAME AND RUN = 'C:\BassBeta\3.sql'
#$SQLFilePath = 'C:\Users\paul.ang\Desktop\UOS\UOS_Discovery\rev\test\*.*'
Get-ChildItem C:\TEST -include *.sql -rec | ForEach-Object {gc $_; ""} | out-file C:\test\test.txt
#CHECKING IF >5000 LINES
$fileContent = Get-Content $SQLFilePath
$Lines = (Get-content $SQLFilePath | Measure-Object –Line).Lines
#$lineNumber = $Lines
IF ($Lines -ge 5000)
{
#Header /OK
$headertxt = 'BEGIN TRAN'
$($headertxt; Get-Content $SQLFilePath) | Set-Content $SQLFilePath
#Body /OK
$textToAdd_B = '
COMMIT TRAN
--ROLLBACK
BEGIN TRAN'
$lineNumber_B = '5002'
$fileContent = Get-Content $SQLFilePath
$fileContent[$lineNumber_B-1] += $textToAdd_B
$fileContent | Set-Content $SQLFilePath
#Footer /OK
$textToAdd_F = '
COMMIT TRAN
--ROLLBACK'
$fileContent = Get-Content $SQLFilePath
$Lines = (Get-content $SQLFilePath | Measure-Object –Line).Lines
$lineNumber_F = $Lines
$fileContent[$lineNumber_F] += $textToAdd_F
$fileContent | Set-Content $SQLFilePath
}
ELSEIF ($Lines -le 5000)
{
#Header /OK
$headertxt = 'BEGIN TRAN'
$($headertxt; Get-Content $SQLFilePath) | Set-Content $SQLFilePath
#Footer /OK
$textToAdd_F = '
COMMIT TRAN
--ROLLBACK'
$fileContent = Get-Content $SQLFilePath
$Lines = (Get-content $SQLFilePath | Measure-Object –Line).Lines
$lineNumber_F = $Lines
$fileContent[$lineNumber_F-1] += $textToAdd_F
$fileContent | Set-Content $SQLFilePath
}
'Hi all experts, First of all I would like to say thank you and sorry if there is duplicated post in there as I not really active in Stack flow. Recently, My company assigned me a task to split sql files and deliver to production team. I hope any expert here can guide me there My requirement as below:- I would like to split multiple text files in the input folder according my setting 10000 lines each text file and after split 10000 lines each file add text 'BEGIN TRAN' to the text file then save it in the output folder. Since this task is very high frequency and previously I run it manually with stress =.=". Somehow I copy source from the internet and hope can mix and match to make it work. I'm no expert to PowerShell but would like to enhance my work and make my work life balance. TQ'

Answer 1

I test following code after creating a test file with this code which puts sequential number in a file
$writer = [System.IO.StreamWriter]::new("c:\temp\test\bigFile.sql")
for($i = 0; $i -lt 1000000; $i++)
{
$writer.WriteLine($i)
}
$writer.Flush()
$writer.Close()
Than used following to split the large file
#Path // CHANGE FILE NAME AND RUN = 'C:\BassBeta\3.sql'
#$SQLFilePath = 'C:\Users\paul.ang\Desktop\UOS\UOS_Discovery\rev\test\*.*'
$basefolder = "c:\temp\test"
$headertxt = 'BEGIN TRAN'
$fileSize = 5000
$files = Get-ChildItem $basefolder -include *.sql -rec
foreach($file in $files)
{
$reader = [System.IO.StreamReader]::new($file.fullname)
$lines = [System.Collections.ArrayList]::new()
Write-Host "Start Read = " $file.FullName
while(($line = $reader.ReadLine()) -ne $null)
{
$lines.Add($line) | Out-Null;
}
Write-Host "End Read = " $file.FullName
if($lines.Count -ge $fileSize)
{
#remove sql from filename
$lastIndex = $file.fullname.LastIndexOf('.')
$folder = $file.fullname.substring(0, $lastIndex)
Write-Host "folder name = " $folder
if(!(Test-Path $folder))
{
Write-Host "create new folder = " $folder
New-Item $folder -ItemType Directory
$fileCount = 0
for($i = 0; $i -lt $lines.Count; $i += $fileSize)
{
++$fileCount
$newFilename = $folder + "\" + $fileCount + ".sql"
Write-Host "new Filename = " $newFilename
$writer = [System.IO.StreamWriter]::new($newFilename)
$writer.Writeline($headertxt)
for($j = $i; ($j -lt $lines.Count) -and ($j -lt ($i + $fileSize)); $j++)
{
$writer.WriteLine($lines[$j])
}
$writer.Flush()
$writer.Close()
}
}
else
{
Write-Host "Folder Exists" $folder
}
}
}