How can subtract a character from csv using PowerShell - powershell

I'm trying to insert my CSV into my SQL Server database but just wondering how can I subtract the last three character from CSV GID column and then assigned it to my $CSVHold1 variable.
My CSV file look like this
GID Source Type Message Time
KLEMOE http://google.com Od Hello 12/22/2022
EEINGJ http://facebook.com Od hey 12/22/2022
Basically I'm trying to get only the first three character from GID and pass that value to my $CSVHold1 variable.
$CSVImport = Import-CSV $Global:ErrorReport
ForEach ($CSVLine1 in $CSVImport) {
$CSVHold1 = $CSVLine1.GID | ForEach-Object { $_.$GID = $_.$GID.subString(0, $_.$GID.Length - 3); $_ }
$CSVGID1 = $CSVLine1.GID
$CSVSource1 = $CSVLine1.Source
$CSVTYPE1 = $CSVLine1.TYPE
$CSVMessage1 = $CSVLine1.Message
}
I'm trying to do like above but some reason I'm getting an error.
You cannot call a method on a null-valued expression.

Your original line 3 was/is not valid syntax as Santiago pointed out.
$CSVHold1 = $CSVLine1.GID | ForEach-Object { $_.$GID = $_.$GID.subString(0, $_.$GID.Length - 3); $_ }
You are calling $_.$GID but you're wanting $_.GID
You also don't need to pipe the object into a loop to achieve what it seems you are asking.
#!/usr/bin/env powershell
$csvimport = Import-Csv -Path $env:HOMEDRIVE\Powershell\TestCSVs\test1.csv
##$CSVImport = Import-CSV $Global:ErrorReport
ForEach ($CSVLine1 in $CSVImport) {
$CSVHold1 = $CSVLine1.GID.SubString(0, $CSVLine1.GID.Length - 3)
$CSVGID1 = $CSVLine1.GID
$CSVSource1 = $CSVLine1.Source
$CSVTYPE1 = $CSVLine1.TYPE
$CSVMessage1 = $CSVLine1.Message
Write-Output -InputObject ('Changing {0} to {1}' -f $CSVLine1.gid, $CSVHold1)
}
Using your sample data, the above outputs:
C:> . 'C:\Powershell\Scripts\dchero.ps1'
Changing KLEMOE to KLE
Changing EEINGJ to EEI
Lastly, be aware that that the SubString method will fail if the length of $CSVLine1.GID is less than 3.

Related

Parsing a log file - find lines which contain a value from array of strings

this is probably really simple... but how do I check which lines of a log/config file contain any of the strings from a string array?
Example of file (I want lines 3 and 5)
[Privilege Rights]
SeSystemtimePrivilege = *S-1-5-19,*S-1-5-21-2262136377-125853592-2400401627-1119,*S-1-5-32-544
SeDenyNetworkLogonRight = Guest
SeCreatePagefilePrivilege = *S-1-5-32-544
SeDenyServiceLogonRight = *S-1-5-21-2262136377-125853592-2400401627-1119
SeRemoteShutdownPrivilege = *S-1-5-32-544
SeAuditPrivilege = *S-1-5-19,*S-1-5-20
Script I am using:
$SeRightsArray = #(
'SeDenyNetworkLogonRight',
'SeDenyBatchLogonRight',
'SeDenyServiceLogonRight',
'SeDenyInteractiveLogonRight',
'SeDenyRemoteInteractiveLogonRight')
$LocalPolicyExport = gc "C:\local-security-policy.inf"
Foreach($Line in $LocalPolicyExport){
if ($Line -match $SeRightsArray ){
write-host "FOUND - $line"
}
}
Normally I would use something like -contains or -in, but that doesn't work
You can use gc "C:\local-security-policy.inf" | select-string $seRightsArray. No need for a loop
PS C:\> gc .\test.txt|select-string $SeRightsArray
SeDenyNetworkLogonRight = Guest
SeDenyServiceLogonRight = *S-1-5-21-2262136377-125853592-2400401627-1119

Windows PowerShell: How to parse the log file?

I have an input file with below contents:
27/08/2020 02:47:37.365 (-0516) hostname12 ult_licesrv ULT 5 LiceSrv Main[108 00000 Session 'session1' (from 'vmpms1\app1#pmc21app20.pm.com') request for 1 additional licenses for module 'SA-XT' - 1 licenses have been allocated by concurrent usage category 'Unlimited' (session module usage now 1, session category usage now 1, total module concurrent usage now 1, total category usage now 1)
27/08/2020 02:47:37.600 (-0516) hostname13 ult_licesrv ULT 5 LiceSrv Main[108 00000 Session 'sssion2' (from 'vmpms2\app1#pmc21app20.pm.com') request for 1 additional licenses for module 'SA-XT-Read' - 1 licenses have been allocated by concurrent usage category 'Floating' (session module usage now 2, session category usage now 2, total module concurrent usage now 1, total category usage now 1)
27/08/2020 02:47:37.115 (-0516) hostname141 ult_licesrv CMN 5 Logging Housekee 00000 Deleting old log file 'C:\Program Files\PMCOM Global\License Server\diag_ult_licesrv_20200824_011130.log.gz' as it exceeds the purge threashold of 72 hours
27/08/2020 02:47:37.115 (-0516) hostname141 ult_licesrv CMN 5 Logging Housekee 00000 Deleting old log file 'C:\Program Files\PMCOM Global\License Server\diag_ult_licesrv_20200824_021310.log.gz' as it exceeds the purge threashold of 72 hours
27/08/2020 02:47:37.625 (-0516) hostname150 ult_licesrv ULT 5 LiceSrv Main[108 00000 Session 'session1' (from 'vmpms1\app1#pmc21app20.pm.com') request for 1 additional licenses for module 'SA-XT' - 1 licenses have been allocated by concurrent usage category 'Unlimited' (session module usage now 2, session category usage now 1, total module concurrent usage now 2, total category usage now 1)
I need to generate and output file like below:
Date,time,hostname,session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage
27/08/2020,02:47:37.365 (-0516),hostname12,1,1,1,1
27/08/2020,02:47:37.600 (-0516),hostname13,2,2,1,1
27/08/2020,02:47:37.115 (-0516),hostname141,0,0,0,0
27/08/2020,02:47:37.115 (-0516),hostname141,0,0,0,0
27/08/2020,02:47:37.625 (-0516),hostname150,2,1,2,1
The output data order is: Date,time,hostname,session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage.
Put 0,0,0,0 if no entry for session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage
I need to get content from the input file and write the output to another file.
Update
I have created a file input.txt in F drive and pasted the log details into it.
Then I form an array by splitting the file content when a new line occurs like below.
$myList = (Get-Content -Path F:\input.txt) -split '\n'
Now I got 5 items in my array myList. Then I replace the multiple blank spaces with a single blank space and formed a new array by splitting each element by blank space. Then I print the 0 to 3 array elements. Now I need to add the end values (session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage).
PS C:\Users\user> $myList = (Get-Content -Path F:\input.txt) -split '\n'
PS C:\Users\user> $myList.Length
5
PS C:\Users\user> $myList = (Get-Content -Path F:\input.txt) -split '\n'
PS C:\Users\user> $myList.Length
5
PS C:\Users\user> for ($i = 0; $i -le ($myList.length - 1); $i += 1) {
>> $newList = ($myList[$i] -replace '\s+', ' ') -split ' '
>> $newList[0]+','+$newList[1]+' '+$newList[2]+','+$newList[3]
>> }
27/08/2020,02:47:37.365 (-0516),hostname12
27/08/2020,02:47:37.600 (-0516),hostname13
27/08/2020,02:47:37.115 (-0516),hostname141
27/08/2020,02:47:37.115 (-0516),hostname141
27/08/2020,02:47:37.625 (-0516),hostname150
If you really need to filter on the granularity that you're looking for, then you may need to use regex to filter the lines.
This would assume that the rows have similarly labeled lines before the values you're looking for, so keep that in mind.
[System.Collections.ArrayList]$filteredRows = #()
$log = Get-Content -Path C:\logfile.log
foreach ($row in $log) {
$rowIndex = $log.IndexOf($row)
$date = ([regex]::Match($log[$rowIndex],'^\d+\/\d+\/\d+')).value
$time = ([regex]::Match($log[$rowIndex],'\d+:\d+:\d+\.\d+\s\(\S+\)')).value
$hostname = ([regex]::Match($log[$rowIndex],'(?<=\d\d\d\d\) )\w+')).value
$sessionModuleUsage = ([regex]::Match($log[$rowIndex],'(?<=session module usage now )\d')).value
if (!$sessionModuleUsage) {
$sessionModuleUsage = 0
}
$sessionCategoryUsage = ([regex]::Match($log[$rowIndex],'(?<=session category usage now )\d')).value
if (!$sessionCategoryUsage) {
$sessionCategoryUsage = 0
}
$moduleConcurrentUsage = ([regex]::Match($log[$rowIndex],'(?<=total module concurrent usage now )\d')).value
if (!$moduleConcurrentUsage) {
$moduleConcurrentUsage = 0
}
$totalCategoryUsage = ([regex]::Match($log[$rowIndex],'(?<=total category usage now )\d')).value
if (!$totalCategoryUsage) {
$totalCategoryUsage = 0
}
$hash = [ordered]#{
Date = $date
time = $time
hostname = $hostname
session_module_usage = $sessionModuleUsage
session_category_usage = $sessionCategoryUsage
module_concurrent_usage = $moduleConcurrentUsage
total_category_usage = $totalCategoryUsage
}
$rowData = New-Object -TypeName 'psobject' -Property $hash
$filteredRows.Add($rowData) > $null
}
$csv = $filteredRows | convertto-csv -NoTypeInformation -Delimiter "," | foreach {$_ -replace '"',''}
$csv | Out-File C:\results.csv
What essentially needs to happen is that we need to get-content of the log, which returns an array with each item terminated on a newline.
Once we have the rows, we need to grab the values via regex
Since you want zeroes in some of the items if those values don't exist, I have if statements that assign '0' if the regex returns nothing
Finally, we add each filtered item to a PSObject and append that object to an array of objects in each iteration.
Then export to a CSV.
You can probably pick apart the lines with a regex and substrings easily enough. Basically something like the following:
# Iterate over the lines of the input file
Get-Content F:\input.txt |
ForEach-Object {
# Extract the individual fields
$Date = $_.Substring(0, 10)
$Time = $_.Substring(12, $_.IndexOf(')') - 11)
$Hostname = $_.Substring(34, $_.IndexOf(' ', 34) - 34)
$session_module_usage = 0
$session_category_usage = 0
$module_concurrent_usage = 0
$total_category_usage = 0
if ($_ -match 'session module usage now (\d+), session category usage now (\d+), total module concurrent usage now (\d+), total category usage now (\d+)') {
$session_module_usage = $Matches[1]
$session_category_usage = $Matches[2]
$module_concurrent_usage = $Matches[3]
$total_category_usage = $Matches[4]
}
# Create custom object with those properties
New-Object PSObject -Property #{
Date = $Date
time = $Time
hostname = $Hostname
session_module_usage = $session_module_usage
session_category_usage = $session_category_usage
module_concurrent_usage = $module_concurrent_usage
total_category_usage = $total_category_usage
}
} |
# Ensure column order in output
Select-Object Date,time,hostname,session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage |
# Write as CSV - without quotes
ConvertTo-Csv -NoTypeInformation |
ForEach-Object { $_ -replace '"' } |
Out-File F:\output.csv
Whether to pull the date, time, and host name from the line with substrings or regex is probably a matter of taste. Same goes for how strict the format must be matched, but that to me mostly depends on how rigid the format is. For more free-form things where different lines would match different regexes, or multiple lines makes up a single record, I also quite like switch -Regex to iterate over the lines.

is there a simple way to output to xlsx?

I am trying to output a query from a DB to a xlsx but it takes so much time to do this because there about 20,000 records to process, is there a simpler way to do this?
I know there is a way to do it for csv but im trying to avoid that, because if the records had any comma is going to take it as a another column and that would mess with the info
this is my code
$xlsObj = New-Object -ComObject Excel.Application
$xlsObj.DisplayAlerts = $false
$xlsWb = $xlsobj.Workbooks.Add(1)
$xlsObj.Visible = 0 #(visible = 1 / 0 no visible)
$xlsSh = $xlsWb.Worksheets.Add([System.Reflection.Missing]::Value, $xlsWb.Worksheets.Item($xlsWb.Worksheets.Count))
$xlsSh.Name = "QueryResults"
$DataSetTable= $ds.Tables[0]
Write-Output "DATA SET TABLE" $DataSetTable
[Array] $getColumnNames = $DataSetTable.Columns | SELECT *
Write-Output "COLUMN NAMES" $DataSetTable.Rows[0]
[Int] $RowHeader = 1
foreach ($ColH in $getColumnNames)
{
$xlsSh.Cells.item(1, $RowHeader).font.bold = $true
$xlsSh.Cells.item(1, $RowHeader) = $ColH.ColumnName
Write-Output "Nombre de Columna"$ColH.ColumnName
$RowHeader++
}
[Int] $rowData = 2
[Int] $colData = 1
foreach ($rec in $DataSetTable.Rows)
{
foreach ($Coln in $getColumnNames)
{
$xlsSh.Cells.NumberFormat = "#"
$xlsSh.Cells.Item($rowData, $colData) = $rec.$($Coln.ColumnName).ToString()
$ColData++
}
$rowData++; $ColData = 1
}
$xlsRng = $xlsSH.usedRange
[void] $xlsRng.EntireColumn.AutoFit()
#Se elimina la pestaña Sheet1/Hoja1.
$xlsWb.Sheets(1).Delete() #Versión 02
$xlsFile = "directory of the file"
[void] $xlsObj.ActiveWorkbook.SaveAs($xlsFile)
$xlsObj.Quit()
Start-Sleep -Milliseconds 700
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsRng)) {''}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsSh)) {''}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsWb)) {''}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsObj)) {''}
[gc]::collect() | Out-Null
[gc]::WaitForPendingFinalizers() | Out-Null
$oraConn.Close()
I'm trying to avoid [CSV files], because if the records had any comma is going to take it as a another column and that would mess with the info
That's only the case if you try to construct the output format manually. Builtin commands like Export-Csv and ConvertTo-Json will automatically quote the values as necessary:
PS C:\> $customObject = [pscustomobject]#{ID = 1; Name = "Solis, Heber"}
PS C:\> $customObject
ID Name
-- ----
1 Solis, Heber
PS C:\> $customObject |ConvertTo-Csv -NoTypeInformation
"ID","Name"
"1","Solis, Heber"
Notice, in the example above, how:
The string value assigned to $customObject.Name does not contain any quotation marks, but
In the output from ConvertTo-Csv we see values and headers clearly enclosed in quotation marks
PowerShell automatically enumerates the row data when you pipe a [DataTable] instance, so creating a CSV might (depending on the contents) be as simple as:
$ds.Tables[0] |Export-Csv table_out.csv -NoTypeInformation
What if you want TAB-separated values (or any other non-comma separator)?
The *-Csv commands come with a -Delimiter parameter to which you can pass a user-defined separator:
# This produces semicolon-separated values
$data |Export-Csv -Path output.csv -Delimiter ';'
I usually try and refrain from recommending specific modules libraries, but if you insist on writing to XSLX I'd suggest checking out ImportExcel (don't let the name fool you, it does more than import from excel, including exporting and formatting data from PowerShell -> XSLX)

How to properly string replace in Powershell without appending the replaced variable to a newline?

I'm pretty new to powershell/programming so bear with me. I have this bug that appends the new renamed path to a new-line without the rest of path.
The console output:
/content/pizza/en/ingredients/
helloworld/menu-eng.html
What I want:
/content/pizza/en/ingredients/helloworld/menu-eng.html
What the code below is supposed to do is rename a bunch paths. Right now testName is hard-coded but after I get this to work properly it will be dynamic.
My code:
$testName = "helloworld"
$text = (Get-Content W:\test\Rename\rename.csv) | Out-String
$listOfUri = Import-Csv W:\test\Rename\rename.csv
foreach ($element in $listOfUri) {
if ($element -match "menu-eng.html") {
$elementString = $element.'ColumnTitle' | Out-String
$elementString = $elementString.Replace('menu-eng.html', '')
$varPath1 = $elementString
$elementString = $elementString.Insert('', 'http://www.pizza.com')
$elementName = ([System.Uri]$elementString).Segments[-1]
$elementString = $elementString.Replace($elementName, '')
$elementString = $elementString.Replace('http://www.pizza.com', '')
$varPath2 = $elementString.Insert($elementString.Length, $testName + '/')
$text = $text.Replace($varPath1.Trim(), $varPath2)
}
}
$text
Assuming your .csv file looks like this:
ColumnTitle,Idk
/content/pizza/en/ingredients/SPAM/menu-eng.html,Stuff
Then:
$testName = 'helloworld'
foreach ($row in Import-CSV d:\rename.csv) {
$bit = $row.'ColumnTitle'.Split('/')[-2]
$row.'ColumnTitle'.replace($bit, $testName)
}
I have no real idea what all the rest of your code is for, particularly my earlier comment, your line:
$text = (Get-Content W:\test\Rename\rename.csv) | Out-String
is making $text into an /array/ of all the lines in the file, including the headers. You can still use .Replace() on it in PowerShell, but it's going to do the replace on every line. I can't quite see how that gives you the output you get, but it will give you multiple lines for every line in the input file.

Powershell get repadmin /istg in array?

Is this possible?
I'm brand new to powershell and am currently in the process of converting a vbscript script to Powershell. The following one-liner command seems to do exactly what the entire vbscript does:
Repadmin /istg
which outputs
Repadmin: running command /istg against full DC ST-DC7.somestuff.com
Gathering topology from site BR-CORP (ST-DC7.somestuff.com):
Site ISTG
================== =================
Portland ST-DC4
Venyu ST-DC5
BR-Office ST-DC3
BR-CORP ST-DC7
The problem is I need to return this info (namely the last 4 lines) as objects which contain a "Site" and "ISTG" field. I tried the following:
$returnValues = Repadmin /istg
$returnValues
But this didin't return anything (possibly because Repadmin writes out the lines instead of actually returning the data?)
Is there a way to get the Info from "Repadmin /istg" into an array?
Here's one possible way, using regular expressions:
$output = repadmin /istg
for ( $n = 10; $n -lt $output.Count; $n++ ) {
if ( $output[$n] -ne "" ) {
$output[$n] | select-string '\s*(\S*)\s*(\S*)$' | foreach-object {
$site = $_.Matches[0].Groups[1].Value
$istg = $_.Matches[0].Groups[2].Value
}
new-object PSObject -property #{
"Site" = $site
"ISTG" = $istg
} | select-object Site,ISTG
}
}
You have to start parsing the 10th item of output and ignore empty lines because repadmin.exe seems to insert superflous line breaks (or at least, PowerShell thinks so).