Trying to extract specific text and merge output with existing output - powershell

I want to extract text from a .txt file. The way the file is layed out is in this format (below first block). Optimally, I would like for the powershell script to take the content of username and votecount and output them side by side. With an integer of 25>= add the letter D beside it. With the output adding itself to a pre-existing output file. Say this week is week 1. And testuser voted 25 times. They should have the output "testuser" 25D. But say in week 2 they voted 24 times. Then it should be "testuser" 49D. However say they had 25 again. Output should then be "testuser" 50DD or 50D2?.. I have what I think should work as an initial baseline for the script which in itself doesn't work.. But combining an output with a pre existing output is beyond my capability. This needs to parse an entire txt file of some 100+ people. So imagine there's like an extra 100 users..
{
"username": "testuser",
"votecount": "42",
"votesclaimed": "0",
"lastvotetime": "2022-11-04 09:08:29",
"steamid": "00000000000000000000"
}
Below is what I am working with.
Get-Content -Raw C:\Users\--------\Desktop\votes.txt |
ConvertFrom-txt |
ForEach-Object {
[pscustomobject] #{
UserName = $_.username
VoteCount = '{0}{1}' -f $_.votecount, ('', 'D')[[int] $_.votecount -gt 25]
}
} |
Export-Csv -NoTypeInformation -Encoding utf8 C:\Users\---------\Desktop\outvotes.csv

Try following :
$match = Select-String -Path "c:\temp\test.txt" -Pattern '^\s*"(?<key>[^"]+)"\s*:\s*"(?<value>[^"]+)'
$table = [System.Collections.ArrayList]::new()
foreach( $row in $match.Matches )
{
$key = $row.Groups["key"].Value
$value = $row.Groups["value"].Value
if($key -eq "username") {
$newRow = New-Object -TypeName psobject
$table.Add($newRow) | Out-Null
}
$newRow | Add-Member -NotePropertyName $key -NotePropertyValue $value
}
$table | Format-Table
$groups = $table | Group-Object {$_.username}

Related

append rows in CSV using powershell

I have piece of code like below.
$month = Get-Date -Format "yyyy_MM"
$csv_location = "C:\brivo\csv\" + $month + ".csv"
if (!(Test-Path $csv_location))
{
$newcsv = {} | Select "Time","Name","Surname","Email","Telephone","Company","Department","Address","Postcode","City","State","Country" | Export-Csv $csv_location -NoTypeInformation
}
ForEach($line in $lines){
Try
{
$line = $line.Trim()
$file = "C:\brivo\json\" + $line
$data = Get-Content $file | ConvertFrom-Json
$timestamp = Get-Date -Format "yyyy-MM-dd HH:mm"
$userline = $timestamp,$data.name,$data.surname,$data.email,$data.telephone,$data.company_name,$data.department,$data.address,$data.postcode,$data.city,$data.state,$data.country
$userline | out-file $csv_location -Append
}
Catch [Exception]
{
Write-Host $_.Exception | format-list -force
}
}
where first part is creating csv file if not exist with headers. and in second part $lines is file names like 123.json, 456.json... and all those json files has content like below.
{
"name": "kamal",
"sur_name": "wewetwe",
"email": "asdasd#gmail.com",
"telephone": "311234544567",
"company_name": "",
"department": "",
"address": "qwe",
"postcode": "1234 ad",
"city": "qwe",
"state": "GR",
"country": "NL"
}
what I want is to append all this json data to csv file. I have tried like above but it adds data in 1st column itself.
Don't try to initialize your CSV file without data using Export-Csv, because that won't work:
The dummy object with $null property values created by Select-Object results not in just in a header row, but invariably also in a data row representing the $null values, which end up as empty strings; in other words, you'll get a data row that looks like this: ,,,,,,,,,,,
Instead, make sure that the properties of the objects representing data rows you pass to Export-Csv are named for the desired CSV columns.
Don't try to construct your data rows as an array of values to save to the CSV file as plain text;[1] instead, construct a [pscustomobject] for each data row, which Export-Csv automatically converts to a data row, with the property names serving as column names (as you attempted with your header row).
As the code below shows, you can construct an object in each iteration and pipe it to a single Export-Csv call for efficiency.
Use the -Encoding parameter to control the output character encoding as needed; notably, in Windows PowerShell the default encoding is ASCII(!).
-Append is still used in that single call, because it looks like you want to append to a preexisting target file, if present.
Important:
The first object sent to Export-Csv locks in the list of columns and their names based on its properties; subsequent objects are expected to have the same set of properties (or a meaningful subset).
Similarly, when appending to a preexisting CSV file with -Append, the objects to append must match the existing columns, although you can override this requirement with -Force.
$month = Get-Date -Format "yyyy_MM"
$csv_location = "C:\brivo\csv\" + $month + ".csv"
# $lines is assumed to be an array of your input JSON file names.
$lines | ForEach-Object {
Try
{
$file = "C:\brivo\json\" + $_.Trim()
$data = Get-Content -ErrorAction Stop -Raw $file |
ConvertFrom-Json -ErrorAction Stop
# Construct and output an object with the desired values and the
# properties named for the desired CSV columns.
[pscustomobject] #{
Time = Get-Date -Format 'yyyy-MM-dd HH:mm'
Name = $data.name
Surname = $data.surname
Email = $data.email
Telephone = $data.telephone
Company = $data.company_name
Department = $data.department
Address = $data.address
Postcode = $data.postcode
City = $data.city
State = $data.state
Country = $data.country
}
}
Catch
{
Write-Host $_.Exception | format-list -force
}
} | Export-Csv -NoTypeInformation -Append $csv_location
[1] If you send an array of values to Out-File, each value becomes its own line in the output file. While you could address that with $userline -join ',', such plain-text processing is brittle, because values with embedded , chars. wouldn't be handled correctly.
**this is some example **
#add your json files on temp dir
$a= Get-ChildItem C:\Temp\PatchingServer*
foreach($g in $a){
$j=gc $g
$f=$j| ConvertFrom-Json
$obj=New-Object PSobject
$obj | Add-Member Noteproperty "Port" $f.Port
$obj | Add-Member Noteproperty "ApplicationName" $f.ApplicationName
$obj | Add-Member Noteproperty "MaintenanceWindow" $f.MaintenanceWindow
$obj | Add-Member Noteproperty "BusinessUnit" $f.BusinessUnit
$obj | Add-Member Noteproperty "AppOwner" $f.AppOwner
$obj | Add-Member Noteproperty "AppID" $f.AppID
$obj | Add-Member Noteproperty "Location" $f.Location
$obj | export-csv C:\Temp\Patching.csv -NoTypeInformation -Append
}

How can I add string and create new column in my csv file using PowerShell

In my existing CSV file I have a column called "SharePoint ID" and it look like this
1.ylkbq
2.KlMNO
3.
4.MSTeam
6.
7.MSTEAM
8.LMNO83
and I'm just wondering how can I create a new Column in my CSV call "SharePoint Email" and then add "#gmail.com" to only the actual Id like "ylkbq", "KLMNO" and "LMNO83" instead of applying to all even in the blank space. And Maybe not add/transfer "MSTEAM" to the new Column since it's not an Id.
$file = "C:\AuditLogSearch\New folder\OriginalFile.csv"
$file2 = "C:\AuditLogSearch\New folder\newFile23.csv"
$add = "#GMAIL.COM"
$properties = #{
Name = 'Sharepoint Email'
Expression = {
switch -Regex ($_.'SharePoint ID') {
#Not sure what to do here
}
}
}, '*'
Import-Csv -Path $file |
Select-Object $properties |
Export-Csv $file2 -NoTypeInformation
Using calculated properties with Select-Object this is how it could look:
$add = "#GMAIL.COM"
$expression = {
switch($_.'SharePoint ID')
{
{[string]::IsNullOrWhiteSpace($_) -or $_ -match 'MSTeam'}
{
# Null value or mathces MSTeam, leave this Null
break
}
Default # We can assume these are IDs, append $add
{
$_.Trim() + $add
}
}
}
Import-Csv $file | Select-Object *, #{
Name = 'SharePoint Email'
Expression = $expression
} | Export-Csv $file2 -NoTypeInformation
Sample Output
Index SharePoint ID SharePoint Email
----- ------------- ----------------
1 ylkbq ylkbq#GMAIL.COM
2 KlMNO KlMNO#GMAIL.COM
3
4 MSTeam
5
6 MSTEAM
7 LMNO83 LMNO83#GMAIL.COM
A more concise expression, since I misread the point, it can be reduced to just one if statement:
$expression = {
if(-not [string]::IsNullOrWhiteSpace($_.'SharePoint ID') -and $_ -notmatch 'MSTeam')
{
$_.'SharePoint ID'.Trim() + $add
}
}

Exporting PowerShell Results In To CSV for Each User In The Domain That Last Changed Their Password

I have a Powershell script that queries for the pwdLastSet attribute for every user in
the Active Directory domain. Essentially, the script determines when each user in the domain last changed their password. However, when I try and output the result using scriptname.ps1 | Export-Csv "filename.csv" it creates the file, however, I'm not getting the results I see in the console. I'm getting the following:
When I run the script without Export-Csv the results I desire display correctly.
This is the Powershell script:
Trap {"Error: $_"; Break;}
$D = [System.DirectoryServices.ActiveDirectory.Domain]::GetCurrentDomain()
$Domain = [ADSI]"LDAP://$D"
$Searcher = New-Object System.DirectoryServices.DirectorySearcher
$Searcher.PageSize = 200
$Searcher.SearchScope = "subtree"
$Searcher.Filter = "(&(objectCategory=person)(objectClass=user))"
$Searcher.PropertiesToLoad.Add("distinguishedName") > $Null
$Searcher.PropertiesToLoad.Add("pwdLastSet") > $Null
$Searcher.PropertiesToLoad.Add("userAccountControl") > $Null
$Searcher.SearchRoot = "LDAP://" + $Domain.distinguishedName
$Results = $Searcher.FindAll()
ForEach ($Result In $Results)
{
$DN = $Result.Properties.Item("distinguishedName")
$PLS = $Result.Properties.Item("pwdLastSet")
$UAC = $Result.Properties.Item("userAccountControl")
# Retrieve user password settings to check if password can expire.
$blnPwdExpires = -not (($UAC.Item(0) -band 64) -or ($UAC.Item(0) -band 65536))
If ($PLS.Count -eq 0)
{
$Date = [DateTime]0
}
Else
{
# Interpret 64-bit integer as a date.
$Date = [DateTime]$PLS.Item(0)
}
If ($Date -eq 0)
{
# 0 really means never.
$PwdLastSet = "<Never>"
}
Else
{
# Convert from .NET ticks to Active Directory Integer8 ticks.
# Also, convert from UTC to local time.
$PwdLastSet = $Date.AddYears(1600).ToLocalTime()
}
"$DN;$blnPwdExpires;$PwdLastSet"
}
There are two possible issues on your code, the first one, Export-Csv is expecting an object or object[] as input and will convert it to CSV format, you're already passing a formatted semi-colon delimited string[].
In this case you should use | Out-File path\to\csv.csv instead of Export-Csv.
Do not format objects before sending them to the Export-CSV cmdlet. If Export-CSV receives formatted objects the CSV file contains the format properties rather than the object properties.
An example of what you're passing to the cmdlet and what it actually expects:
PS \> 0..5 | ForEach-Object{ 'asd;asd;asd' } | ConvertTo-Csv
#TYPE System.String
"Length"
"12"
"12"
"12"
"12"
"12"
"12"
PS \> 0..5 | ForEach-Object{ [pscustomobject]#{col1='asd';col2='asd';col3='asd'} } | ConvertTo-Csv -Delimiter ';'
#TYPE System.Management.Automation.PSCustomObject
"col1";"col2";"col3"
"asd";"asd";"asd"
"asd";"asd";"asd"
"asd";"asd";"asd"
"asd";"asd";"asd"
"asd";"asd";"asd"
"asd";"asd";"asd"
The alternative to this, and cleaner approach in my opinion, would be to cast a [pscustomobject]on each iteration of your loop and then pass the resulting array to Export-Csv (code below).
The other possible issue, assuming you're choosing the path of using [pscustomobject], could be that $Result.Properties.Item(...) will yield an object of the type System.DirectoryServices.ResultPropertyValueCollection and you would need to convert it to [string] before passing the results to Export-Csv (also code below).
# Save the resulting pscustomobject array to the $output variable
$output = ForEach ($Result In $Results)
{
...
...
...
# All code should be as is up until:
# "$DN;$blnPwdExpires;$PwdLastSet" => Remove this line
[pscustomobject]#{
DistinguishedName = [string]$DN
blnPwdExpires = [string]$blnPwdExpires
pwdLastSet = [string]$PwdLastSet
}
}
# Then pipe the result to Export-Csv
$output | Export-Csv path\to\csv.csv -NoTypeInformation -Delimiter ';'

How to convert text file containing double quotes to csv format using powershell

I have a text file(with header Actual_Output and saved it as actual.txt) containing data such as
Actual_Output
W
à
é
"
'
(
_
ç
²"
^
^
*
END
I want to convert it into csv file using powershell. I doing in this way
$DB = import-csv E:\actual.txt
$outarray = #()
foreach ($Data in $DB)
{
$First = $Data.Actual_Output
$outarray += New-Object PsObject -property #{
'Actual_Output' = $First
}
write-host "Actual_Output: " $First
write-Host ""
}
$outarray | export-csv 'E:\result.csv' -NoTypeInformation -Encoding utf8
I am getting the output like this as shown in screenshot
I want each data to be listed in seperate cell. Actually double quote " is creating problem here. Please help in resolving this. Sorry if i am unclear in describing the issue
Tested this, and it seems to work better:
Get-Content actual.txt | select -Skip 1 |
foreach {
New-Object PSObject -Property #{Actual_Output = $_}
} | export-csv result.csv -NoTypeInformation -Encoding UTF8
The file isn't well-formed as CSV initially, so Import-CSV isn't able to parse it correctly.

Get a variable by dynamic variable name

How does one access data imported from a CSV file by using dynamic note property names? That is, one doesn't know the colunm names beforehand. They do match a pattern and are extracted from the CSV file when the script runs.
As for an example, consider a CSV file:
"Header 1","Header A","Header 3","Header B"
0,0,0,0
1,2,3,4
5,6,7,8
I'd like to extract only columns that end with a letter. To do this, I read the header row and extract names with a regex like so,
$reader = new-object IO.StreamReader("C:\tmp\data.csv")
$line = $reader.ReadLine()
$headers = #()
$line.Split(",") | % {
$m = [regex]::match($_, '("Header [A-Z]")')
if($m.Success) { $headers += $m.value } }
This will get all the column names I care about:
"Header A"
"Header B"
Now, to access a CSV file I import it like so,
$csvData = import-csv "C:\tmp\data.csv"
Import-CSV will create a custom object that has properties as per the header row. One can access the fields by NoteProperty names like so,
$csvData | % { $_."Header A" } # Works fine
This obviously requires one to know the column name in advance. I'd like to use colunn names I extracted and stored into the $headers. How would I do that?
Some things I've tried so far
$csvData | % { $_.$headers[0] } # Error: Cannot index into a null array.
$csvData | % { $np = $headers[0]; $_.$np } # Doesn't print anything.
$csvData | % { $_.$($headers[0]) } # Doesn't print anything.
I could change the script like so it will write another a script that does know the column names. Is that my only solution?
I think you want this:
[string[]]$headers = $csvdata | gm -MemberType "noteproperty" |
?{ $_.Name -match "Header [a-zA-Z]$"} |
select -expand Name
$csvdata | select $headers
Choose the headers that match the condition (in this case, ones ending with characters) and then get the csv data for those headers.
the first thing ( and the only one... sorry) that came in my mind is:
$csvData | % { $_.$(( $csvData | gm | ? { $_.membertype -eq "noteproperty"} )[0].name) }
for get the first's column values and
$csvData | % { $_.$(( $csvData | gm | ? { $_.membertype -eq "noteproperty"} )[1].name) }
for second column and so on....
is this what you need?
you can use custom script to parse csv manually:
$content = Get-Content "C:\tmp\data.csv"
$header = $content | Select -first 1
$columns = $header.Split(",")
$indexes = #()
for($i; $i -lt $columns.Count;$i++)
{
# to verify whether string end with letter matches this regex: "[A-Za-z]$"
if ($column[$i] -match "[A-Za-z]$")
{
$indexes += $i
}
}
$outputFile = "C:\tmp\outdata.csv"
Remove-Item $outputFile -ea 0
foreach ($line in $content)
{
$output = ""
$rowcol = $line.Split(",")
[string]::Join(",", ($indexes | foreach { $rowcol[$_] })) | Add-Content $outputFile
}