Problem error checking -match and IF statement - powershell

The below script will import an exported CSV from our MIS system so that we can upload to google classroom. In tern, this will allow bulk creation of classes with our custom classnames based on word matching within the regex.csv.
As you can see from the exporteddata.csv, 10552 is blank.
Is it possible for this to be omitted from the final export and added into its own errors.csv file?
Any help would be great!
Script.ps1
$data = Import-Csv "$PSScriptRoot\data.csv" -Delimiter ','
$patterns = Import-Csv "$PSScriptRoot\Regex\regex.csv" -Delimiter ','
$interimexportedData = "$PSScriptRoot\classesinterim.csv"
$exportclasses = "$PSScriptRoot\exporteddata.csv"
## Imports the initial SIMS export of classes and created a 'prefered' name for the class, then exports to a CSV.
$data | Select-Object *,#{Name='preference'; Expression={
foreach ($p in $patterns) {
if ($_.title -match $p.'regex_key') {
$p.preference + " " + "-" + " " + $_.title
return
}
}
}
} | Select-Object -property sourcedID, preference | Export-Csv $interimexportedData -NoTypeInformation
## The below re-imports the csv file and renames the header
Import-Csv $interimexportedData |
Select-Object -property sourcedID, #{ expression={$_.preference}; label='title' } |
Export-Csv -NoTypeInformation $exportclasses
## Delete the classesinterim.csv from the folder
Remove-Item $interimexportedData
data.csv
"sourcedId","title"
9443,"10A/BS1"
9444,"10A/FR1"
10598,"10A/Ft"
9445,"10A/GG1"
9446,"10A/HI1"
9447,"10A/ME1"
9451,"10A/ME2"
9448,"10A/RM1"
9449,"10A/SCTrX"
9452,"10A/SCTrY"
10552,"10A/SOS"
9450,"10A/SP1"
exporteddata.csv
"sourcedId","title"
"9443","Business Studies - 10A/BS1"
"9444","French - 10A/FR1"
"10598","Form Time - 10A/Ft"
"9445","Geography - 10A/GG1"
"9446","History - 10A/HI1"
"9447","Media Studies - 10A/ME1"
"9451","Media Studies - 10A/ME2"
"9448","Resistant Materials - 10A/RM1"
"9449","Science - 10A/SCTrX"
"9452","Science - 10A/SCTrY"
"10552",""
regex.csv
"regex_key","preference"
BS,"Business Studies"
FR, "French"
Ar,"Art"
Bt,"Eng & Maths Booster"
Bs,"Business"
Cn,"Construction"
Co,"Computing"

Use Where-Object to filter out objects with a blank value:
$data | Select-Object *,#{Name='preference'; Expression={
foreach ($p in $patterns) {
if ($_.title -match $p.'regex_key') {
$p.preference + " " + "-" + " " + $_.title
return
}
}
} |Where-Object preference -ne ''

Related

How can I add string and create new column in my csv file using PowerShell

In my existing CSV file I have a column called "SharePoint ID" and it look like this
1.ylkbq
2.KlMNO
3.
4.MSTeam
6.
7.MSTEAM
8.LMNO83
and I'm just wondering how can I create a new Column in my CSV call "SharePoint Email" and then add "#gmail.com" to only the actual Id like "ylkbq", "KLMNO" and "LMNO83" instead of applying to all even in the blank space. And Maybe not add/transfer "MSTEAM" to the new Column since it's not an Id.
$file = "C:\AuditLogSearch\New folder\OriginalFile.csv"
$file2 = "C:\AuditLogSearch\New folder\newFile23.csv"
$add = "#GMAIL.COM"
$properties = #{
Name = 'Sharepoint Email'
Expression = {
switch -Regex ($_.'SharePoint ID') {
#Not sure what to do here
}
}
}, '*'
Import-Csv -Path $file |
Select-Object $properties |
Export-Csv $file2 -NoTypeInformation
Using calculated properties with Select-Object this is how it could look:
$add = "#GMAIL.COM"
$expression = {
switch($_.'SharePoint ID')
{
{[string]::IsNullOrWhiteSpace($_) -or $_ -match 'MSTeam'}
{
# Null value or mathces MSTeam, leave this Null
break
}
Default # We can assume these are IDs, append $add
{
$_.Trim() + $add
}
}
}
Import-Csv $file | Select-Object *, #{
Name = 'SharePoint Email'
Expression = $expression
} | Export-Csv $file2 -NoTypeInformation
Sample Output
Index SharePoint ID SharePoint Email
----- ------------- ----------------
1 ylkbq ylkbq#GMAIL.COM
2 KlMNO KlMNO#GMAIL.COM
3
4 MSTeam
5
6 MSTEAM
7 LMNO83 LMNO83#GMAIL.COM
A more concise expression, since I misread the point, it can be reduced to just one if statement:
$expression = {
if(-not [string]::IsNullOrWhiteSpace($_.'SharePoint ID') -and $_ -notmatch 'MSTeam')
{
$_.'SharePoint ID'.Trim() + $add
}
}

For each thing in one CSV check for multiple types of matches in another CSV

Sorry if the description is unclear, but I couldn't think of how else to word it.
I have two CSV files:
LocalAdmins.csv -- ColumnA = PC name; ColumnB = username in local admin group
Exempt.csv -- ColumnA = PC name; ColumnB = username allowed to be a local admin
What I'm trying to do is loop through LocalAdmins.csv, and for each one check to see if the PC name shows up in Exempt.csv (or matches any defined naming patterns in that file), and if a match is found, check to see if the local admin username for that PC in LocalAdmins.csv shows up in the list of AllowedUsers for that PC in Exempt.csv.
If the username is NOT in the AllowedUsers list, or if the PC name is not in Exempt.csv, then output the entry from LocalAdmins.csv. Here is what I have so far:
$admins = Import-Csv .\LocalAdmins.csv
$exempt = Import-Csv .\Exempt.csv
$violations = ".\Violations.csv"
foreach ($admin in $admins) {
foreach ($item in $exempt) {
if ($admin.PC -like $item.PC) {
if ($admin.Name -notin ($item.AllowedUsers -split ",")) {
$admin | Export-Csv $violations -Append -NoTypeInformation
}
}
else {
$admin | Export-Csv $violations -Append -NoTypeInformation
}
}
}
The problem is the nested foreach loop generates duplicates, meaning if there are 3 lines in Exempt.csv then a single entry in LocalAdmins.csv will have 3 duplicate outputs (one for each line in Exempt.csv). So the output looks like this:
When it should look like this:
I'm guessing the problem is somewhere in the structure of the loops, but I just need some help figuring out what to tweak. Any input is greatly appreciated!
Not optimized (unique sort by any property should work):
$admins = Import-Csv .\LocalAdmins.csv
$exempt = Import-Csv .\Exempt.csv
$violations = ".\Violations.csv"
$(
foreach ($admin in $admins) {
foreach ($item in $exempt) {
if ($admin.PC -like $item.PC) {
if ($admin.Name -notin ($item.AllowedUsers -split ",")) {
$admin
}
}
else {
$admin
}
}
}
) | Sort-Object -Property PC, Name -Unique |
Export-Csv $violations -Append -NoTypeInformation
With better restrictions of the forEach, there shouldn't be duplicates
and no need to Sort -unique.
Getting input from here-strings
## Q:\Test\2019\02\05\SO_54523868.ps1
$admins = #'
PC,NAME
XYZlaptop,user6
workstationXYZ,user7
computerABC,user8
ABClaptop,user1
'# | ConvertFrom-Csv # .\LocalAdmins.csv
$exempt = #'
PC,AllowedUsers
*laptop,"user1,user2"
computerXYZ,"user3,user4"
workstation*,"user5"
'# | ConvertFrom-Csv # .\Exempt.csv
$violationsFile = ".\Violations.csv"
$violations = foreach ($admin in $admins) {
$violation = $True
foreach ($item in ($exempt|Where-Object {$admin.PC -like $_.PC})){
if ($admin.NAME -in ($item.AllowedUsers -split ',')){
$violation = $False
}
}
if ($violation){$admin}
}
$violations
$violations | Export-Csv $violationsFile -NotypeInformation
## with Doug Finke's ImportExcel module installed, you can directly get the excel file:
#$violations | Export-Excel .\Violatons.xlsx -AutoSize -Show

PowerShell - transpose results from a hashtable

I need to check the warranty of many servers, but the output returned by the module I found in https://www.powershellgallery.com/packages/HPWarranty/2.6.2 seems to be a hashtable and the first column contains what I want to be my rows.
the script below will return this where every 5 rows the fields repeat - output1.csv:
TYPE System.Management.Automation.PSCustomObject
"Component","Codecount"
"SerialNumber","CZ36092P5H"
"ProductNumber","727021-B21"
"OverallEntitlementStartDate","2016-03-04"
"OverallEntitlementEndDate","2019-04-02"
"ActiveEntitlement","true"
"SerialNumber","CZ36092P5K"
"ProductNumber","727021-B21"
"OverallEntitlementStartDate","2016-03-04"
"OverallEntitlementEndDate","2019-04-02"
"ActiveEntitlement","true"
How can I transpose the output so that SerialNumber, ProductNumber, OverallEntitlementStartDate, OverallEntitlementEndDate and ActiveEntitlement are the columns?
# variables
$dest_path = "C:\Scripts\HPE\HPWarranty"
$export_date = Get-Date -Format o | ForEach-Object {$_ -replace ":", "-"}
$myScriptName = $MyInvocation.MyCommand.Name
$transcriptPath = $dest_path + "\" + $myScriptName + "_transcript_" + $export_date + ".txt"
$csvPath = $dest_path + "\" + "hpe_list1.csv"
#Start transcript of script activities and set transcript location
start-transcript -append -path $transcriptPath | Out-Null
# import serials & part numbers to be processed
$csv_info = Import-Csv $csvPath
foreach ($line in $csv_info) {
$hash = (Get-HPEntWarrantyEntitlement -ProductNumber $line.ProductNumber -SerialNumber $line.SerialNumber)
&{$hash.getenumerator() |
ForEach-Object {new-object psobject -Property #{Component = $_.name;Codecount=$_.value}}
} | Export-Csv "C:\Scripts\HPE\HPWarranty\output1.csv" -Append
}
# Stop Transcript
Stop-Transcript | Out-Null
hpe_list1.csv that the script processes contains the details for two servers:
ProductNumber,SerialNumber
727021-B21,CZ36092P5H
727021-B21,CZ36092P5K
Cast the output hashtable to a [pscustomobject]:
$WarrantyInfo = foreach ($line in $csv_info) {
[pscustomobject](Get-HPEntWarrantyEntitlement -ProductNumber $line.ProductNumber -SerialNumber $line.SerialNumber)
}
$WarrantyInfo | Export-Csv "C:\Scripts\HPE\HPWarranty\output1.csv"

Combining like objects in an array

I am attempting to analyze a group of text files (MSFTP logs) and do counts of IP addresses that have submitted bad credentials. I think I have it worked out except I don't think that the array is passing to/from the function correctly. As a result, I get duplicate entries if the same IP appears in multiple log files. What am I doing wrong?
Function LogBadAttempt($FTPLog,$BadPassesArray)
{
$BadPassEx="PASS - 530"
Foreach($Line in $FTPLog)
{
if ($Line -match $BadPassEx)
{
$IP=($Line.Split(' '))[1]
if($BadPassesArray.IP -contains $IP)
{
$CurrentIP=$BadPassesArray | Where-Object {$_.IP -like $IP}
[int]$CurrentCount=$CurrentIP.Count
$CurrentCount++
$CurrentIP.Count=$CurrentCount
}else{
$info=#{"IP"=$IP;"Count"='1'}
$BadPass=New-Object -TypeName PSObject -Property $info
$BadPassesArray += $BadPass
}
}
}
return $BadPassesArray
}
$BadPassesArray=#()
$FTPLogs = Get-Childitem \\ftpserver\MSFTPSVC1\test
$Result = ForEach ($LogFile in $FTPLogs)
{
$FTPLog=Get-Content ($LogFile.fullname)
LogBadAttempt $FTPLog
}
$Result | Export-csv C:\Temp\test.csv -NoTypeInformation
The result looks like...
Count IP
7 209.59.17.20
20 209.240.83.135
18441 209.59.17.20
13059 200.29.3.98
and would like it to combine the entries for 209.59.17.20
You're making this way too complicated. Process the files in a pipeline and use a hashtable to count the occurrences of each IP address:
$BadPasswords = #{}
Get-ChildItem '\\ftpserver\MSFTPSVC1\test' | Get-Content | ? {
$_ -like '*PASS - 530*'
} | % {
$ip = ($_ -split ' ')[1]
$BadPasswords[$ip]++
}
$BadPasswords.GetEnumerator() |
select #{n='IP';e={$_.Name}}, #{n='Count';e={$_.Value}} |
Export-Csv 'C:\Temp\test.csv' -NoType

I need help formatting output with PowerShell's Out-File cmdlet

I have a series of documents that are going through the following function designed to count word occurrences in each document. This function works fine outputting to the console, but now I want to generate a text file containting the information, but with the file name appended to each word in the list.
My current console output is:
"processing document1 with x unique words occuring as follows"
"word1 12"
"word2 8"
"word3 3"
"word4 4"
"word5 1"
I want a delimited file in this format:
document1;word1;12
document1;word2;8
document1;word3;3
document1;word4;4
document1;word1;1
document2;word1;16
document2;word2;11
document2;word3;9
document2;word4;9
document2;word1;13
While the function below gets me the lists of words and occurences, I'm having a hard time figuring out where or how to insert the filename variable so that it prints at the head of each line. MSDN has been less-than helpful, and most of the places I try to insert the variable result in errors (see below)
function Count-Words ($docs) {
$document = get-content $docs
$document = [string]::join(" ", $document)
$words = $document.split(" `t",[stringsplitoptions]::RemoveEmptyEntries)
$uniq = $words | sort -uniq
$words | % {$wordhash=#{}} {$wordhash[$_] += 1}
Write-Host $docs "contains" $wordhash.psbase.keys.count "unique words distributed as follows."
$frequency = $wordhash.psbase.keys | sort {$wordhash[$_]}
-1..-25 | %{ $frequency[$_]+" "+$wordhash[$frequency[$_]]} | Out-File c:\out-file-test.txt -append
$grouped = $words | group | sort count
Do I need to create a string to pass to the out-file cmdlet? is this just something I've been putting in the wrong place on the last few tries? I'd like to understand WHY it's going in a particular place as well. Right now I'm just guessing, because I know I have no idea where to put the out-file to achieve my selected results.
I've tried formatting my command per powershell help, using -$docs and -FilePath, but each time I add anything to the out-file above that runs successfully, I get the following error:
Out-File : Cannot validate argument on parameter 'Encoding'. The argument "c:\out-file-test.txt" does not bel
ong to the set "unicode,utf7,utf8,utf32,ascii,bigendianunicode,default,oem" specified by the ValidateSet attribute. Sup
ply an argument that is in the set and then try the command again.
At C:\c.ps1:39 char:71
+ -1..-25 | %{ $frequency[$_]+" "+$wordhash[$frequency[$_]]} | Out-File <<<< -$docs -width 1024 c:\users\x46332\co
unt-test.txt -append
+ CategoryInfo : InvalidData: (:) [Out-File], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.OutFileCommand
I rewrote most of your code. You should utilize objects to make it easier formatting the way you want. This one splits on "space" and groups words together. Try this:
Function Count-Words ($paths) {
$output = #()
foreach ($path in $paths) {
$file = Get-ChildItem $path
((Get-Content $file) -join " ").Split(" ", [System.StringSplitOptions]::RemoveEmptyEntries) | Group-Object | Select-Object -Property #{n="FileName";e={$file.BaseName}}, Name, Count | % {
$output += "$($_.FileName);$($_.Name);$($_.Count)"
}
}
$output | Out-File test-out2.txt -Append
}
$filepaths = ".\test.txt", ".\test2.txt"
Count-Words -paths $filepaths
It outputs like you asked(document;word;count). If you want documentname to include extension, change $file.BaseName to $file.Name . Testoutput:
test;11;1
test;9;2
test;13;1
test2;word11;5
test2;word1;4
test2;12;1
test2;word2;2
Slightly different approach:
function Get-WordCounts ($doc)
{
$text_ = [IO.File]::ReadAllText($doc.fullname)
$WordHash = #{}
$text_ -split '\b' -match '\w+'|
foreach {$WordHash[$_]++}
$WordHash.GetEnumerator() |
foreach {
New-Object PSObject -Property #{
Word = $_.Key
Count = $_.Value
}
}
}
$docs = gci c:\testfiles\*.txt |
sort name
&{
foreach ($doc in dir $docs)
{
Get-WordCounts $doc |
sort Count -Descending |
foreach {
(&{$doc.Name;$_.Word;$_.Count}) -join ';'
}
}
} | out-file c:\somedir\wordcounts.txt
Try this:
$docs = #("document1", "document2", ...)
$docs | % {
$doc = $_
Get-Content $doc `
| % { $_.split(" `t",[stringsplitoptions]::RemoveEmptyEntries) } `
| Group-Object `
| select #{n="Document";e={$doc}}, Name, Count
} | Export-CSV output.csv -Delimiter ";" -NoTypeInfo
If you want to make this into a function you could do it like this:
function Count-Words($docs) {
foreach ($doc in $docs) {
Get-Content $doc `
| % { $_.split(" `t",[stringsplitoptions]::RemoveEmptyEntries) } `
| Group-Object `
| select #{n="Document";e={$doc}}, Name, Count
}
}
$files = #("document1", "document2", ...)
Count-Words $files | Export-CSV output.csv -Delimiter ";" -NoTypeInfo