Compare Multiple Spreadsheets and Produce CSV output - powershell

I am trying to compare two csv files, the first is called tickets.csv and contains the column headers: "tcn", "two", "twod". The second file is called tracker.csv and contains the headers: "cn", "wo", and "wod". I would like to output the contents of tracker.csv, and if any "tcn" (tickets.csv) match "cn" (tracker.csv), then write its associated "two" (tickets.csv) to "wo" (tracker.csv), and "twod" (tickets.csv) to "wod" (tracker.csv).
I'm running into the issue where my code isn't moving the "two" or "wod", and its also writing each object 4 times.
Input CSVs
Import-Csv "$home\Desktop\nestTest\tracker.csv" |
ForEach-Object {
$machine = $_.machine
$wo = ""
$wod = ""
$tickets = import-csv "$home\Desktop\nestTest\tickets.csv"
foreach ($ticket in $tickets) {
try {
$tcn = ""
$two = ""
$twod = ""
$two = $_.wo
$twod = $_.wod
$tcn = $_.cn
if ($tcn -eq $machine) {
$wo = $two
$wod = $twod
}
else {
$wo = ""
$wod = ""
}
}
catch {
}
[pscustomobject]#{
"Machine Name"=$machine
"Work Order #"=$wo
"Work Order Date"=$wod
}
}
} |
select "Machine Name","Work Order #","Work Order Date" |
Export-Csv "$home\Desktop\nestTest\output.csv"

I was looking in the wrong direction before. But I figured it out.
$tracker = Import-Csv "$home\Desktop\nestTest\tracker.csv" |
ForEach-Object {
$machine = ""
$wo = ""
$wod = ""
$machine = $_.machine
$tickets = import-csv "$home\Desktop\nestTest\tickets.csv" |
ForEach-Object {
$two = ""
$twod = ""
$two = $_."wo"
$twod = $_."wod"
$tcn = $_."cn"
if ($tcn -eq $machine) {
$wo = $two
$wod = $twod
}
else {
}
}
[pscustomobject]#{
"Machine Name"=$machine
"Work Order #"=$wo
"Work Order Date"=$wod
}
} |
select "Machine Name","Work Order #","Work Order Date" |
Export-Csv "$home\Desktop\nestTest\output.csv" -NoTypeInformation

Related

Powershell - Exchange JSON output without needing to write to a file

EDIT: Added Setupconfigfiles.ps1
I'm a bit new to detailed scripting so please bear with me.
I have two Powershell scripts
Setupconfigfiles.ps1 generates JSON output to be fed to an API.
Script2 uses that JSON data to execute API commands.
Script 2 can call setupconfigfiles.ps1 as indicated below and use the output data.
.\SetupConfigFiles.ps1 -type $Type -outfile .\Templist.json
$servers = Get-Content -Raw -Path .\templist.json | ConvertFrom-Json
setupconfigfiles.ps1:
param (
# If this parameter is set, format the output as csv.
# If this parameter is not set, just return the output so that the calling program can use the info
[string]$outfile,
# this parameter can be 'production', 'development' or 'all'
[string]$type
)
enum MachineTypes {
production = 1
development = 2
all = 3
}
$Servers = Get-ADObject -Filter 'ObjectClass -eq "computer"' -SearchBase 'Obfuscated DSN' | Select-Object Name
$output = #()
$count = 0
# Set this to [MachineTypes]::production or [MachineTypes]::development or [MachineTypes]::all
if ($type -eq "all") {
$server_types = [MachineTypes]::all
}
ElseIf ($type -eq "production") {
$server_types = [MachineTypes]::production
}
else {
$server_types = [MachineTypes]::development
}
ForEach ($Server in $Servers)
{
$count = $count + 1
$this_server = #{}
$this_server.hostname = $Server.Name
$this_server.id = $count
$this_server."site code" = $this_server.hostname.substring(1,3)
$this_server."location code" = $this_server.hostname.substring(4,2)
if ($this_server.hostname.substring(7,1) -eq "P") {
$this_server.environment = "Production"
}
ElseIf ($this_server.hostname.substring(7,1) -eq "D") {
$this_server.environment = "Development"
}
Else {
$this_server.environment = "Unknown"
}
if (($server_types -eq [MachineTypes]::production ) -and ($this_server.environment -eq "Production")) {
$output += $this_server
}
ElseIf (($server_types -eq [MachineTypes]::development ) -and ($this_server.environment -eq "Development")) {
$output += $this_server
}
Else {
if ($server_types -eq [MachineTypes]::all ) {
$output += $this_server
}
}
}
if ($outfile -eq "")
{
ConvertTo-Json $output
}
else {
ConvertTo-Json $output | Out-File $outfile
}
How can I do it without needing to write to the Templist.json file?
I've called this many different ways. The one I thought would work is .\SetupConfigFiles.ps1 $servers
Y'all are great. #Zett42 pointed me in a direction and #Mathias rounded it out.
The solution was to change:
"ConvertTo-Json $output" to "Write-Output $output"
Then it's handled in the calling script.
thanks!

CSV multiple filters

Im importing a csv and have multiple filters that the end user can select to filter the csv.
foreach filter Im += to an array.
Problem I run into is-
If a user selects 'Medium' and USD, they should only be able to see Rows of data which has both 'Medium' from the impact column and USD from country.
My results are returning 'Medium' from any country AND 'USD' events?
I feel like Im going about this the wrong way. Any ideas?
Thank you for your time.
$buttonFilter_Click={
$r = #()
#Filter Impact Column#
if ($checkboxLow.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Impact -eq 'Low' }
}
if ($checkboxMedium.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Impact -eq 'Medium' }
}
if ($checkboxHigh.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Impact -eq 'High' }
}
#Filter Country column#
if ($checkboxUSD.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'USD' }
}
if ($checkboxCHF.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'CHF' }
}
if ($checkboxGBP.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'GBP' }
}
if ($checkboxAUD.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'AUD' }
}
if ($checkboxCNY.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'CNY' }
}
if ($checkboxJPY.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'JPY' }
}
if ($checkboxCAD.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'CAD' }
}
if ($checkboxEUR.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'EUR' }
}
if ($checkboxNZD.Checked -eq $true)
{
$r += $script:WorldWideNews | ? { $_.Country -eq 'NZD' }
}
$table = ConvertTo-DataTable -InputObject $r
Update-DataGridView -DataGridView $datagridviewUpcomingNews -Item $table
}
Your code forms the union of all filter results, along both dimensions, impact and country (currency).
Instead, you need the intersection of the results from these two dimensions.
The immediate fix is to collect an intermediate result for the filtering performed along the first dimension, and then base the filtering along the second dimension on that:
$r = #()
#Filter Impact Column#
if ($checkboxLow.Checked)
{
$r += $script:WorldWideNews | ? { $_.Impact -eq 'Low' }
}
# ...
# Save the results of the impact filtering
# and base all subsequent filtering on that.
$impactFilterResults = $r
$r = #()
#Filter Country column# - note the use of $impactFilterResults as input.
if ($checkboxUSD.Checked)
{
$r += $impactFilterResults | ? { $_.Country -eq 'USD' }
}
if ($checkboxCHF.Checked)
{
$r += $impactFilterResults | ? { $_.Country -eq 'CHF' }
}
# ...
Note, however, that your code can be streamlined, by ultimately combining all filters into a composite one that a single Where-Object (?) call could perform:
$buttonFilter_Click = {
# Map the checkboxes to the corresponding values to filter by.
$impactValueMap = #{
$checkboxLow = 'Low'
$checkboxMedium = 'Medium'
$checkboxHigh = 'High'
}
# Based on which are checked, determine the list of values to filter by.
[array] $impactsToFilterBy = foreach ($checkbox in $checkboxLow, $checkboxMedium, $checkboxHigh) {
if ($checkbox.Checked) { $impactValueMap[$checkbox] }
}
# Note: Truncated to 3 entries for brevity
$countryValueMap = #{
$checkboxUSD = 'USD'
$checkboxCHF = 'CHF'
$checkboxGBP = 'GBP'
# ...
}
[array] $countriesToFilterBy = foreach ($checkbox in $checkboxUSD, $checkboxCHF, $checkboxGBP) {
if ($checkbox.Checked) { $countryValueMap[$checkbox] }
}
# Use a single Where-Object call to filter, based
# on *arrays* of values to match.
$r =
$script:WorldWideNews |
Where-Object { $_.Impact -in $impactsToFilterBy -and $_.Country -in $countriesToFilterBy }
$table = ConvertTo-DataTable -InputObject $r
Update-DataGridView -DataGridView $datagridviewUpcomingNews -Item $table
}

Two Objects with different information

When I get email delegation I do it like this
If ($Delegate.count -eq 0) {
$Result = [PSCustomObject]#{
PrimaryEmail = $SourceEmail
Delegate = "None"
}
$resultsEmail += $Result
}
Else {
Foreach ($user in $Delegate) {
$Result = [PSCustomObject]#{
PrimaryEmail = $SourceEmail
Delegate = $user.DelegateAddress
}
$resultsEmail += $Result
}
}
I would like to also get Calendar delegation and add it. The problem is it is two separate amounts of information. Either can be various amounts.
Foreach ($user in $DelegateCalendarl) {
$Result = [PSCustomObject]#{
CalendarDelegateEmail = $DelegateCalendar.'scope.value'
CalendarRole = $DelegateCalendar.role
CalendarDelegateType = $DelegateCalendar.'scope.type'
}
$ResultsCal += $Result
The end out out would be a nice table like
PrimaryEmail | Delegate | CalendarDelegateEmail | CalendarRole | CalendarDelegateType
user#email Delagate#email Delagate#email manager user
user#email "" Delagate2#email Reader user
How can I combine both Results into one nice table csv file out like this example above?
(I know how to output them separately )
This is what I did. Not pretty but it works.
$SourceEmail = 'test#email.com'
[array]$DelegateEmail = gam user $SourceEmail print delegates | ConvertFrom-Csv
[array]$DelegateCalendar = gam calendar $SourceEmail print acls | ConvertFrom-Csv
[int]$max = $DelegateCalendar.Count
if ([int]$DelegateEmail.count -gt [int]$DelegateCalendar.count) { $max = $DelegateEmail.Count; }
$Output = #()
$Result = $null
If ($DelegateEmail.count -eq 0) {$mailD = "None"}
If ($DelegateEmail.count -eq 1) {$mailD = $DelegateEmail.DelegateAddress}
for ( $i = 0; $i -lt $max; $i++) {
If ($DelegateEmail.count -eq 0 -and $i -ige 1) {$mailD = ""}
If ($DelegateEmail.count -eq 1 -and $i -gt 0) {$mailD = ""}
if ($DelegateEmail.count -gt 1) {$MailD = $DelegateEmail.DelegateAddress[$i]}
$Result = [PSCustomObject]#{
PrimaryEmail = $SourceEmail
DelegateEmailAddress = $mailD
CalendarDelegateEmail = $DelegateCalendar.'scope.value'[$i]
CalendarRole = $DelegateCalendar.Role[$i]
CalendarDelegateType = $DelegateCalendar.'scope.type'[$i]
}
$Output += $Result
}
$Output | Export-Csv C:\test.csv -NoTypeInformation

Why is powershell changing the variable data? Can i make it stop?

My script takes the content of 2 files, which have an id in the first element of each line. I'm trying to take the id of each line in file 1, find a matching line in file 2, then compare the file 1 line to the file 2 line. If they are different, that difference will be sent to a database file to update. For some reason, the variables holding my data just decide to not hold data anymore. In the following example, the CompareData loses the value of $web and $data, sometime after the first if ($web -match) statement.
function CompareData($web, $data)
{
$webs = $web -split "###"
$datas = $data -split "###"
if ($web -match "1862185823")
{
write-host "$web"
###this will display the full line of the web file, and was only
#added to show what i'm talking about
}
##########################The value of web seems to break sometime after these
##########################variable declarations.
$wid = $webs[0];$wit = $webs[1];$wst = $webs[2];$wpr = $webs[3];$won = $webs[4];$wsd = $webs[5];$wtd = $webs[6];$wcd = $webs[7];$weh = $webs[8];$wdh = $webs[9];$wah = $webs[10];$wac = $webs[11];$wpd = $webs[12]
$did = $datas[0];$dit = $datas[1];$dst = $datas[2];$dpr = $datas[3];$don = $datas[4];$dsd = $datas[5];$dtd = $datas[6];$dcd = $datas[7];$deh = $datas[8];$ddh = $datas[9];$dah = $datas[10];$dac = $datas[11];$dpd = $datas[12]
$wehstr = [string]$weh;$wdhstr = [string]$wdh;$wahstr = [string]$wah;$dehstr = [string]$deh;$dahstr = [string]$dah;$ddhstr = [string]$ddh
$wdCount = $wit.Length / 2
$dtCount = $wit.Length / 2
$newWtitle = $wit.Substring(0, $wdCount)
$newDtitle = $dit.Substring(0, $dtCount)
if ($web -match "1862185823")
{
#the actual contents of $web are now wrong!!!!!!!!!!!!!!!!
}
if ($wehstr.Length -gt $dehstr.Length){$dehstr = $dehstr.PadRight($dehstr.Length + 1, "0")}
if ($wdhstr.Length -gt $ddhstr.Length){$ddhstr = $ddhstr.PadRight($ddhstr.Length + 1, "0")}
if ($wahstr.Length -gt $dahstr.Length){$dahstr = $dahstr.PadRight($dahstr.Length + 1, "0")}
if ($dehstr.Length -gt $wehstr.Length){$wehstr = $wehstr.PadRight($wehstr.Length + 1, "0")}
if ($dahstr.Length -gt $wahstr.Length){$wahstr = $wahstr.PadRight($wahstr.Length + 1, "0")}
if ($ddhstr.Length -gt $wdhstr.Length){$wdhstr = $wdhstr.PadRight($wdhstr.Length + 1, "0")}
if ($newWtitle -match $newDtitle -and $wehstr -match $dehstr -and $wahstr -match $dahstr -and $ddhstr -match $wdhstr -and $wsd -match $dsd -and $wtd -match $dtd -and $wcd -match $dcd -and $won -match $don -and $wst -match $dst -and $wpr -match $dpr -and $wac -match $dac -and $wpd -match $dpd)
{
#write-host "$wid and $did match"
}
else
{
write-host "they didn't match"
}
$wid = "";$wit = "";$wst = "";$wpr = "";$won = "";$wsd = "";$wtd = "";$wcd = "";$weh = "";$wdh = "";$wah = "";$wac = "";$wpd = "";
$did = "";$dit = "";$dst = "";$dpr = "";$don = "";$dsd = "";$dtd = "";$dcd = "";$deh = "";$ddh = "";$dah = "";$dac = "";$dpd = "";
$webs = ""
$datas = ""
}
$dc = gc "${DatabaseFile}"
$wc = gc "${WebFile}"
write-host "Started"
foreach ($w in $wc)
{
try
{
$x = 0
$ws = $w -split "###"#custom delimiter since people are using tabs pipes and everything else in the titles....
$webid = $ws[0]
$result = select-string -Path "${DatabaseFile}" -pattern $webid
if ($result -ne $null)
{
$newresult = $result -replace "D\:\\phi\\phs\\Oversight\\Projects\\Database.txt:\d{1,}:", ""
$doublecheck = $newresult -split "###"
if ($doublecheck[0] -eq $webid)
{
CompareData $w $newresult
}
else
{
write-host "Found a mismatch id, searching individual records line by line for a matching id"
foreach ($d in $dc)#this slow method is only called when someone has another ID in the title in addition to the
{#first element which is also id. Select-string grabs a match in the entire document, so this is here for single
#searching each record for a matching id in the first element
$ds = $d -split "###"
$did = $ds[0]
$found = 0
if ($did -eq $webid)
{
CompareData $w $d
$found = 1
break
}
if ($found -eq 0)
{
write-host "Added $w to Insert file"
$w|out-file -append "${InsertFile}"
}
}
}
}
else
{
write-host "Added $w to Insert file"
$w|out-file -append "${InsertFile}"
}
}
catch
{
}
}
It turns out, the bug in this instance was with the lines
$wdCount = $wit.Length / 2
$dtCount = $wit.Length / 2
$newWtitle = $wit.Substring(0, $wdCount)
$newDtitle = $dit.Substring(0, $dtCount)
These actually take the length of the title element then halve it. I don't need an exact title comparison, just the first half of it's letters just incase someone did infact change the title completely. The new code has corrected this issue, and things are working again.
$wtl = $wit.length
$dtl = $dit.length
$sublength = 0
if ($wtl -ge $dtl)
{
$sublength = $dtl / 2
}
else
{
$sublength = $wtl / 2
}
$newWtitle = $wit.Substring(0, $sublength)
$newDtitle = $dit.Substring(0, $sublength)

Encoding CSV content to UTF-8

So I have this powershell script Which import a csv file, replacing null into '0' and export this csv.
The issue is that the content and header of this csv is in hebrew
I tried almost everything
Used -Encoding for all types of encoding but nothing
Any Suggestions?
$propertyTranslation = #(
#{ Name = 'Customer__c'; Expression = { $_.'לקוח' } }
#{ Name = 'Name__c'; Expression = { $_.'שם' } }
#{ Name = 'CheckCount__c'; Expression = { $_.'כמות' } }
#{ Name = 'Deal'; Expression = { $_.'עסקהוזה' } }
#{ Name = 'Amount__c'; Expression = { $_.'סכום' } }
#{ Name = 'Discount__c'; Expression = { $_.'ניסיון' } }
# And so on
)
$csv = Import-Csv C:\Users\alon\Documents\again.csv -Header "Customer__c","Name__c","Deal","Amount__c","CheckCount__c","Discount__c"
$csv | ForEach-Object {
if($_.Customer__c -eq "") { $_.Customer__c = "0" }
if($_.Name__c -eq "") { $_.Name__c = "0" }
if($_.Deal -eq "") { $_.Deal = "0" }
if($_.Amount__c -eq "") { $_.Amount__c = "0" }
if($_.Discount__c -eq "") { $_.Discount__c = "0" }
if($_.CheckCount__c -eq "") { $_.CheckCount__c = "0" }
}
Select-Object -Property $propertyTranslation
$csv | Export-Csv C:\Users\alon\Documents\CheckDealBeforeUpsert.csv -NoTypeInformation -Encoding UTF8
The term "ANSI" as it is used in Windows is basically an umbrella term for a number of encodings (or code pages). Usually it refers to the windows-1252 encoding. Your input file, however, appears to be encoded using the windows-1255 code page.
I'm not sure if in PowerShell -Encoding ASCII always means windows-1252 encoding, or if that is adjusted for localized Windows versions. If it's not adjusted you probably need to convert your input file to an encoding Import-Csv can handle before you can import and modify the data:
$inFile = 'C:\path\to\input.csv'
$outFile = 'C:\path\to\input_utf8.csv'
$reader = New-Object IO.StreamReader ($inFile, [Text.Encoding]::GetEncoding(1255))
$writer = New-Object IO.StreamWriter ($outFile, $false, [Text.Encoding]::UTF8)
while ($reader.Peek() -ge 0) {
$writer.WriteLine($reader.ReadLine())
}
$reader.Close(); $reader.Dispose()
$writer.Close(); $writer.Dispose()