This is a follow up to this question
If I have 2 json files
file1.json
{
"foo": {
"honk": 42
}
}
file2.json
{
"foo": {
"honk": 9000,
"toot": 9000
}
}
And I create an object using ConvertFrom-Json
$bar = #(Get-ChildItem . -Filter *.json -Recurse | Get-Content -Raw |ConvertFrom-Json)
Powershell will happily take both, and overwrite foo.
foo
---
#{honk=42}
#{honk=9000; toot=9000}
The contents of $bar.foo are merged
$bar.foo
honk
----
42
9000
How can I error if importing duplicate objects?
Each JSON file is imported as a separate object, so there's nothing overwritten really. You just get a list of objects.
To throw an error when you get multiple objects with the same top-level property you can group the objects by property name and throw an error if you get a count >1.
$bar | Group-Object { $_.PSObject.Properties.Name } |
Where-Object { $_.Count -gt 1 } |
ForEach-Object { throw "Duplicate object $($_.Name)" }
When importing to an array, every object is unique. In this example it isn't ideal to leave the objects in an array, since there is no way to predictably iterate over them, since some objects might contain multiple keys.
{
"foo": 42
}
vs
{
"bar": 9000,
"buzz": 9000
}
This will cause heartache when trying to loop through all objects.
Instead, I took all array items and combined them into 1 powershell object. Since powershell objects are basically hashes, and hashes by design must have all keys unique, powershell will automatically error if overwriting a key.
function Load-Servers {
$allObjects = #(
Get-ChildItem '.\servers' -Filter *.json -Recurse | Get-Content -Raw | ConvertFrom-Json
)
$object = New-Object PSObject
Foreach ($o in $allObjects) {
$o.psobject.members | ? {$_.Membertype -eq "noteproperty" } | %{$object | add-member $_.Name $_.Value }
}
return $object
}
Related
I keep running into the same problem again, and i have my default way of handling it, but it keeps bugging me.
Isn't there any better way?
So basicly i have a pipline running, do stuff within the pipline, and want to return a Key/Value Pair from within the pipline.
I want the whole pipline to return a object of type psobject (or pscustomobject).
Here is the way i do it everytime.
I create a hashtable at the beginning of the pipline and add key/Value Pairs from within the pipline to this hashtable using the .Add() method.
Afterwards i create a psobject by passing the hashtbale to New-Object`s -Property Parameter. This gives me the desired result.
Get-Process | Sort -Unique Name | ForEach-Object -Begin { $ht = #{} } -Process {
# DO STUFF
$key = $_.Name
$val = $_.Id
# Add Entry to Hashtable
$ht.Add($key,$val)
}
# Create PSObject from Hashtable
$myAwesomeNewObject = New-Object psobject -Property $ht
# Done - returns System.Management.Automation.PSCustomObject
$myAwesomeNewObject.GetType().FullName
But this seems a bit cluncky, isn't there a more elegant way of doing it?
Something like this:
[PSObject]$myAwesomeNewObject = Get-Process | Sort -Unique Name | ForEach-Object -Process {
# DO STUFF
$key = $_.Name
$val = $_.Id
# return Key/Val Pair
#{$key=$val}
}
# Failed - returns System.Object[]
$myAwesomeNewObject.GetType().FullName
This unfortunally dosn't work, since the pipe returns an array of hashtables, but i hope you know now what iam trying to achieve.
Thanks
Not sure if this is more elegant but just another way of doing it, this uses an anonymous function so $ht will no longer be available after execution, and casts to [pscustomobject] instead of using New-Object:
[pscustomobject] (Get-Process | Sort -Unique Name | & {
begin { $ht = #{ } }
process {
# DO STUFF
$key = $_.Name
$val = $_.Id
# Add Entry to Hashtable
$ht.Add($key, $val)
}
end { $ht }
})
You can also use the -End parameter to convert the final hash table to a pscustomobject as part of the pipeline, without needing to set the whole thing to a variable
$ht[$key]=$val is also a nice shorthand for $ht.Add($key,$val):
Get-Process |
Sort -Unique Name |
Foreach -Begin { $ht = #{} } -Process {
$ht[$_.Name] = $_.Id
} -End {[pscustomobject]$ht} |
## continue pipeline with pscustomobject
Thanks to #Santiago Squarzon and #Cpt.Whale answers, i were able to combine them to create a solution that pleases me:
$myAwesomeNewObject = `
Get-Process | Sort -Unique Name | & {
begin { $ht = #{} }
process {
# DO STUFF
$key = $_.Name
$val = $_.Id
# Add Entry to Hashtable
$ht[$key]=$val
}
end {[pscustomobject]$ht}
}
# Success - System.Management.Automation.PSCustomObject
$myAwesomeNewObject.Gettype().FullName
# And helper Hashtable is NULL thanks to the
# anonym function
$null -eq $ht
Thanks alot Guys
Alternatively you may create a hashtable using Group-Object -AsHashTable:
# Store the PIDs of all processes into a PSCustomObject, keyed by the process name
$processes = [PSCustomObject] (Get-Process -PV proc |
Select-Object -Expand Id |
Group-Object { $proc.Name } -AsHashtable)
# List all PIDs of given process
$processes.chrome
Notes:
Common parameter -PV (alias of -PipelineVariable) makes sure that we can still access the full process object from within the calculated property of the Group-Object command, despite that we have a Select-Object command in between.
The values of the properties are arrays, which store the process IDs of all instances of each process. E. g. $processes.chrome outputs a list of PIDs of all instances of the chrome process.
I'm using Powershell to set up IIS bindings on a web server, and having a problem with the following code:
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
if ($serverIps.length -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}
$primaryIp = $serverIps[0]
$secondaryIp = $serverIps[1]
If there's 2+ IPs on the server, fine - Powershell returns an array, and I can query the array length and extract the first and second addresses just fine.
Problem is - if there's only one IP, Powershell doesn't return a one-element array, it returns the IP address (as a string, like "192.168.0.100") - the string has a .length property, it's greater than 1, so the test passes, and I end up with the first two characters in the string, instead of the first two IP addresses in the collection.
How can I either force Powershell to return a one-element collection, or alternatively determine whether the returned "thing" is an object rather than a collection?
Define the variable as an array in one of two ways...
Wrap your piped commands in parentheses with an # at the beginning:
$serverIps = #(gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort)
Specify the data type of the variable as an array:
[array]$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
Or, check the data type of the variable...
IF ($ServerIps -isnot [array])
{ <error message> }
ELSE
{ <proceed> }
Force the result to an Array so you could have a Count property. Single objects (scalar) do not have a Count property. Strings have a length property so you might get false results, use the Count property:
if (#($serverIps).Count -le 1)...
By the way, instead of using a wildcard that can also match strings, use the -as operator:
[array]$serverIps = gwmi Win32_NetworkAdapterConfiguration -filter "IPEnabled=TRUE" | Select-Object -ExpandProperty IPAddress | Where-Object {($_ -as [ipaddress]).AddressFamily -eq 'InterNetwork'}
You can either add a comma(,) before return list like return ,$list or cast it [Array] or [YourType[]] at where you tend to use the list.
If you declare the variable as an array ahead of time, you can add elements to it - even if it is just one...
This should work...
$serverIps = #()
gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort | ForEach-Object{$serverIps += $_}
You can use Measure-Object to get the actual object count, without resorting to an object's Count property.
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
if (($serverIps | Measure).Count -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}
Return as a referenced object, so it never converted while passing.
return #{ Value = #("single data") }
I had this problem passing an array to an Azure deployment template. If there was one object, PowerShell "converted" it to a string. In the example below, $a is returned from a function that gets VM objected according to the value of a tag. I pass the $a to the New-AzureRmResourceGroupDeployment cmdlet by wrapping it in #(). Like so:
$TemplateParameterObject=#{
VMObject=#($a)
}
New-AzureRmResourceGroupDeployment -ResourceGroupName $RG -Name "TestVmByRole" -Mode Incremental -DeploymentDebugLogLevel All -TemplateFile $templatePath -TemplateParameterObject $TemplateParameterObject -verbose
VMObject is one of the template's parameters.
Might not be the most technical / robust way to do it, but it's enough for Azure.
Update
Well the above did work. I've tried all the above and some, but the only way I have managed to pass $vmObject as an array, compatible with the deployment template, with one element is as follows (I expect MS have been playing again (this was a report and fixed bug in 2015)):
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
foreach($vmObject in $vmObjects)
{
#$vmTemplateObject = $vmObject
$asJson = (ConvertTo-Json -InputObject $vmObject -Depth 10 -Verbose) #-replace '\s',''
$DeserializedJson = (New-Object -TypeName System.Web.Script.Serialization.JavaScriptSerializer -Property #{MaxJsonLength=67108864}).DeserializeObject($asJson)
}
$vmObjects is the output of Get-AzureRmVM.
I pass $DeserializedJson to the deployment template' parameter (of type array).
For reference, the lovely error New-AzureRmResourceGroupDeployment throws is
"The template output '{output_name}' is not valid: The language expression property 'Microsoft.WindowsAzure.ResourceStack.Frontdoor.Expression.Expressions.JTokenExpression'
can't be evaluated.."
There is a way to deal with your situation. Leave most of you code as-is, just change the way to deal with the $serverIps object. This code can deal with $null, only one item, and many items.
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
# Always use ".Count" instead of ".Length".
# This works on $null, only one item, or many items.
if ($serverIps.Count -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}
# Always use foreach on a array-possible object, so that
# you don't have deal with this issue anymore.
$serverIps | foreach {
# The $serverIps could be $null. Even $null can loop once.
# So we need to skip the $null condition.
if ($_ -ne $null) {
# Get the index of the array.
# The #($serverIps) make sure it must be an array.
$idx = #($serverIps).IndexOf($item)
if ($idx -eq 0) { $primaryIp = $_ }
if ($idx -eq 1) { $secondaryIp = $_ }
}
}
In PowerShell Core, there is a .Count property exists on every objects. In Windows PowerShell, there are "almost" every object has an .Count property.
I have an $array of PSCustomObjects which contain a path,days,filter and recurse property
I Test-Path the path of each PSCustomObject and if it's false, I save only the path in another variable like $failpath
Now I want to remove all Objects inside $array when the path is inside $failpath
I tried things like the .remove() method for the $array, but that doesn't work and gave me this error (example pic from web): https://i0.wp.com/www.sapien.com/blog/wp-content/uploads/2014/11/image8.png
So I tried creating a new array, but it's giving me a hard time because I don't know how to iterate over the failpaths correctly. so that each correct objects only gets sent to the new array once (when I tried it, the correct object was there multiple times) - i can't show you the code for this because I already edited it too many times and now it's just a mess.
this is how $array and $faultypath look like
$array = #(
[pscustomobject]#{
path = "\\server\daten\Alle Adressen\Dokumente 70"
filter = "*.pdf"
days = "90"
recurse = "false"
}
[pscustomobject]#{
path = "\\server\Tobit\itacom\ERP2UMS"
filter = "*.fax"
days = "7"
recurse = "false"
}
)
[string[]]$faultypath = #()
$pfade | % { if (!(Test-Path $_.path)) { $faultypath += $_.path } }
How can I substract everything which is in $faultypath from $array?
For PowerShell 3 or higher
$faultyPath = $pfade | Where-Object { -not (Test-Path $_.Path) } | ForEach-Object Path
$array | Where-Object Path -notin $faultyPath
For PowerShell 2 or lower
$faultyPath = $pfade | Where-Object { -not (Test-Path $_.Path) } | ForEach-Object { $_.Path }
$array | Where-Object { $faultyPath -notcontains $_.Path }
This is potentially an expensive array comparison if both sets are large. In that case dictionaries or hashtables will provide better performance for the comparison.
I'm importing a csv-file which looks like this:
id,value1.1,value1.2,value1.3,Value2.1,Value2.2,Value3.1,Value3.2
row1,v1.1,,v1.3
row2,,,,v2.1,v2.2
row3,,,,,,,v3.2
Now I want to check, if any of the value-properties in one group is set.
I can do
Import-Csv .\test.csv | where {$_.Value1.1 -or $_.Value1.2 -or $_.Value1.3}
or
Import-Csv .\test.csv | foreach {
if ($_.Value1 -or $_.Value2 -or $_.Value3) {
Write-Output $_
}
}
But my "real" csv-file contains about 200 columns and I have to check 31 properties x 5 different object types that are mixed up in this csv. So my code will be realy ugly.
Is there anything like
where {$_.Value1.*}
or
where {$ArrayWithPropertyNames}
?
You could easily use the Get-Member cmdlet to get the properties which have the correct prefix (just use * as a wildcard after the prefix).
So to achieve what you want you could just filter the data based on whether any of the properties with the correct prefix contains data.
The script below uses your sample data, with a row4 added, and filters the list to find all items which have a value in any property starting with value1.
$csv = #"
id,value1.1,value1.2,value1.3,Value2.1,Value2.2,Value3.1,Value3.2
row1,v1.1,,v1.3
row2,,,,v2.1,v2.2
row3,,,,,,,v3.2
row4,v1.1,,v1.3
"#
$data = ConvertFrom-csv $csv
$data | Where {
$currentDataItem = $_
$propertyValues = $currentDataItem |
# Get's all the properties with the correct prefix
Get-Member 'value1*' -MemberType NoteProperty |
# Gets the values for each of those properties
Foreach { $currentDataItem.($_.Name) } |
# Only keep the property value if it has a value
Where { $_ }
# Could just return $propertyValues, but this makes the intention clearer
$hasValueOnPrefixedProperty = $propertyValues.Length -gt 0
Write-Output $hasValueOnPrefixedProperty
}
Alternate solution:
$PropsToCheck = 'Value1*'
Import-csv .\test.csv |
Where {
(($_ | Select $PropsToCheck).psobject.properties.value) -contains ''
}
I'm using Powershell to set up IIS bindings on a web server, and having a problem with the following code:
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
if ($serverIps.length -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}
$primaryIp = $serverIps[0]
$secondaryIp = $serverIps[1]
If there's 2+ IPs on the server, fine - Powershell returns an array, and I can query the array length and extract the first and second addresses just fine.
Problem is - if there's only one IP, Powershell doesn't return a one-element array, it returns the IP address (as a string, like "192.168.0.100") - the string has a .length property, it's greater than 1, so the test passes, and I end up with the first two characters in the string, instead of the first two IP addresses in the collection.
How can I either force Powershell to return a one-element collection, or alternatively determine whether the returned "thing" is an object rather than a collection?
Define the variable as an array in one of two ways...
Wrap your piped commands in parentheses with an # at the beginning:
$serverIps = #(gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort)
Specify the data type of the variable as an array:
[array]$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
Or, check the data type of the variable...
IF ($ServerIps -isnot [array])
{ <error message> }
ELSE
{ <proceed> }
Force the result to an Array so you could have a Count property. Single objects (scalar) do not have a Count property. Strings have a length property so you might get false results, use the Count property:
if (#($serverIps).Count -le 1)...
By the way, instead of using a wildcard that can also match strings, use the -as operator:
[array]$serverIps = gwmi Win32_NetworkAdapterConfiguration -filter "IPEnabled=TRUE" | Select-Object -ExpandProperty IPAddress | Where-Object {($_ -as [ipaddress]).AddressFamily -eq 'InterNetwork'}
You can either add a comma(,) before return list like return ,$list or cast it [Array] or [YourType[]] at where you tend to use the list.
If you declare the variable as an array ahead of time, you can add elements to it - even if it is just one...
This should work...
$serverIps = #()
gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort | ForEach-Object{$serverIps += $_}
You can use Measure-Object to get the actual object count, without resorting to an object's Count property.
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
if (($serverIps | Measure).Count -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}
Return as a referenced object, so it never converted while passing.
return #{ Value = #("single data") }
I had this problem passing an array to an Azure deployment template. If there was one object, PowerShell "converted" it to a string. In the example below, $a is returned from a function that gets VM objected according to the value of a tag. I pass the $a to the New-AzureRmResourceGroupDeployment cmdlet by wrapping it in #(). Like so:
$TemplateParameterObject=#{
VMObject=#($a)
}
New-AzureRmResourceGroupDeployment -ResourceGroupName $RG -Name "TestVmByRole" -Mode Incremental -DeploymentDebugLogLevel All -TemplateFile $templatePath -TemplateParameterObject $TemplateParameterObject -verbose
VMObject is one of the template's parameters.
Might not be the most technical / robust way to do it, but it's enough for Azure.
Update
Well the above did work. I've tried all the above and some, but the only way I have managed to pass $vmObject as an array, compatible with the deployment template, with one element is as follows (I expect MS have been playing again (this was a report and fixed bug in 2015)):
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
foreach($vmObject in $vmObjects)
{
#$vmTemplateObject = $vmObject
$asJson = (ConvertTo-Json -InputObject $vmObject -Depth 10 -Verbose) #-replace '\s',''
$DeserializedJson = (New-Object -TypeName System.Web.Script.Serialization.JavaScriptSerializer -Property #{MaxJsonLength=67108864}).DeserializeObject($asJson)
}
$vmObjects is the output of Get-AzureRmVM.
I pass $DeserializedJson to the deployment template' parameter (of type array).
For reference, the lovely error New-AzureRmResourceGroupDeployment throws is
"The template output '{output_name}' is not valid: The language expression property 'Microsoft.WindowsAzure.ResourceStack.Frontdoor.Expression.Expressions.JTokenExpression'
can't be evaluated.."
There is a way to deal with your situation. Leave most of you code as-is, just change the way to deal with the $serverIps object. This code can deal with $null, only one item, and many items.
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
# Always use ".Count" instead of ".Length".
# This works on $null, only one item, or many items.
if ($serverIps.Count -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}
# Always use foreach on a array-possible object, so that
# you don't have deal with this issue anymore.
$serverIps | foreach {
# The $serverIps could be $null. Even $null can loop once.
# So we need to skip the $null condition.
if ($_ -ne $null) {
# Get the index of the array.
# The #($serverIps) make sure it must be an array.
$idx = #($serverIps).IndexOf($item)
if ($idx -eq 0) { $primaryIp = $_ }
if ($idx -eq 1) { $secondaryIp = $_ }
}
}
In PowerShell Core, there is a .Count property exists on every objects. In Windows PowerShell, there are "almost" every object has an .Count property.