Reference argument to a PowerShell class method? [Ref][Uint64] - powershell

I have a powershell class that I need pass in a reference to a UInt64 Variable and return a value from the Method. I'm trying to do something like in the code below but it gives me a syntax error.
Maybe I don't need to add ?[ref]? because all variables are references by default in powershell including uint64? but i just wasn't sure if this is true.... If I follow normal C-language convention then the method gets a copy of the argument not the actual reference to the variable. I seem to recall something in C# about boxing and unboxing int types... boxing and unboxing c# int type How does it work for boxing Int for Powershell class methods?
class PowerHe11 {
# Constructor
PowerHe11() {
}
[void]AccessChannel(
[UInt64]$ichan,
[UInt64]$idata,
[UInt64]$isize,
[UInt64]$ireal,
[ref][UInt64]$otimeout,
[ref][UInt64]$odata,
[ref][UInt64]$ochan,
[ref][UInt64]$osize
) {
$osize = 64; #Return this value to caller of method
}
}
Error message is:
At C:\Users\wmoore\Documents\fpga\zynq_pl\run_ps1\Untitled1.ps1:13 char:11
+ [ref][UInt64]$otimeout,
+ ~~~~~~~~
Multiple type constraints are not allowed on a method parameter.
At C:\Users\wmoore\Documents\fpga\zynq_pl\run_ps1\Untitled1.ps1:14 char:14
+ [ref][UInt64]$odata,
+ ~~~~~~~~
Multiple type constraints are not allowed on a method parameter.
At C:\Users\wmoore\Documents\fpga\zynq_pl\run_ps1\Untitled1.ps1:15 char:14
+ [ref][UInt64]$ochan,
+ ~~~~~~~~
Multiple type constraints are not allowed on a method parameter.
At C:\Users\wmoore\Documents\fpga\zynq_pl\run_ps1\Untitled1.ps1:16 char:14
+ [ref][UInt64]$osize
+ ~~~~~~~~
Multiple type constraints are not allowed on a method parameter.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : MultipleTypeConstraintsOnMethodParam

Your own answer shows the solution; let me add an explanation and background information:
about_Ref is the official help topic.
The short of it:
[ref]'s primary purpose is to support ref and out parameters in .NET methods; e.g., to call the System.Int32.TryParse method, use the following:
[int] $int = 0; [int]::TryParse('42', [ref] $int)
Note that the variable's type must match the type expected by the .NET method, and that the only way to create a variable in PowerShell is to also assign it a value (even if that value is irrelevant, as is the case with the out parameter at hand); the [ref] cast is required (analogous to the need to use ref or out in C#.
You may pass [ref] $null if you're not interested in the value being assigned by the method.
While you can use it in PowerShell code, doing so is awkward, as your example shows:
PowerShell code that receives a [ref] argument must refer to its value via the .Value property, as your answer shows.
You cannot type-constrain such a parameter, so you lose type safety.
Note that in function and scripts - as opposed to custom-class methods - it is syntactically allowed, but pointless to use both [ref] and a target data type (such as [ref][UInt64]$otimeout in your example), because the latter is effectively ignored; e.g.:
function foo { param([ref] [int] $p) $p.Value += '!' }; $bar = 'none'; foo ([ref] $bar); $bar
The call succeeds and $bar contains 'none!', which implies that the [int] type constraint was ignored.
Calling scripts and functions requires argument-mode syntax (shell-like, whitespace-separated arguments, bareword strings allowed), making the [ref]-cast invocation more awkward by requiring (...) around the argument, as shown above (foo ([ref] $bar)).
With respect to use in custom PowerShell classes, note that PowerShell's class support is - by design - not on par with that of OO-focused languages such as C#; that said, enhancements are planned; unfortunately, even the limited feature set still has problem as of PowerShell 7.1 - see GitHub issue #6652.
Technical background on [ref]:
[ref], unlike ref in C#, is not a language keyword: it is a type, namely [System.Management.Automation.PSReference] (whose type accelerator is [ref]).
As such, an expression such as [ref] $var is a cast that implicitly constructs a [ref] instance, which is required to bind to a [ref]-typed parameter.
Caveats:
There is parser magic involved in a [ref] cast, because you cannot directly use the normally equivalent [ref]::new($var) constructor expression as an alternative to cast [ref] $var.
PowerShell needs magic to know that $var refers to a variable object itself rather than to the variable's value, as is normally the case in an expression - and the latter is indeed what happens with [ref]::new($var) - $var's value is wrapped in a [ref] instance.
The true constructor equivalent of [ref] $var is [ref]::new((Get-Variable var))
# BROKEN: Do not use a constructor - [ref]::new($int) -
# in lieu of a cast - [ref] $int
PS> [int] $int = 0; $null = [int]::TryParse('42', [ref]::new($int)); $int
0 # !! $int was NOT updated, because its *value* was wrapped in a [ref] instance
It generally only makes sense to use a [ref] cast with a variable (object) operand.
PowerShell lets you pass any value as the operand, which technically works, but makes no sense unless you explicitly save the [ref] instance in a variable, pass it as the argument, and then use the [ref] instance's .Value property to access the updated value:
# Save a [ref] instance in a variable, pass it, then use .Value to get the
# updated value.
PS> [ref] $intRef = 0; $null = [int]::TryParse('42', $intRef); $intRef.Value
42

class powerhe11 {
# Constructor
powerhe11() {
}
[void]AccessChannel(
[ref]$ichan
) {
$ichan.value = 0xbeef;
Write-Host("ichan: {0}" -f $ichan.value)
}
}
[UInt64]$dude = 0
$ser = [gvmuart]::new()
$ser.AccessChannel([ref]$dude);
Write-Host -BackgroundColor Green "DUDE: $dude"

Related

Cannot find an overload for "ToLookup" and the argument count: "2" LINQ

I have a PowerShell script that runs every hour, it essentially updates product info through a drip process. The script is failing in the dev environment, but continues to work in prod and qual. The failure point is at a line that uses LINQ.
Here is the script:
$productInfos = [pscustomobject]#{
product_name = 'Television'
price = 1000
product_id = 101
} | ForEach-Object{$_}
$productIdDelegate = [Func[object,int]] {$args[0].product_id}
$productInfos = [Linq.Enumerable]::ToLookup($productInfos, $productIdDelegate)
Technically, it fails on the last line. but the second to last line is included because while researching this issue, I found a lot of other posts that pointed out if the two arguments in the ToLookup linq function are not of the same type, it can cause this issue. The second to last line was the "supposed" fix for many users, but sadly not myself. I still receive the
Cannot find an overload for "ToLookup" and the argument count: "2"
I checked any dependencies on all 3 environments, and everything is the same. And, the powershell versions are the same on all as well.
Even though the error doesn't give much details, from your currently reproducible code we can tell that the issue is because $productInfos is a single object of the type PSObject and this class does not implement IEnumerable Interface:
$productInfos = [pscustomobject]#{
product_name = 'Television'
price = 1000
product_id = 101
}
$productInfos -is [System.Collections.IEnumerable] # => False
To ensure $productInfos is always an enumberable you can use the Array subexpression operator #( ):
#($productInfos) -is [System.Collections.IEnumerable] # => True
Casting [object[]] or [array] should also be an option:
[object[]] $productInfos -is [System.Collections.IEnumerable] # => True
[array] $productInfos -is [System.Collections.IEnumerable] # => True
However above methods only work as long as $productInfos has at least 1 element, as opposed to #(..) which will always ensure an array:
The result is always an array of 0 or more objects.
To summarize:
$productInfos = [Linq.Enumerable]::ToLookup([object[]] $productInfos, $productIdDelegate)
$productInfos = [Linq.Enumerable]::ToLookup(#($productInfos), $productIdDelegate)
$productInfos = [Linq.Enumerable]::ToLookup([array] $productInfos, $productIdDelegate)
Are all valid options having what's explained above in consideration, if we're not sure if the variable will always be populated, the safest alternative will always be #(...):
# Returns Null instead of an error:
[Linq.Enumerable]::ToLookup(#(), $productIdDelegate)
A note up front:
... | ForEach-Object{ $_ }, with a single input object, as in your case, never outputs an array (unless the input object is an array as a whole, which is atypical), even if the input was a single-element array, because of the pipeline's enumeration behavior. In fact, the pipeline itself never creates arrays on output, it streams objects one by one, and it is only when that stream is captured that an array must be created - but only for two or more output objects; see this answer for background information.
Santiago Squarzon has provided the crucial pointer: wrapping the first argument passed to [System.Linq.Enumerable]::ToLookup[TSource, TKey]() in an [object[]] array allows PowerShell to find the right overload of and type arguments for this method:
$productInfos = [Linq.Enumerable]::ToLookup(
#($productInfos), # wrap $productInfos in an [object[]] array with #(...)
$productIdDelegate
)
Like all LINQ methods, ToLookup:
operates on enumerables (loosely speaking, collections of objects),
is a generic method, which means that it must be instantiated with the types it operates on (type arguments that bind to the method's type parameters).
PowerShell has to infer the type arguments in your call, because you're not providing the type arguments explicitly, the way you'd need to in C# (e.g., Array.Empty<int>())
Note: Prior to the upcoming verion 7.3 of PowerShell (Core), PowerShell didn't even support specifying type arguments explicitly - see next section.
The signature of the ToLookup overload you're targeting is (you can see this by executing [Linq.Enumerable]::ToLookup, i.e. without (), though the following has been simplified and reformatted for readability):
# TSource and TKey are the generic type parameters.
static ... ToLookup[TSource, TKey](
IEnumerable[TSource] source,
Func[TSource,TKey] keySelector
)
Since the argument you're passing to the keySelector parameter, $productIdDelegate, is an instance of [Func[object,int]], PowerShell can infer that the argument for type parameter TSource is [object], and for TKey it is [int].
Since TSource is [object], the argument you pass to the source parameter must be of type IEnumerable[object].
Your $productInfos value, due to being a single object, does not implement the IEnumerable[object] interface, but an [object[]]-typed array does.
Given that #(...), the array-subexpression operator, always returns an [object[]]-typed array, it is sufficient to use it to wrap $productInfos.
Note that had you used a specific type in lieu of [object], e.g. [Func[datetime, int]], #(...) would not work: you would then need a type-specific cast, such as [datetime[]] in this example.
Conversely, in your case you could have used [object[]] $productInfos (or its equivalent, [Array] $productInfos) as an alternative to #($productInfos).
See also:
This answer for an overview of using LINQ from PowerShell, which has its limitations, because PowerShell even in 7.3 won't support extension methods (yet?), which are necessary for a fluid experience.
Adding support for extension methods, and possibly also an equivalent of LINQ query syntx - is being discussed for a future version of PowerShell (Core) in GitHub issue #2226.
PowerShell 7.3+ alternative:
Versions 7.3+ have - optional - support for specifying type arguments explicitly, which is good news for two reasons:
The new syntax makes a given method call clearly communicate the type arguments it uses.
The new syntax makes it possible to directly call generic methods where the type arguments can not be inferred.
For instance, this applies to parameter-less generic methods, which previously required cumbersome workarounds; see this post
A simple example is the static System.Array.Empty[T] method, which you can now call as [Array]::Empty[int]()
If you apply this syntax to your call, you may omit casting your script block to [Func[object, int]], because this cast is implied by the explicit type parameters; using a simplified, self-contained version of your call:
# PS v7.3+ only
# Note the explicit type arguments ([object, int])
# and the use of the script block without a [Funct[object, int]] cast.
$lookup = [Linq.Enumerable]::ToLookup[object, int](
#([pscustomobject] #{ foo = 1 } , [pscustomobject] #{ foo = 2 }),
{ $args[0].foo }
)
$lookup[2] # -> [pscustomobject] #{ foo = 2 }
As in the case of inferred type arguments, if the first type argument is something other than [object], say [pscustomobject], you'd need a type-specific array cast (#(...) alone won't be enough):
# PS v7.3+ only
# Note the [pscustomobject] type argument and the
# [pscustomobject[]] cast.
$lookup = [Linq.Enumerable]::ToLookup[pscustomobject, int](
[pscustomobject[]] ([pscustomobject] #{ foo = 1 } , [pscustomobject] #{ foo = 2 }),
{ $args[0].foo }
)

PowerShell classes with single-argument constructors do not validate data type

I'm designing a module and using classes to type-validate my parameters. I noticed that, when attempting to type-validate input parameters, a class with a single-argument constructor appears to act as a type accelerator instead of validating data type.
Example:
Class stack {
$a
$b
stack($inp) {
$this.a = $inp
$this.b = 'anything'
}
}
function foo {
Param(
[stack]$bar
)
$bar
}
PS>foo -bar 'hello'
a b
- -
hello anything
$bar has been type accelerated into an instantiation of stack.
Compare this to the same class with a constructor that takes 2 arguments:
Class stack {
$a
$b
stack($inp,$inp2) {
$this.a = $inp
$this.b = 'anything'
}
}
function foo {
Param(
[stack]$bar
)
$bar
}
PS>foo -bar 'hello'
foo : Cannot process argument transformation on parameter 'bar'. Cannot convert the "hello" value of type "System.String" to type "stack".
Now the class type is correctly validating the input parameter.
I first saw this in PS5.1 on Windows 10, but I just tried it on my private laptop with pwsh 7.2.1 and seems to be the same.
Is there a workaround to this behavior? Is it a bug?
Edit: Well, after further testing, I realized this also happens if I supply 2 input parameters for the constructor with 2 arguments, e.g., foo -bar 'hello' 'world'. So I guess it's probably intended, and I'm doing something wrong. Can I use classes to validate my data types for input parameters? How?
What you're seeing is unrelated to type accelerators, which are simply short alias names for .NET type names; e.g., [regex] is short for [System.Text.RegularExpressions.Regex].
Instead, you're seeing PowerShell's flexible automatic type conversions, which include translating casts (e.g. [stack] ...) and type constraints (ditto, in the context of an assignment or inside a param(...) block) into constructor calls or ::Parse() calls, as explained in this answer.
Therefore, given that your [stack] class has a (non-type-constrained) single-argument constructor, something like [stack] 'hello' is automatically translated into [stack]::new('hello'), i.e. a constructor call - and that is also what happens when you pass argument 'hello' to a parameter whose type is [stack].
I suggest not fighting these automatic conversions, as they are usually helpful.
In the rare event that you do need to ensure that the type of the argument passed is exactly of the type specified in the parameter declaration (or of a derived type), you can use the following technique (using type [datetime] as an example, whose full .NET type name is System.DateTime):
function Foo {
param(
# Ensure that whatever argument is passed is already of type [datetime]
[PSTypeName('System.DateTime')]
$Bar
)
"[$Bar]"
}
Kudos to you for discovering the [PSTypeName()] attribute for this use case.
Without the [PSTypeName(...)] attribute, a call such as Foo 1/1/1970 would work, because the string '1/1/1970' is automatically converted to [datetime] by PowerShell.
With the [PSTypeName(...)] attribute, only an actual [datetime] argument is accepted (or, for types that can be sub-classed, an instance of a type derived from the specified type).
Important: Specify the target type's full .NET type name (e.g. 'System.DateTime' rather than just 'datetime') to target it unambiguously.
However, for PowerShell custom classes, their name is the full name (they are not inside a namespace), so in the case of your [stack] class, the attribute would be [PSTypeName('stack')]
Any type name is accepted, even if it doesn't refer to an existing .NET type or custom class, and any such non-existent type would require an argument to use a matching virtual ETS (PowerShell's Extended Type System) type name. In fact, supporting such virtual type names is the primary purpose of this attribute.[1] E.g., if [PSTypeName('Bar')] were used, you could pass a custom object with an ETS type name of Bar as follows:
[pscustomobject] #{ PSTypeName = 'Bar'; Baz = 'quux' }
[1] To quote from the linked docs (emphasis added): "This attribute is used to restrict the type name of the parameter, when the type goes beyond the .NET type system."
If you really want to validate the passed type you need to actually validate, not just cast the input as a specific type.
function foo {
Param(
[ValidateScript({$_ -is [stack]})]
$bar
)
$bar
}
Doing this will not try to cast the input as a specific type, and will fail if the input type is wrong.
Your 'foo' function requires a [stack] argument, it doesn't create one.
So your call should be as follow:
foo -bar ([stack]::new('fun:foo','hello'))
I don't know exactly how you will use it but if the goal is to validate arguments, I would suggest to specify the types everywhere... here is a small example but it can be improved, just an example:
Class stack {
[string]$a
[string]$b
stack([string]$inp,[string]$inp2) {
$this.a = $inp
$this.b = $inp2
}
}
function foo {
Param([stack]$bar)
$bar
}
function foo2 {
Param([array]$bar)
[stack]::new($bar[0],$bar[1])
}
foo -bar ([stack]::new('fun:foo','hello'))
foo2 -bar 'fun:foo2','hello'
foo2 -bar #('fun:foo2','hello2')
Aha, I thought I had seen something to this effect somewhere. I luckily managed to get it working by applying the description from an article on using the PSTypeName in PSCustomObjects for type-validation to classes. It turns out that classes also work with the same syntax.
In summary, it seems one has to type [PSTypeName('stack')] to use class types to validate data types.
Class stack {
$a
$b
stack($inp) {
$this.a = $inp
$this.b = 'anything'
}
}
function foo {
Param(
[PSTypeName('stack')]$bar
)
$bar
}
PS>foo -bar 'hello'
foo : Cannot bind argument to parameter 'bar', because PSTypeNames of the argument do not match the PSTypeName required by the parameter: stack.
PS>$test = [stack]::new('Overflow')
PS>foo -bar $test
a b
- -
Overflow anything

Powershell arguments by reference: implied vs explicit

Powershell supports arguments by reference two different ways. Any variable can be passed by reference using [Ref] as the type, or basically I guess casting to the special [Ref] type. Alternatively, complex data types like [Hashtable] are always passed by reference. The former approach then requires using .value when either changing the value or passing the variable to another class method or function. In simplified form this is what that looks like in practice.
# implicit By Reference
class One_i {
static [Void] Act ([System.Collections.Generic.List[string]] $data) {
$data.Add("One_i::Act(): $(Get-Date)")
[Two_i]::Act($data)
}
}
class Two_i {
static [Void] Act ([System.Collections.Generic.List[string]] $data) {
$data.Add("Two_i::Act(): $(Get-Date)")
}
}
# explicit By Reference
class One_e {
static [Void] Act ([Ref] $data) {
$data.value.Add("One_e::Act(): $(Get-Date)")
[Two_e]::Act($data.value)
}
}
class Two_e {
static [Void] Act ([Ref] $data) {
$data.value.Add("Two_e::Act(): $(Get-Date)")
}
}
CLS
$data_i = [System.Collections.Generic.List[string]]::new()
$data_i.Add("Init_i: $(Get-Date)")
[One_i]::Act($data_i)
$data_i.Add("Finalize_i: $(Get-Date)")
foreach ($item in $data_i) {
Write-Host "$item"
}
Write-Host
$data_e = [System.Collections.Generic.List[string]]::new()
$data_e.Add("Init_e: $(Get-Date)")
[One_e]::Act($data_e)
$data_e.Add("Finalize_e: $(Get-Date)")
foreach ($item in $data_e) {
Write-Host "$item"
}
Write-Host
I am conflicted as to which approach I prefer. On the one hand [Ref] makes it clear which variables are By Ref, which is useful. However, the need for .value makes the code a bit harder to read, adds work, and creates an opportunity to forget and induce a bug. I tested performance like this...
(measure-Command {
$data_i = [System.Collections.Generic.List[string]]::new()
$data_i.Add("Init_i: $(Get-Date)")
foreach ($i in 1..1000) {
[One_i]::Act($data_i)
}
$data_i.Add("Finalize_i: $(Get-Date)")
}).TotalSeconds
(measure-Command {
$data_e = [System.Collections.Generic.List[string]]::new()
$data_e.Add("Init_e: $(Get-Date)")
foreach ($e in 1..1000) {
[One_e]::Act($data_e)
}
$data_e.Add("Finalize_e: $(Get-Date)")
}).TotalSeconds
and implicit seems to be VERY slightly faster, but not enough to make an argument for implicit based on performance alone. This leads me to a number of questions as I try to decide which approach to adopt, or perhaps the realization that each approach is the better choice in certain situations and using both in a single program actually has merit.
1: Why are complex types handled differently than simple types? The inconsistent behavior seems odd to me, and I suspect I will learn something fundament if I understand why the behavior is different.
2: Beyond needing to use .value or not, are there other differences between implicit and explicit By Reference? Especially are there any potential problems with one approach or the other?
3: Is there a mechanism to force a complex type to be By Value? And what would be the use case for that?
4: I found this thread, with a comment saying it's best to avoid By Reference (but without explanation). However, I have also found references to Dependency Injection being superior to Global variables and/or the Singleton pattern, and By Reference is fundamental to DI. Is there annoying important I need to be aware of here, ESPECIALLY as it relates to PowerShell/Classes/Dependency Injection, given that PowerShell classes are not as fully implemented as with other languages. For what it's worth I am working in PS 5.1 and will not be moving to Core any time soon, but it would be great to know if there are fundamental differences between Core and pre Core implementations.
EDIT: Based on #mclayton's answer and my understanding of "Changing value" vs "mutation" I tried "breaking" the reference like this
# implicit By Reference
class One_i {
static [Void] Act ([System.Collections.Generic.List[string]] $data) {
$data.Add("One_i::Act(): $(Get-Date)")
[Two_i]::Act($data)
$data = [System.Collections.Generic.List[string]]::new()
}
}
# explicit By Reference
class One_e {
static [Void] Act ([Ref] $data) {
$data.value.Add("One_e::Act(): $(Get-Date)")
[Two_e]::Act($data.value)
$data.value = [System.Collections.Generic.List[string]]::new()
}
}
I tried both $data.value = [System.Collections.Generic.List[string]]::new() and $data = [System.Collections.Generic.List[string]]::new() in the "explicit" example, and none of theme actually seemed to cause an issue. The console output was "correct" in all three cases. Which suggests that I DON'T understand the difference between changing value and mutation after all.
That said, as it relates to my initial reason for asking the question, namely deciding which of the two approaches I want to use, I am leaning towards what I call "implicit", since it does make the code a bit simpler by eliminating the need for .value, and not having the by reference nature of the variable be obvious doesn't change the fact that I have to understand that it is by reference, no matter what I do. I really wish I could find a thread that discusses the relative merits of the two approaches, but thus far my searches have failed to turn up anything useful.
Edit2: So, I found this where mklement0 says there really is no functional difference in the two approaches. And it occurred to me that I could get the best of both worlds by using [Ref] and then assigning $ref.value to a local variable, to simplify things within the method or function. Tentatively feeling like that is the best answer. It add one line at the top of each function or method, but makes the calls more obvious in behavior as well as simplifying use in the body of the method or function.
# explicit By Reference with sugar
class One_x {
static [Void] Act ([Ref] $data) {
$localData = $data.Value
$localData.Add("One_x::Act(): $(Get-Date)")
[Two_e]::Act(([Ref]$localData))
}
}
class Two_x {
static [Void] Act ([Ref] $data) {
$localData = $data.Value
$localData.Add("Two_x::Act(): $(Get-Date)")
}
}
tl;dr
Here's the key differences between value types, reference types, passing by value and passing by reference:
Pass by value
Pass by reference
Value type
Assignments to parameters inside a function are not persisted to outer variables
Assignments to parameters inside a function are persisted to outer variables
Reference type
Assignments to parameters inside a function are not persisted to outer variablesMutations to object inside function are visible outside function
Assignments to parameters inside a function are persisted to outer variablesMutations to object inside function are visible outside function
Loooooooong version
First off, I think the documentation for [ref] is a bit misleading, despite improvements to it in the past (see about_Ref and Improve the about_Ref topic to clarify its primary purpose).
The main problem is there's a subtle difference between "Value types vs Reference types" and "Passing by value vs Passing by reference" - the about_Ref documentation touches on it, but sort of blurs it all into one big single concept.
Value Types vs Reference Types
Ignoring [ref] for a moment, there are two fundamental types in the .Net world, and there are some similarities and differences in the way they're passed as parameters:
Value types - https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/value-type
A variable of a value type contains an instance of the type. This differs from a variable of a reference type, which contains a reference to an instance of the type. By default, on assignment, passing an argument to a method, and returning a method result, variable values are copied. In the case of value-type variables, the corresponding type instances are copied.
Reference types - https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/reference-types
With reference types, two variables can reference the same object; therefore, operations on one variable can affect the object referenced by the other variable.
The default behaviour for both value types and reference types is to pass "by value" - that is, parameters passed into a method are a copy of the variable value from the parent scope. For value types this means a full copy of the entire value (including for struct types), and for reference types it means passing in a duplicate reference to the same underlying object. Assigning a new value or object reference to the parameter within the method won't persist beyond the scope of the method because it only affects the parameter, which is a copy of the outer value-type or reference-type variable.
To use examples from about_Ref:
In the following example, the function changes the value of the variable passed to it. In PowerShell, integers are value types so they are passed by value. Therefore, the value of $var is unchanged outside the scope of the function.
(Note - this should probably say "the function changes the value of the parameter passed to it", since it doesn't actually change the value of the outer variable)
Function Test($data)
{
$data = 3
}
$var = 10
Test -data $var
$var
# 3 # no change
When $var is a value-type, assigning a new value to $data inside the function makes no difference to the value of $var outside the function because the function is passed a "copy" of the value-type variable and any changes only affect the copy.
A similar example for reference types that shows the $var variable is also unchanged after the function finishes:
Function Test($data)
{
$data = #{ "Test" = "New Text" }
}
$var = #{ "xxx" = "yyy"}
Test -data $var
$var
# #{ "xxx" = "yyy" } # no change
Again, $data is a duplicate reference to the same reference-type object as in the $var variable - they're independent references even though they point to the same object, so assigning a new value to $data doesn't affect $var.
Note however, that about_Ref gives this example as well:
In the following example, a variable containing a Hashtable is passed to a function. Hashtable is an object type so by default it is passed to the function by reference.
When passing a variable by reference, the function can change the data and that change persists after the function executes.
This is a bit misleading though because:
A hashtable is not an "object type", it's a "reference type"
The parameter isn't being "passed to the function by reference" - it's actually a reference-type variable being passed into the function by value!
Function Test($data)
{
$data.Test = "New Text"
}
$var = #{}
Test -data $var
$var
# #{ "Test" = "New Test" }
To clarify, this example isn't changing the value of the $data parameter (i.e. which object it points to) - it's simply calling an instance method on the reference-type object the parameter points to, and that method mutates the object. Since the $var variable in the outer scope is pointing to the same reference-type object, changes to $data in the function are "visible" to $var outside the function because they're both referring to the same object in memory.
It's basically equivalent to this after PowerShell has resolved the appropriate type accelerators:
Function Test($data)
{
$data.Add("Test", "New Text")
}
$var = #{}
Test -data $var
$var
# #{ "Test" = "New Text" }
And that's the important difference between value types and reference types - we can mutate reference types inside a function and the changes will be visible in the outer variable when the function exists because even though the parameter and the variable are independent references, they refer to the same object, so mutations made via the function parameter are visible from the outer variable.
Summary: By default, assigning a value to a parameter inside a function doesn't affect what is assigned to the variable in the parent scope, regardless of whether the parent variable is a value type or reference type. Reference type objects can be mutated within the function by calling methods on the object referred to by the parameter, and the effect is visible in the variable outside the function because they're pointing to the same reference-type object.
Passing by value vs passing by reference
When you pass a variable by value you get the behaviour described above in "Value Types vs Reference Types" - that is, assignments to a parameter inside a function do not persist back into the variable in the parent scope.
By contrast, when you pass a variable by reference you can assign a new value-type or reference-type value to the variable in the parent scope:
Passing Parameters - https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/passing-parameters
Passing by reference enables function members, methods, properties, indexers, operators, and constructors to change the value of the parameters and have that change persist in the calling environment.
And about_Ref says:
You can code your functions to take a parameter as a reference, regardless of the type of data passed.
Although it should probably say "You can code your functions to pass a parameter by reference" to avoid confusion with "reference-type parameters".
The example it gives is a value-type parameter passed by reference:
Function Test([ref]$data)
{
$data.Value = 3
}
$var = 10
Test -data ([ref]$var)
$var
# 3
Note that here, changes to $data.Value inside Test persist into $var after the function call because the variable is passed by reference.
Similarly for reference types passed by reference:
Function Test([ref]$data)
{
$data.Value = #{ "Test" = "New Text" }
}
$var = #{ "xxx" = "yyy" }
Test -data ([ref]$var)
$var
# #{ "Test" = "New Text" } # new value
In this case, we've not mutated the original hashtable - we've simply assigned a whole different hashtable to $var. We can prove this with a couple more lines of code:
$new = #{ "xxx" = "yyy" }
$old = $new
Test -data ([ref]$new)
$new
# #{ "Test" = "New Text" } # new value
$old
# #{ "xxx" = "yyy" } # old value is still there unchanged
$new and $old both pointed to the same hashtable before the function call, but afterwards $new points to a completely different object.
Summary: When passing parameters by reference, assigning a new value to a parameter inside a function also assigns the new value to the variable in the parent scope, regardless of whether the parent variable is a value type or reference type.
Grand Summary
Bringing it right back to your original question, you can hopefully see now that your two examples aren't really equivalent:
class One_i {
static [Void] Act ([System.Collections.Generic.List[string]] $data) {
$data.Add("One_i::Act(): $(Get-Date)")
[Two_i]::Act($data)
}
}
class One_e {
static [Void] Act ([Ref] $data) {
$data.value.Add("One_e::Act(): $(Get-Date)")
[Two_e]::Act($data.value)
}
}
There's no "implicit" and "explicit" [ref] as such - both approaches simply mutate the contents of the $data reference-type parameter in-place but (if it wanted to) One_e could potentially change the object referred to by $data_e which would be persisted into $var after the function call ends. To prove this, try adding this at the end of your two methods $data.Value = #{ "xxx" = "yyy" } and see which version persists the change back up to $data_i or $data_e (you can probably guess One_e does :-).

Powershell: coerce or cast to type as named in a string variable

I am trying to use the method outlined here to create a number of custom data types, and rather than having a line for each I would like to define a hash table of the names and types like this
$pxAccelerators = #{
pxListObject = '[System.Collections.Generic.List[Object]]'
pxListString = '[System.Collections.Generic.List[String]]'
pxOrderedDictionary = '[System.Collections.Specialized.OrderedDictionary]'
}
Then I could use something like this
$typeAccelerators = [PowerShell].Assembly.GetType("System.Management.Automation.TypeAccelerators")
foreach ($key in $pxAccelerators.Keys) {
$name = $key
$type = $pxAccelerators.$key
$typeAccelerators::Add($key,$type)
}
to loop through the hash table and add each one.
However, the issue of course is that $type isn't an actual type, it's a string. And $typeAccelerators::Add($key,$type) needs a string and an actual type. So basically I need to coerce a string like '[System.Collections.Specialized.OrderedDictionary]' to the actual type. I have found plenty of references to casting or coercing from one data type to another, but I can't seem to find any reference to how to convert a string to a type as defined BY the string.
I have tried all these stabs in the dark
([System.Type]'[System.Collections.ArrayList]')::new()
[System.Type]'[System.Collections.ArrayList]'
[System.Type]'[System.Collections.ArrayList]' -as [System.Type]
'[System.Collections.ArrayList]' -as ([PowerShell].Assembly.GetType('[System.Collections.ArrayList]')
to no avail.
$type = ([PowerShell].Assembly.GetType('[System.Collections.ArrayList]')) seems to work, in that it doesn't throw an exception. But $type.GetType() does throw You cannot call a method on a null-valued expression..
Interestingly, auto completion with [PowerShell].Assembly.GetType('[System.Collections.ArrayList]'). shows properties like BaseType and FullName are available, suggesting that I have actually produced a type, but using .GetType() on the result throws an exception.
I tried
$pxAccelerators = #{
pxListObject = 'System.Collections.Generic.List[Object]'
pxListString = 'System.Collections.Generic.List[String]'
pxOrderedDictionary = 'System.Collections.Specialized.OrderedDictionary'
}
$typeAccelerators = [PowerShell].Assembly.GetType("System.Management.Automation.TypeAccelerators")
foreach ($key in $pxAccelerators.Keys) {
$name = $key
$type = [PowerShell].Assembly.GetType($pxAccelerators.$key)
$typeAccelerators::Add($key,$type)
}
[PSObject].Assembly.GetType("System.Management.Automation.TypeAccelerators")::Get
and the accelerators are being added, but the type to be acclerated is not there, suggesting that the GetType() line is not actually producing a type.
Lastly, I found this which seems to be getting closer. But I can't seem to rock how to access the method without starting from some sort of type already, and [System.Type].GetType('System.Int32') is throwing, so that seems to be a dead end.
Am I trying to do something impossible? Or just missing the proper mechanism?
-as [type] will do
The -as operator gladly takes a type name as it's right-hand side operand.
Change the values of your dictionary to contain just a valid type name, and it becomes as easy as:
$pxAccelerators = #{
pxListObject = 'System.Collections.Generic.List[Object]'
pxListString = 'System.Collections.Generic.List[String]'
pxOrderedDictionary = 'System.Collections.Specialized.OrderedDictionary'
}
$typeAccelerators = [PowerShell].Assembly.GetType("System.Management.Automation.TypeAccelerators")
foreach ($acc in $pxAccelerators.GetEnumerator()) {
$name = $acc.Key
$type = $acc.Value -as [type]
$typeAccelerators::Add($name,$type)
}
Result:
PS ~> [pxOrderedDictionary] -is [type]
True
PS ~> [pxOrderedDictionary].FullName
System.Collections.Specialized.OrderedDictionary
Ultimately, your only problem was that you accidentally enclosed your type-name string values in [...] (e.g, '[System.Collections.ArrayList]'), whereas this notation only works in - unquoted - type literals:
[...] is a PowerShell-specific notation for type literals (e.g., [System.DateTime]); .NET methods do not recognize this, although the string inside [...] (e.g., System.DateTime) uses a superset of the language-agnostic notation recognized by .NET methods such as System.Type.GetType() - see bottom section.
In your case, you don't even need explicit conversion of your type-name strings to [type] objects, because PowerShell implicitly performs this conversion when you call the ::Add() method. Therefore, the following is sufficient:
#{
pxListObject = 'System.Collections.Generic.List[Object]'
pxListString = 'System.Collections.Generic.List[String]'
pxOrderedDictionary = 'System.Collections.Specialized.OrderedDictionary'
}.GetEnumerator() | ForEach-Object {
# Calling ::Add() implicitly converts the type-name *string* to the required
# [type] instance.
[powershell].Assembly.GetType('System.Management.Automation.TypeAccelerators')::Add(
$_.Key,
$_.Value
)
}
Of course, you could have directly used type literals as your hashtable values:
# Note the use of [...] values (type literals) instead of strings.
#{
pxListObject = [System.Collections.Generic.List[Object]]
pxListString = [System.Collections.Generic.List[String]]
pxOrderedDictionary = [System.Collections.Specialized.OrderedDictionary]
}.GetEnumerator() | ForEach-Object {
[powershell].Assembly.GetType('System.Management.Automation.TypeAccelerators')::Add(
$_.Key,
$_.Value
)
}
Caveat:
Type System.Management.Automation.TypeAccelerators is non-public (hence the need to access it via reflection), and relying on non-public APIs is problematic.
The alternative, available in PowerShell v5+, is to use the using namespace statement so you don't have to specify a full type name (see the bottom section for an example) - though note that it invariably makes all types in the specified namespace available by simple name (no namespace part).
As for what you tried:
$type = [PowerShell].Assembly.GetType('[System.Collections.ArrayList]') seems to work, in that it doesn't throw an exception.
As stated, the outer [...] are not a part of the type name; however, removing them wouldn't be enough here, because the specified type isn't part of the same assembly as [PowerShell].
Generally, System.Type.GetType() quietly returns $null if the specified type cannot be found; e.g.: [PowerShell].Assembly.GetType('DefinitelyNoSuchType')
[PowerShell].Assembly.GetType($pxAccelerators.$key)
suggesting that the GetType() line is not actually producing a type.
The reason is that System.Type.GetType() doesn't recognize closed generic types such as System.Collections.Generic.List[String], because the underlying notation only supports open ones (such as System.Collections.Generic.List`1); the former notation is a PowerShell-specific extension to the notation - see the bottom section.
Converting type names stored in strings - without the enclosing [...] - to System.Type ([type]) objects:
Cast to [type]:
# Same as: [System.Collections.Generic.List[string]]; case does not matter.
[type] 'System.Collections.Generic.List[string]'
Casting to [type] does more than calling System.Type.GetType(), because PowerShell's type-name strings support a superset of the latter's notation - see the bottom section.
The above will cause a statement-terminating error (exception) if no such type is found (or the syntax is incorrect); note that only types among currently loaded assemblies are found.
Alternatively, you can use 'System.Collections.Generic.List[string]' -as [type], as shown in Mathias R. Jessen's answer, which quietly returns $null if conversion to a type isn't possible; see -as, the conditional type conversion operator.
Note that both -as and -is also accept type-name strings as their RHS; e.g. 42 -is 'int' or '42' -as 'int'
Also, the New-Object cmdlet accepts strings as its -TypeName argument, so again the type-literal-enclosing [...] must not be used; e.g.,
New-Object -TypeName System.Text.UTF8Encoding
PowerShell's type-literal and type-name notation:
A PowerShell type literal (e.g., [System.DateTime]) is a type name enclosed in [...], but the [ and ] are not part of the name.
Type literals are themselves instance of System.Type ([type]), or, more accurately, instances of the non-public [System.RuntimeType] type, which derives from System.Reflection.TypeInfo, which in turn derives from System.Type.
Type-name strings can generally also be used where type literals (objects representing types) are accepted, with PowerShell converting to [type] implicitly, notably during parameter binding and on the RHS of -as, the conditional type-conversion operator and -is, the type(-inheritance) / interface test operator.
PowerShell implements a superset of the language-agnostic type-name notation used by .NET:
The language-agnostic .NET notation, as used by System.Type.GetType(), for instance, is documented in Specifying fully qualified type names, and notably requires and includes:
Type names must be (at least) namespace-qualified names (e.g., System.Int32 rather than just Int32), and must be specified case-exactly by default; however, an overload with an ignoreCase parameter is available; names can optionally be assembly-qualified so as to also indicate the type's assembly of origin.
Suffix `<num> indicates the arity of generic types, i.e, the number (<num>) of type arguments the generic type requires (e.g, `1 for a type with one generic type parameter, such as System.Collections.Generic.List`1).
Using only a generic type's arity returns a generic type definition.
A generic type definition cannot be directly instantiated; instead, you must call the .MakeGenericType() instance method on the type object returned and pass (closed) type arguments for all generic type parameters, which then returns a closed [constructed] generic type, i.e. a concrete instantiation of the generic type definition from which instances can be constructed.
If you optionally follow the generic arity specifier with a list of type arguments to bind all type parameters defined by the generic type, a constructed generic type is directly returned; if all type arguments are closed themselves, transitively, i.e. have no unbound type parameters, if applicable, the type returned is a closed [constructed] type, which can directly be instantiated (its constructors can be called). The list of type arguments has the form [<type>] or, for multiple type arguments, [<type>, ...]; <type> references are subject to the same rules as the enclosing generic type's name, and may additionally be enclosed in [...] individually.
For instance, the following two type literals, which construct a closed Dictionary<TKey,TValue> type (to use C# notation for a change) with [string] keys and [int] values, are therefore equivalent:
System.Collections.Generic.Dictionary`2[System.String, System.Int32]
System.Collections.Generic.Dictionary`2[[System.String], [System.Int32]]
Suffixes [] / [,] indicate a single-dimensional / two-dimensional array; e.g., System.Int32[].
+ is used to separate nested types from their containing class; e.g., System.Environment+SpecialFolder.
The PowerShell-specific extensions are:
Name matching is invariably case-insensitive; e.g., type System.Text.UTF8Encoding can be targeted as [system.text.utf8encoding].
You may omit the System. component of the full name; e.g., [Text.UTF8Encoding]
You may use the names of PowerShell's type accelerators; e.g., [regex].
In PowerShell v5+ you can also use the using namespace statement so you don't have to specify a full type name:
E.g., placing using namespace System.Text at the top of a script or script module then allows you to refer to [System.Text.UTF8Encoding] as just [UTF8Encoding]
When you pass a list of generic type arguments, you may omit the type's arity specifier.
For instance, the following two type literals, which construct a closed Dictionary<TKey,TValue> type with [string] keys and [int] values, are therefore equivalent:
[System.Collections.Generic.Dictionary[string, int]]
[System.Collections.Generic.Dictionary`2[string, int]]
Example Use of Script Block to Solve the Problem
Maybe I was doing something wrong, but for some reason this dynamically built string didn't seem to like any of the official answers here:
[System.Collections.Generic.HashSet[System.Action[string, object]]]::new()
I gave up and did the following, perhaps this will be useful to someone else:
. ([ScriptBlock]::Create(#"
param(`$MyClass)
`$MyClass | Add-Member -NotePropertyName '$HashSetName' -NotePropertyValue ([System.Collections.Generic.HashSet$EventDef]::new())
"#)) -MyClass $this

How do I call a parameterless generic method from Powershell v3?

For example I have a .NET object $m with the following method overloads:
PS C:\Users\Me> $m.GetBody
OverloadDefinitions
-------------------
T GetBody[T]()
T GetBody[T](System.Runtime.Serialization.XmlObjectSerializer serializer)
If I try to invoke the parameterless method I get:
PS C:\Users\Me> $m.GetBody()
Cannot find an overload for "GetBody" and the argument count: "0".
At line:1 char:1
+ $m.GetBody()
+ ~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodException
+ FullyQualifiedErrorId : MethodCountCouldNotFindBest
I understand PowerShell v3.0 is supposed to work more easily with generics. Obviously I need to tell it somehow what type I want returned but I cannot figure out the syntax.
It looks like you are trying to invoke a generic method.
In powershell this can be done by:
$nonGenericClass = New-Object NonGenericClass
$method = [NonGenericClass].GetMethod("SimpleGenericMethod")
$gMethod = $method.MakeGenericMethod([string])
# replace [string] with the type you want to use for T.
$gMethod.Invoke($nonGenericClass, "Welcome!")
See this wonderful blog post for more info and additional examples.
For your example you could try:
$Source = #"
public class TestClass
{
public T Test<T>()
{
return default(T);
}
public int X;
}
"#
Add-Type -TypeDefinition $Source -Language CSharp
$obj = New-Object TestClass
$Type = $obj.GetType();
$m = $Type.GetMethod("Test")
$g = new-object system.Guid
$gType = $g.GetType()
$gm = $m.MakeGenericMethod($gType)
$out = $gm.Invoke( $obj, $null)
#$out will be the default GUID (all zeros)
This can be simplified by doing:
$Type.GetMethod("Test").MakeGenericMethod($gType).Invoke( $obj, $null)
This has been testing in powershell 2 and powershell 3.
If you had a more detailed example of how you came across this generic method I would be able to give more details. I have yet to see any microsoft cmdlets return anything that give you generic methods. The only time this comes up is when custom objects or methods from c# or vb.net are used.
To use this without any parameters you can use Invoke with just the first parameter.
$gMethod.Invoke($nonGenericClass)
Calling a generic method on an object instance:
$instance.GetType().GetMethod('MethodName').MakeGenericMethod([TargetType]).Invoke($instance, $parameters)
Calling a static generic method (see also Calling generic static method in PowerShell):
[ClassType].GetMethod('MethodName').MakeGenericMethod([TargetType]).Invoke($null, $parameters)
Note that you will encounter an AmbiguousMatchException when there is also a non-generic version of the method (see How do I distinguish between generic and non generic signatures using GetMethod in .NET?). Use GetMethods() then:
([ClassType].GetMethods() | where {$_.Name -eq "MethodName" -and $_.IsGenericMethod})[0].MakeGenericMethod([TargetType]).Invoke($null, $parameters)
(Mind that there could be more than one method that match the above filter, so make sure to adjust it to find the one you need.)
Hint: You can write complex generic type literals like this (see Generic type of Generic Type in Powershell):
[System.Collections.Generic.Dictionary[int,string[]]]
To call a (parameterless) generic method with overloads from Powershell v3, as shown in the OP example, use the script Invoke-GenericMethod.ps1 from the reference provided by #Chad Carisch, Invoking Generic Methods on Non-Generic Classes in PowerShell.
It should look something like
Invoke-GenericMethod $m GetBody T #()
This is a verified working code sample that I am using:
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Practices.ServiceLocation") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Practices.SharePoint.Common") | Out-Null
$serviceLocator = [Microsoft.Practices.SharePoint.Common.ServiceLocation.SharePointServiceLocator]::GetCurrent()
# Want the PowerShell equivalent of the following C#
# config = serviceLocator.GetInstance<IConfigManager>();
# Cannot find an overload for "GetInstance" and the argument count: "0".
#$config = $serviceLocator.GetInstance()
# Exception calling "GetMethod" with "1" argument(s): "Ambiguous match found."
#$config = $serviceLocator.GetType().GetMethod("GetInstance").MakeGenericMethod([IConfigManager]).Invoke($serviceLocator)
# Correct - using Invoke-GenericMethod
$config = C:\Projects\SPG2013\Main\Scripts\Invoke-GenericMethod $serviceLocator GetInstance Microsoft.Practices.SharePoint.Common.Configuration.IConfigManager #()
$config.CanAccessFarmConfig
Here is an alternate script that I haven't tried but is more recent and being actively maintained, Invoke Generic Methods from PowerShell.
Update: PowerShell (Core) 7.3+ now does support calling
generic methods with explicit type arguments.
Note: As of this writing, only preview versions of 7.3 are available.
# PS v7.3+ only; using [string] as an example type argument.
$m.GetBody[string]()
See the conceptual about_Calling_Generic_Methods help topic
PowerShell (Core) 7.2- and Windows PowerShell:
marsze's helpful answer contains great general information about calling generic methods, but let me address the aspect of calling a parameter-less one specifically, as asked:
As hinted at in the question:
in PSv3+ PowerShell can infer the type from the parameter values (arguments) passed to a generic method,
which by definition cannot work with a parameter-less generic method, because there is nothing to infer the type from.
Prior to PowerShell (Core) 7.3, PowerShell previously had no syntax that would allow you to specify the type explicitly in this scenario.
In such older versions, reflection must be used:
# Invoke $m.GetBody[T]() with [T] instantiated with type [decimal]
$m.GetType().GetMethod('GetBody', [type[]] #()).
MakeGenericMethod([decimal]).
Invoke($m, #())
.GetMethod('GetBody', [type[]] #()) unambiguously finds the parameter-less overload of.GetBody(), due to passing in an empty array of parameter types.
.MakeGenericMethod([decimal]) instantiates the method with example type [decimal].
.Invoke($m, #()) then invokes the type-instantiated method on input object ($m) with no arguments (#(), the empty array).