I am reading Swift Evolution proposal 244 (Opaque Result Types) and don't understand what the following means:
... existential type ...
One could compose these transformations by
using the existential type Shape instead of generic arguments, but
doing so would imply more dynamism and runtime overhead than may be
desired.
An example of an existential type is given in the evolution proposal itself:
protocol Shape {
func draw(to: Surface)
}
An example of using protocol Shape as an existential type would look like
func collides(with: any Shape) -> Bool
as opposed to using a generic argument Other:
func collides<Other: Shape>(with: Other) -> Bool
Important to note here that the Shape protocol is not an existential type by itself, only using it in "protocols-as-types" context as above "creates" an existential type from it. See this post from the member of Swift Core Team:
Also, protocols currently do double-duty as the spelling for existential types, but this relationship has been a common source of confusion.
Also, citing the Swift Generics Evolution article (I recommend reading the whole thing, which explains this in more details):
The best way to distinguish a protocol type from an existential type is to look at the context. Ask yourself: when I see a reference to a protocol name like Shape, is it appearing at a type level, or at a value level? Revisiting some earlier examples, we see:
func addShape<T: Shape>() -> T
// Here, Shape appears at the type level, and so is referencing the protocol type
var shape: any Shape = Rectangle()
// Here, Shape appears at the value level, and so creates an existential type
Deeper dive
Why is it called an "existential"? I never saw an unambiguous confirmation of this, but I assume that the feature is inspired by languages with more advanced type systems, e.g. consider Haskell's existential types:
class Buffer -- declaration of type class `Buffer` follows here
data Worker x y = forall b. Buffer b => Worker {
buffer :: b,
input :: x,
output :: y
}
which is roughly equivalent to this Swift snippet (if we assume that Swift's protocols more or less represent Haskell's type classes):
protocol Buffer {}
struct Worker<X, Y> {
let buffer: any Buffer
let input: X
let output: Y
}
Note that the Haskell example used forall quantifier here. You could read this as "for all types that conform to the Buffer type class ("protocol" in Swift) values of type Worker would have exactly the same types as long as their X and Y type parameters are the same". Thus, given
extension String: Buffer {}
extension Data: Buffer {}
values Worker(buffer: "", input: 5, output: "five") and Worker(buffer: Data(), input: 5, output: "five") would have exactly the same types.
This is a powerful feature, which allows things such as heterogenous collections, and can be used in a lot more places where you need to "erase" an original type of a value and "hide" it under an existential type. Like all powerful features it can be abused and can make code less type-safe and/or less performant, so should be used with care.
If you want even a deeper dive, check out Protocols with Associated Types (PATs), which currently can't be used as existentials for various reasons. There are also a few Generalized Existentials proposals being pitched more or less regularly, but nothing concrete as of Swift 5.3. In fact, the original Opaque Result Types proposal linked by the OP can solve some of the problems caused by use of PATs and significantly alleviates lack of generalized existentials in Swift.
I'm sure you've already used existentials a lot before without noticing it.
A summarized answer of Max is that for:
var rec: Shape = Rectangle() // Example A
only Shape properties can be accessed. While for:
func addShape<T: Shape>() -> T // Example B
Any property of T can be accessed. Because T adopts Shape then all properties of Shape can also be accessed as well.
The first example is an existential the second is not.
Example of real code:
protocol Shape {
var width: Double { get }
var height: Double { get }
}
struct Rectangle: Shape {
var width: Double
var height: Double
var area: Double
}
let rec1: Shape = Rectangle(width: 1, height: 2, area: 2)
rec1.area // ❌
However:
let rec2 = Rectangle(width: 1, height: 2, area: 2)
func addShape<T: Shape>(_ shape: T) -> T {
print(type(of: shape)) // Rectangle
return shape
}
let rec3 = addShape(rec2)
print(rec3.area) // ✅
I'd argue that for most Swift users, we all understand Abstract class and Concrete class. This extra jargon makes it slightly confusing.
The trickiness is that with the 2nd example, to the compiler, the type you return isn't Shape, it's Rectangle i.e. the function signature transforms to this:
func addShape(_ shape: Rectangle) -> Rectangle {
This is only possible because of (constrained) generics.
Yet for rec: Shape = Whatever() to the compiler the type is Shape regardless of the assigning type. <-- Box Type
Why is it named Existential?
The term "existential" in computer science and programming is borrowed from philosophy, where it refers to the concept of existence and being. In the context of programming, "existential" is used to describe a type that represents the existence of any specific type, without specifying which type it is.
The term is used to reflect the idea that, by wrapping a value in an existential type, you are abstracting away its specific type and only acknowledging its existence.
In other words, an existential type provides a way to handle values of different types in a unified way, while ignoring their specific type information†. This allows you to work with values in a more generic and flexible manner, which can be useful in many situations, such as when creating collections of heterogeneous values, or when working with values of unknown or dynamic types.
The other day I took my kid to an ice-cream shop. She asked what are you having, and I didn't know the flavor I picked, so I didn't say it's strawberry flavored or chocolate, I just said "I'm having an ice-cream".
I just specified that it's an ice-cream without saying its flavor. My daughter could no longer determine if it was Red, or Brown. If it was having a fruit flavor or not. I gave her existential-like information.
Had I told her it's a chocolate, then I would have gave her specific information. Which is then not existential.
†: In Example B, we're not ignoring the specific type information.
Special thanks to a friend who helped me come up with this answer
I feel like it's worth adding something about why the phrase is important in Swift. And in particular, I think almost always, Swift is talking about "existential containers". They talk about "existential types", but only really with reference to "stuff that is stored in an existential container". So what is an "existential container"?
As I see it, the key thing is, if you have a variable that you're passing as a parameter or using locally, etc. and you define the type of the variable as Shape then Swift has to do some things under the hood to make it work and that's what they are (obliquely) referring to.
If you think about defining a function in a library/framework module that you're writing that is publicly available and takes for example the parameters public func myFunction(shape1: Shape, shape2: Shape, shape1Rotation: CGFloat?) -> Shape... imagine it (optionally) rotates shape1, "adds" it to shape2 somehow (I leave the details up to your imagination) then returns the result. Coming from other OO languages, we instinctively think that we understand how this works... the function must be implemented only with members available in the Shape protocol.
But the question is for the compiler, how are the parameters represented in memory? Instinctively, again, we think... it doesn't matter. When someone writes a new program that uses your function at some point in the future, they decide to pass their own shapes in and define them as class Dinosaur: Shape and class CupCake: Shape. As part of defining those new classes, they will have to write implementations of all the methods in protocol Shape, which might be something like func getPointsIterator() -> Iterator<CGPoint>. That works just fine for classes. The calling code defines those classes, instantiates objects from them, passes them into your function. Your function must have something like a vtable (I think Swift calls it a witness table) for the Shape protocol that says "if you give me an instance of a Shape object, I can tell you exactly where to find the address of the getPointsIterator function". The instance pointer will point to a block of memory on the stack, the start of which is a pointer to the class metadata (vtables, witness tables, etc.) So the compiler can reason about how to find any given method implementation.
But what about value types? Structs and enums can have just about any format in memory, from a one byte Bool to a 500 byte complex nested struct. These are usually passed on the stack or registers on function calls for efficiency. (When Swift exactly knows the type, all code can be compiled knowing the data format and passed on the stack or in registers, etc.)
Now you can see the problem. How can Swift compile the function myFunction so it will work with any possible future value type/struct defined in any code? As I understand it, this is where "existential containers" come in.
The simplest approach would be that any function that takes parameters of one of these "existential types" (types defined just by conforming to a Protocol) must insist that the calling code "box" the value type... that it store the value in a special reference counted "box" on the heap and pass a pointer to this (with all the usual ARC retain/release/autorelease/ownership rules) to your function when the function takes a parameter of type Shape.
Then when a new, weird and wonderful, type is written by some code author in the future, compiling the methods of Shape would have to include a way to accept "boxed" versions of the type. Your myFunction would always handle these "existential types" by handling the box and everything works. I would guess that C# and Java do something like this (boxing) if they have the same problem with non class types (Int, etc.)?
The thing is that for a lot of value types, this can be very inefficient. After all, we are compiling mostly for 64 bit architecture, so a couple of registers can handle 8 bytes, enough for many simple structures. So Swift came up with a compromise (again I might be a bit inaccurate on this, I'm giving my idea of the mechanism... feel free to correct). They created "existential containers" that are always 4 pointers in size. 16 bytes on a "normal" 64 bit architecture (most CPUs that run Swift these days).
If you define a struct that conforms to a protocol and it contains 12 bytes or less, then it is stored in the existential container directly. The last 4 byte pointer is a pointer to the type information/witness tables/etc. so that myFunction can find an address for any function in the Shape protocol (just like in the classes case above). If your struct/enum is larger than 12 bytes then the 4 pointer value points to a boxed version of the value type. Obviously this was considered an optimum compromise, and seems reasonable... it will be passed around in 4 registers in most cases or 4 stack slots if "spilled".
I think the reason the Swift team end up mentioning "existential containers" to the wider community is because it then has implications for various ways of using Swift. One obvious implication is performance. There's a sudden performance drop when using functions in this way if the structs are > 12 bytes in size.
Another, more fundamental, implication I think is that protocols can be used as parameters only if they don't have protocol or Self requirements... they are not generic. Otherwise you're into generic function definitions which is different. That's why we sometimes need to change things like: func myFunction(shape: Shape, reflection: Bool) -> Shape into something like func myFunction<S:Shape>(shape: S, reflection: Bool) -> S. They are implemented in very different ways under the hood.
I have a function with following syntax:
func myfunc<V1,V2>(to: (V1)->V2, from: (V2)->V1)
and as you see, I'm providing 2 closures to this function. These 2 closures are used for converting V1 to V2 and vice versa. Remember that I cannot provide these conversions as extension to V1 or V2. I like to improve function usability and I want to know what is more common and better approach for providing these 2 as a Swift user.
I have though of 3 ways, each with its own pros and cons.
1st approach: Use current syntax.
func myfunc<V1,V2>(to: (V1)->V2, from: (V2)->V1)
Pros: User provides 2 closure, so he can organize code better. In addition, he can provide them in-place.
Cons: Function call will become long and can only use trailing closure for second closure.
2nd approach: Use a single closure for both and distinguish them through a paramether.
enum ConvertClosure {
case .to(Any)
case .from(Any)
}
func myfunc<V1,V2>(operation: (ConvertClosure)->Any)
Pros: Function call becomes simpler and we can use trailing closure too.
Cons: Responsibility of 2 closures are now in a single one, so it becomes more complex. Cannot add generic type checking and need to use Any for enum case argument and return type of function.
3rd approach: Use protocol rather than closure.
protocol ConvertOperation {
associatedtype InVal
associatedtype OutVal
func to(_ val: InVal) -> OutVal
func from(_ val: OutVal) -> InVal
}
func myfunc<V1,V2>(operation: ConvertOperation)
where ConvertOperation.InVal == V1, ConvertOperation.OutVal == V2
Pros: Function call becomes simpler. We have type checking from generics.
Cons: We need to conform to protocol which cannot be done in place. Approach is not using closure, so it may not be very Swifty.
Which method is more suitable for a Swift user? Can you suggest any better method?
This might be a bit opinion based but the general rule is:
Use the most specific type as possible. You want types to catch bugs for you during compilation.
Use protocols when you want to reuse the functionality. A protocol will require you to use a struct/class that will implement it, therefore making it harder to use. Therefore it's better when you already have an object that can conform to that protocol or when the functionality is reused, therefore you can create an object and use it multiple times.
To comment more on your cases:
You should not be using trailing closures if there is more than one closure parameter because that affects readability. However, this is not a bad solution and it's pretty common.
This is generally a bad solution because it uses Any type which breaks rule 1 above. Always minimize usage of Any. Most applications should not use it at all.
This is not a bad solution, however see rule 2. I will be optimal only in specific use cases.
Also consider NSObject is basically the same as Any. In general you should avoid its usage in Swift code because you are basically resigning on type checking.
Also note that you should not aim for code to be as short as possible. Always aim for readability.
We cannot probably give a more specific advise without knowing the exact use case.
I've seen some code that uses constraints like <T: Any> and can't find the difference in the docs between that and not specifying a constraint. Is there any difference at all, like restricting to non optionals?
The constraint is indeed redundant, as all types are subtypes of Any (including Optional).
Internally, the compiler actually models the type Any as being a protocol composition type (e.g P1 & P2) consisting of zero protocols. There's no way to spell this in the language, which is why Any isn't defined in the standard library anymore1, it's just a keyword that's parsed as a type.
So the constraint T : Any is literally interpreted as "T must conform to all of the protocols in this empty list of protocols", which is quite evidently a redundant constraint. Really the compiler ought to warn on it (I have actually started working on a patch to do so – aiming to open a pull request sometime this week all things going well).
1. Any used to be defined in the standard library as a typealias for protocol<>, when protocol compositions were spelt protocol<P1, P2> rather than P1 & P2.
I've noticed a lot of swift built ins take or return Ints and not UInts:
Here are some examples from Array:
mutating func reserveCapacity(minimumCapacity: Int)
var capacity: Int { get }
init(count: Int, repeatedValue: T)
mutating func removeAtIndex(index: Int) -> T
Given that the language is completely new, and assuming that this design choice was not arbitrary - I'm wondering: Why do swift built ins take Ints and not UInts?
Some notes: Asking because I'm working on a few collections myself and I'm wondering what types I should use for things like reserveCapacity etc. What I'd naturally expect is for reserveCapacity to take a UInt instead.
UInt is a common cause of bugs. It is very easy to accidentally generate a -1 and wind up with infinite loops or similar problems. (Many C and C++ programmers have learned the hard way that you really should just use int unless there's a need for unsigned.) Because of how Swift manages type conversion, this is even more important. If you've ever worked with NSUInteger with "signed-to-unsigned" warnings turned on (which are an error in Swift, not an optional warning like in C), you know what a pain that is.
The Swift Programming Guide explains their specific rationale in the section on UInt:
NOTE
Use UInt only when you specifically need an unsigned integer type with the same size as the platform’s native word size. If this is not the case, Int is preferred, even when the values to be stored are known to be non-negative. A consistent use of Int for integer values aids code interoperability, avoids the need to convert between different number types, and matches integer type inference, as described in Type Safety and Type Inference.
Here is a possible explanation (I am no expert on this subject): Suppose you have this code
let x = 3
test(x)
func test(t: Int) {
}
This will compile without a problem, since the type of 'x' is inferred to be Int.
However, if you change the function to
func test(t: UInt) {
}
The compiler will give you a build error ('Int' is not convertible to 'UInt')
So my guess is that it is just for convenience, because Swift's type safety would otherwise require you to convert them manually each time.
I'm wanting to make the Functor type in Swift as a protocol, but I'm having trouble with the type clauses.
protocol Functor {
typealias A
func fmap<
B, C
where
C == Self
C.A == B>
(A -> B) -> C
}
The problem is that C == Self sort of implies that C is the same as Self, and therefore has the same A. Is there any way to define a generic that is the same type but with a different type parameter?
EDIT: Maybe I should clarify that the goal here is to function like fmap in other functional languages, except self is the in parameter, instead of fmap being a global function that takes an in parameter.
No, not today. Swift lacks higher-kinded types. It doesn't even have support for first-order types. So as a first step, you'd want to be able to talk about let x: Array<_> where you didn't yet know the type of the contained object. Swift can't handle that. It can't handle generic closures, either. For example:
func firstT<T>(xs: [T]) -> T {
return xs[0]
}
let ft = firstT // "Argument for generic parameter 'T' could not be inferred"
Those are the simple cases, and Swift's type system already starts falling apart (it can handle it as a function definition, but not as a type). Swift's type system completely falls apart when asked to represent higher-kinded types like Functor.
A little deeper discussion of the issue (but mostly reiterates what I've said here: https://gist.github.com/rnapier/5a84f65886e1ee40f62e)
Note that this isn't specifically a feature so much of "other functional languages." First, Swift isn't really a functional language; it just has some functional features, and second, not all functional languages have this feature (Haskell and Scala, yes, ML and LISP, no). The feature you want is higher-kinded types, and that can exist with or without other functional features.
A nice intro to this is available in section 4.3 Higher-kinded types of Scala for generic programmers. Swift has some very interesting similarities to Scala, so the history of Scala can provide some hints about what might make sense for Swift (Higher-kinded types didn't show up until Scala 2.5).