I'm learning Swift with Paul Hudson's 100 days of Swift. In one of his extension lessons, I have found a notion that more advanced developers could write a Sequence extension that would service Arrays, Sets and Dictionaries:
https://www.hackingwithswift.com/quick-start/understanding-swift/when-are-protocol-extensions-useful-in-swift
I have given this a shot but:
I don't know how to create a variable that could change its type (wonder if that's even possible at all)
I don't know how to create a sequence that would service dictionaries too, since their syntax for allSatisfy is a bit different
Would you be so kind and give me a hand here? :)
The code:
extension Sequence {
var isAllEven:Bool {
numbers.allSatisfy { $0.isMultiple(of:2)}
}
}
let numbers = Set([4, 8, 15, 16])
print(numbers.isAllEven)
I can change numbers to be both Array and Set but as soon as I understood what Paul said, there is a possibility to create an extension that could service all 3 in one passage of code without having to change the variables content.
As isMultiple(of:) belongs to all integer types a generic version must be constrained to BinaryInteger
extension Sequence where Element : BinaryInteger {
var isAllEven : Bool {
allSatisfy {$0.isMultiple(of: 2)}
}
}
But this cannot cover Dictionary, because although Dictionary conforms to Sequence the Element type is different.
You could write a second extension of Sequence which matches the Dictionary tuple type
extension Sequence where Element == (key:String, value:Int) {
var isAllEven : Bool {
allSatisfy {$0.value.isMultiple(of:2)}
}
}
but this considers only String keys and Int values
A more generic way is to extend Dictionary directly
extension Dictionary where Value : BinaryInteger {
var isAllEven : Bool {
allSatisfy {$0.value.isMultiple(of: 2)}
}
}
Related
As part of a custom Encoder, I am coding an UnkeyedEncodingContainer. However, the specific format I am making it for asks that all elements of an array be of the same type. Specifically, arrays can contain :
Integers one same size
Floats or Doubles
Other arrays (not necessarily all containing the same kinds of elements)
Objects
Here is the type of answer I need : The basis of an UnkeyedEncodingContainer implementation that conforms to the protocol, and enforces that all elements be of one same type among the above specified ones.
As requested, here are examples of things that should or should not be encodable :
var valid1 = []
var valid2 = [3, 3, 5, 9]
var valid3 = ["string", "array"]
var invalid1 = [3, "test"]
var invalid2 = [5, []]
var invalid3 = [[3, 5], {"hello" : 3}]
// These may not all even be valid Swift arrays, they are only
// intended as examples
As an example, here is the best I have come up with, which does not work :
The UnkeyedEncodingContainer contains a function, checkCanEncode, and an instance variable, ElementType :
var elementType : ElementType {
if self.count == 0 {
return .None
} else {
return self.storage[0].containedType
}
}
func checkCanEncode(_ value : Any?, compatibleElementTypes : [ElementType]) throws {
guard compatibleElementTypes.contains(self.elementType) || self.elementType == .None else {
let context = EncodingError.Context(
codingPath: self.nestedCodingPath,
debugDescription: "Cannot encode value to an array of \(self.elementType)s"
)
throw EncodingError.invalidValue(value as Any, context)
}
}
// I know the .None is weird and could be replaced by an optional,
// but it is useful as its rawValue is 0. The Encoder has to encode
// the rawValue of the ElementType at some point, so using an optional
// would actually be more complicated
Everything is then encoded as a contained singleValueContainer :
func encode<T>(_ value: T) throws where T : Encodable {
let container = self.nestedSingleValueContainer()
try container.encode(value)
try checkCanEncode(value, compatibleElementTypes: [container.containedType])
}
// containedType is an instance variable of SingleValueContainer that is set
// when a value is encoded into it
But this causes an issue when it comes to nestedContainer and nestedUnkeyedContainer : (used for stored dictionaries and arrays respectively)
// This violates the protocol, this function should not be able to throw
func nestedContainer<NestedKey>(keyedBy keyType: NestedKey.Type) throws -> KeyedEncodingContainer<NestedKey> where NestedKey : CodingKey {
let container = KeyedContainer<NestedKey>(
codingPath: self.nestedCodingPath,
userInfo: self.userInfo
)
try checkCanEncode(container, compatibleElementTypes: [.Dictionary])
self.storage.append(container)
return KeyedEncodingContainer(container)
}
As you can see, since I need checkCanEncode to know whether it is even possible to create a NestedContainer in the first place (because if the array already has stuff inside that aren't dictionaries, then adding dictionaries to it is invalid), I have to make the function throw. But this breaks the UnkeyedEncodingContainer protocol which demands non-throwing versions.
But I can't just handle the error inside the function ! If something tries to put an array inside an array of integers, it must fail. Therefore this is an invalid solution.
Additional remarks :
Checking after having encoded the values already feels sketchy, but checking only when producing the final encoded payload is definitely a violation of the "No Zombies" principle (fail as soon as the program enters an invalid state) which I would rather avoid. However if no better solution is possible I may accept it as a last resort.
One other solution I have thought about is encoding the array as a dictionary with numbered keys, since dictionaries in this format may contain mixed types. However this is likely to pose decoding issues, so once again, it is a last resort.
You will be advised not to edit other people’s questions. If you have edits to suggest please do so in the comments, otherwise mind your own business
Unless anyone has a better idea, here is the best I could come up with :
Do not enforce that all elements be of the same type inside the UnkeyedEncodingContainer
If all elements are the same type, encode it as an array
If elements have varying types, encode it as a dictionary with integers as keys
This is completely fine as far as the encoding format goes, has minimal costs and only slightly complicates decoding (check whether keys contain integers) and greatly widens how many different Swift object will be compatible with the format.
Note : Remember that the "real" encoding step where the data is generated is not actually part of the protocol. That is where I am proposing the shenanigans should take place 😈
I don't know whether this helps you but in Swift you can overload functions. This means that you can declare functions with the same signature but with different parameter types or constraints. The compiler will take always the right choice. It's much more efficient than a type check at runtime.
The first method is called if the array conforms to Encodable
func encode<T: Encodable>(object: [T]) throws -> Data {
return try JSONEncoder().encode(object)
}
Otherwise the second method is called. If the array cannot even be encoded with JSONSerialization you can add custom encoding logic.
func encode<T>(object: [T]) throws -> Data {
do {
return try JSONSerialization.data(withJSONObject: object)
} catch {
// do custom encoding and return Data
return try myCustomEncoding(object)
}
}
This example
let array = [["1":1, "2":2]]
try encode(object: array)
calls the first method.
On the other hand this – even if the actual type is not heterogenous – calls the second method
let array : [[String:Any]] = [["1":1, "2":2]]
try encode(object: array)
Before Swift 5, I had this extension working:
fileprivate extension String {
func indexOf(char: Character) -> Int? {
return firstIndex(of: char)?.encodedOffset
}
}
Now, I get a deprecated message:
'encodedOffset' is deprecated: encodedOffset has been deprecated as most common usage is incorrect. Use `utf16Offset(in:)` to achieve the same behavior.
Is there a simpler solution to this instead of using utf16Offset(in:)?
I just need the index of the character position passed back as an Int.
After some time I have to admit that my original answer was incorrect.
In Swift are two methods: firstIndex(of:) and lastIndex(of:)
Both returns Int? representing index of first/last element in Array which is equal to passed element (if there is any, otherwise it returns nil).
So, you should avoid using your custom method to get index because there could be two same elements and you wouldn't know which index you need. So try to think about your usage and decide which index is more suitable for you; first or last. 🙏
Original answer:
And what is wrong with utf16Offset(in:)? This is way to go with Swift 5
fileprivate extension String {
func indexOf(char: Character) -> Int? {
return firstIndex(of: char)?.utf16Offset(in: self)
}
}
I am making a structure that acts like a String, except that it only deals with Unicode UTF-32 scalar values. Thus, it is an array of UInt32. (See this question for more background.)
What I want to do
I want to be able to use my custom ScalarString struct as a key in a dictionary. For example:
var suffixDictionary = [ScalarString: ScalarString]() // Unicode key, rendered glyph value
// populate dictionary
suffixDictionary[keyScalarString] = valueScalarString
// ...
// check if dictionary contains Unicode scalar string key
if let renderedSuffix = suffixDictionary[unicodeScalarString] {
// do something with value
}
Problem
In order to do that, ScalarString needs to implement the Hashable Protocol. I thought I would be able to do something like this:
struct ScalarString: Hashable {
private var scalarArray: [UInt32] = []
var hashValue : Int {
get {
return self.scalarArray.hashValue // error
}
}
}
func ==(left: ScalarString, right: ScalarString) -> Bool {
return left.hashValue == right.hashValue
}
but then I discovered that Swift arrays don't have a hashValue.
What I read
The article Strategies for Implementing the Hashable Protocol in Swift had a lot of great ideas, but I didn't see any that seemed like they would work well in this case. Specifically,
Object property (array is does not have hashValue)
ID property (not sure how this could be implemented well)
Formula (seems like any formula for a string of 32 bit integers would be processor heavy and have lots of integer overflow)
ObjectIdentifier (I'm using a struct, not a class)
Inheriting from NSObject (I'm using a struct, not a class)
Here are some other things I read:
Implementing Swift's Hashable Protocol
Swift Comparison Protocols
Perfect hash function
Membership of custom objects in Swift Arrays and Dictionaries
How to implement Hashable for your custom class
Writing a good Hashable implementation in Swift
Question
Swift Strings have a hashValue property, so I know it is possible to do.
How would I create a hashValue for my custom structure?
Updates
Update 1: I would like to do something that does not involve converting to String and then using String's hashValue. My whole point for making my own structure was so that I could avoid doing lots of String conversions. String gets it's hashValue from somewhere. It seems like I could get it using the same method.
Update 2: I've been looking into the implementation of string hash codes algorithms from other contexts. I'm having a little difficulty knowing which is best and expressing them in Swift, though.
Java hashCode algorithm
C algorithms
hash function for string (SO question and answers in C)
Hashing tutorial (Virginia Tech Algorithm Visualization Research Group)
General Purpose Hash Function Algorithms
Update 3
I would prefer not to import any external frameworks unless that is the recommended way to go for these things.
I submitted a possible solution using the DJB Hash Function.
Update
Martin R writes:
As of Swift 4.1, the compiler can synthesize Equatable and Hashable
for types conformance automatically, if all members conform to
Equatable/Hashable (SE0185). And as of Swift 4.2, a high-quality hash
combiner is built-in into the Swift standard library (SE-0206).
Therefore there is no need anymore to define your own hashing
function, it suffices to declare the conformance:
struct ScalarString: Hashable, ... {
private var scalarArray: [UInt32] = []
// ... }
Thus, the answer below needs to be rewritten (yet again). Until that happens refer to Martin R's answer from the link above.
Old Answer:
This answer has been completely rewritten after submitting my original answer to code review.
How to implement to Hashable protocol
The Hashable protocol allows you to use your custom class or struct as a dictionary key. In order to implement this protocol you need to
Implement the Equatable protocol (Hashable inherits from Equatable)
Return a computed hashValue
These points follow from the axiom given in the documentation:
x == y implies x.hashValue == y.hashValue
where x and y are values of some Type.
Implement the Equatable protocol
In order to implement the Equatable protocol, you define how your type uses the == (equivalence) operator. In your example, equivalence can be determined like this:
func ==(left: ScalarString, right: ScalarString) -> Bool {
return left.scalarArray == right.scalarArray
}
The == function is global so it goes outside of your class or struct.
Return a computed hashValue
Your custom class or struct must also have a computed hashValue variable. A good hash algorithm will provide a wide range of hash values. However, it should be noted that you do not need to guarantee that the hash values are all unique. When two different values have identical hash values, this is called a hash collision. It requires some extra work when there is a collision (which is why a good distribution is desirable), but some collisions are to be expected. As I understand it, the == function does that extra work. (Update: It looks like == may do all the work.)
There are a number of ways to calculate the hash value. For example, you could do something as simple as returning the number of elements in the array.
var hashValue: Int {
return self.scalarArray.count
}
This would give a hash collision every time two arrays had the same number of elements but different values. NSArray apparently uses this approach.
DJB Hash Function
A common hash function that works with strings is the DJB hash function. This is the one I will be using, but check out some others here.
A Swift implementation provided by #MartinR follows:
var hashValue: Int {
return self.scalarArray.reduce(5381) {
($0 << 5) &+ $0 &+ Int($1)
}
}
This is an improved version of my original implementation, but let me also include the older expanded form, which may be more readable for people not familiar with reduce. This is equivalent, I believe:
var hashValue: Int {
// DJB Hash Function
var hash = 5381
for(var i = 0; i < self.scalarArray.count; i++)
{
hash = ((hash << 5) &+ hash) &+ Int(self.scalarArray[i])
}
return hash
}
The &+ operator allows Int to overflow and start over again for long strings.
Big Picture
We have looked at the pieces, but let me now show the whole example code as it relates to the Hashable protocol. ScalarString is the custom type from the question. This will be different for different people, of course.
// Include the Hashable keyword after the class/struct name
struct ScalarString: Hashable {
private var scalarArray: [UInt32] = []
// required var for the Hashable protocol
var hashValue: Int {
// DJB hash function
return self.scalarArray.reduce(5381) {
($0 << 5) &+ $0 &+ Int($1)
}
}
}
// required function for the Equatable protocol, which Hashable inheirits from
func ==(left: ScalarString, right: ScalarString) -> Bool {
return left.scalarArray == right.scalarArray
}
Other helpful reading
Which hashing algorithm is best for uniqueness and speed?
Overflow Operators
Why are 5381 and 33 so important in the djb2 algorithm?
How are hash collisions handled?
Credits
A big thanks to Martin R over in Code Review. My rewrite is largely based on his answer. If you found this helpful, then please give him an upvote.
Update
Swift is open source now so it is possible to see how hashValue is implemented for String from the source code. It appears to be more complex than the answer I have given here, and I have not taken the time to analyze it fully. Feel free to do so yourself.
Edit (31 May '17): Please refer to the accepted answer. This answer is pretty much just a demonstration on how to use the CommonCrypto Framework
Okay, I got ahead and extended all arrays with the Hashable protocol by using the SHA-256 hashing algorithm from the CommonCrypto framework. You have to put
#import <CommonCrypto/CommonDigest.h>
into your bridging header for this to work. It's a shame that pointers have to be used though:
extension Array : Hashable, Equatable {
public var hashValue : Int {
var hash = [Int](count: Int(CC_SHA256_DIGEST_LENGTH) / sizeof(Int), repeatedValue: 0)
withUnsafeBufferPointer { ptr in
hash.withUnsafeMutableBufferPointer { (inout hPtr: UnsafeMutableBufferPointer<Int>) -> Void in
CC_SHA256(UnsafePointer<Void>(ptr.baseAddress), CC_LONG(count * sizeof(Element)), UnsafeMutablePointer<UInt8>(hPtr.baseAddress))
}
}
return hash[0]
}
}
Edit (31 May '17): Don't do this, even though SHA256 has pretty much no hash collisions, it's the wrong idea to define equality by hash equality
public func ==<T>(lhs: [T], rhs: [T]) -> Bool {
return lhs.hashValue == rhs.hashValue
}
This is as good as it gets with CommonCrypto. It's ugly, but fast and not manypretty much no hash collisions for sure
Edit (15 July '15): I just made some speed tests:
Randomly filled Int arrays of size n took on average over 1000 runs
n -> time
1000 -> 0.000037 s
10000 -> 0.000379 s
100000 -> 0.003402 s
Whereas with the string hashing method:
n -> time
1000 -> 0.001359 s
10000 -> 0.011036 s
100000 -> 0.122177 s
So the SHA-256 way is about 33 times faster than the string way. I'm not saying that using a string is a very good solution, but it's the only one we can compare it to right now
It is not a very elegant solution but it works nicely:
"\(scalarArray)".hashValue
or
scalarArray.description.hashValue
Which just uses the textual representation as a hash source
One suggestion - since you are modeling a String, would it work to convert your [UInt32] array to a String and use the String's hashValue? Like this:
var hashValue : Int {
get {
return String(self.scalarArray.map { UnicodeScalar($0) }).hashValue
}
}
That could conveniently allow you to compare your custom struct against Strings as well, though whether or not that is a good idea depends on what you are trying to do...
Note also that, using this approach, instances of ScalarString would have the same hashValue if their String representations were canonically equivalent, which may or may not be what you desire.
So I suppose that if you want the hashValue to represent a unique String, my approach would be good. If you want the hashValue to represent a unique sequence of UInt32 values, #Kametrixom's answer is the way to go...
TLDR
My custom structure implements the Hashable Protocol. However, when hash collisions occur while inserting keys in a Dictionary, they are not automatically handled. How do I overcome this problem?
Background
I had previously asked this question How to implement the Hashable Protocol in Swift for an Int array (a custom string struct). Later I added my own answer, which seemed to be working.
However, recently I have detected a subtle problem with hashValue collisions when using a Dictionary.
Most basic example
I have simplified the code down as far as I can to the following example.
Custom structure
struct MyStructure: Hashable {
var id: Int
init(id: Int) {
self.id = id
}
var hashValue: Int {
get {
// contrived to produce a hashValue collision for id=1 and id=2
if id == 1 {
return 2
}
return id
}
}
}
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.hashValue == rhs.hashValue
}
Note the global function to overload the equality operator (==) in order to conform to the Equatable Protocol, which is required by the Hashable Protocol.
Subtle Dictionary key problem
If I create a Dictionary with MyStructure as the key
var dictionary = [MyStructure : String]()
let ok = MyStructure(id: 0) // hashValue = 0
let collision1 = MyStructure(id: 1) // hashValue = 2
let collision2 = MyStructure(id: 2) // hashValue = 2
dictionary[ok] = "some text"
dictionary[collision1] = "other text"
dictionary[collision2] = "more text"
print(dictionary) // [MyStructure(id: 2): more text, MyStructure(id: 0): some text]
print(dictionary.count) // 2
the equal hash values cause the collision1 key to be overwritten by the collision2 key. There is no warning. If such a collision only happened once in a dictionary with 100 keys, then it could easily be missed. (It took me quite a while to notice this problem.)
Obvious problem with Dictionary literal
If I repeat this with a dictionary literal, though, the problem becomes much more obvious because a fatal error is thrown.
let ok = MyStructure(id: 0) // hashValue = 0
let collision1 = MyStructure(id: 1) // hashValue = 2
let collision2 = MyStructure(id: 2) // hashValue = 2
let dictionaryLiteral = [
ok : "some text",
collision1 : "other text",
collision2 : "more text"
]
// fatal error: Dictionary literal contains duplicate keys
Question
I was under the impression that it was not necessary for hashValue to always return a unique value. For example, Mattt Thompson says,
One of the most common misconceptions about implementing a custom hash
function comes from ... thinking that hash values must be distinct.
And the respected SO user #Gaffa says that one way to handle hash collisions is to
Consider hash codes to be non-unique, and use an equality comparer for
the actual data to determine uniqueness.
In my opinion, the question Do swift hashable protocol hash functions need to return unique values? has not been adequately answered at the time of this writing.
After reading the Swift Dictionary question How are hash collisions handled?, I assumed that Swift automatically handled hash collisions with Dictionary. But apparently that is not true if I am using a custom class or structure.
This comment makes me think the answer is in how the Equatable protocol is implemented, but I am not sure how I should change it.
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.hashValue == rhs.hashValue
}
Is this function called for every dictionary key lookup or only when there is a hash collision? (Update: see this question)
What should I do to determine uniqueness when (and only when) a hash collision occurs?
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.hashValue == rhs.hashValue
}
Note the global function to overload the equality operator (==) in order to conform to the Equatable Protocol, which is required by the Hashable Protocol.
Your problem is an incorrect equality implementation.
A hash table (such as a Swift Dictionary or Set) requires separate equality and hash implementations.
hash gets you close to the object you're looking for; equality gets you the exact object you're looking for.
Your code uses the same implementation for hash and equality, and this will guarantee a collision.
To fix the problem, implement equality to match exact object values (however your model defines equality). E.g.:
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.id == rhs.id
}
I think you have all the pieces of the puzzle you need -- you just need to put them together. You have a bunch of great sources.
Hash collisions are okay. If a hash collision occurs, objects will be checked for equality instead (only against the objects with matching hashes). For this reason, objects' Equatable conformance needs to be based on something other than hashValue, unless you are certain that hashes cannot collide.
This is the exact reason that objects that conform to Hashable must also conform to Equatable. Swift needs a more domain-specific comparison method for when hashing doesn't cut it.
In that same NSHipster article, you can see how Mattt implements isEqual: versus hash in his example Person class. Specifically, he has an isEqualToPerson: method that checks against other properties of a person (birthdate, full name) to determine equality.
- (BOOL)isEqualToPerson:(Person *)person {
if (!person) {
return NO;
}
BOOL haveEqualNames = (!self.name && !person.name) || [self.name isEqualToString:person.name];
BOOL haveEqualBirthdays = (!self.birthday && !person.birthday) || [self.birthday isEqualToDate:person.birthday];
return haveEqualNames && haveEqualBirthdays;
}
He does not use a hash value when checking for equality - he uses properties specific to his person class.
Likewise, Swift does not let you simply use a Hashable object as a dictionary key -- implicitly, by protocol inheritance -- keys must conform to Equatable as well. For standard library Swift types this has already been taken care of, but for your custom types and class, you must create your own == implementation. This is why Swift does not automatically handle dictionary collisions with custom types - you must implement Equatable yourself!
As a parting thought, Mattt also states that you can often just do an identity check to make sure your two objects are at different memory address, and thus different objects. In Swift, that would like like this:
if person1 === person2 {
// ...
}
There is no guarantee here that person1 and person2 have different properties, just that they occupy separate space in memory. Conversely, in the earlier isEqualToPerson: method, there is no guarantee that two people with the same names and birthdates are actually same people. Thus, you have to consider what makes sense for you specific object type. Again, another reason that Swift does not implement Equatable for you on custom types.
the equal hash values cause the collision1 key to be overwritten by
the collision2 key. There is no warning. If such a collision only
happened once in a dictionary with 100 keys, then it could easily be
missed.
Hash collision has nothing to do with it. (Hash collisions never affect the result, only the performance.) It is working exactly as documented.
Dictionary operations work on equality (==) of keys. Dictionaries do not contain duplicate keys (meaning keys that are equal). When you set a value with a key, it overwrites any entry containing an equal key. When you get an entry with a subscript, it finds a value with a key that is equal to, not necessarily the same as, the thing you gave. And so on.
collision1 and collision2 are equal (==), based on the way you defined the == operator. Therefore, setting an entry with key collision2 must overwrite any entry with key collision1.
P.S. The same exact thing applies with dictionaries in other languages. For example, in Cocoa, NSDictionary does not allow duplicate keys, i.e. keys that are isEqual:. In Java, Maps do not allow duplicate keys, i.e. keys that are .equals().
You can see my comments on this page's answers and this answer. I think all answers are still written in a VERY confusing way.
tl;dr 0) you don't need to write the implementation isEqual ie == between the hashValues. 1) Only provide/return hashValue. 2) just implement Equatable as you normally would
0) To conform to hashable you must have a computed value named hashValue and give it an appropriate value. Unlike equatable protocol, the comparison of hashValues is already there. You DON'T need to write:
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.hashValue == rhs.hashValue
// Snippet A
}
1) Then it uses the hashValue to check to see if for that hashValue's index (calculated by its modulo against the array's count) the key being looked exists or not. It looks within the array of key/value pairs of that index.
2) However as a fail safe ie in case there are matching hashes you fall back to the regular == func. (Logically you need it because of a fail safe. But you also you need it because Hashable protocol conforms to Equatable and therefore you must write an implementation for ==. Otherwise you would get a compiler error)
func == (lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.id == rhs.id
//Snippet B
}
Conclusion:
You must include Snippet B, exclude Snippet A, while also having a hashValue property
I am writing a mutaitin function for Array. I can't compare the array components like below:
extension Array {
mutating func mutFunc() {
while self[1]>self[2]{
}
}
}
The Array must is Int type array. I can't even use this way to compare.
while Int(self[1])>Int(self[2]){
}
What is wrong in my code?
You can't do it in Swift 1.2 or before. This is exactly the problem that extension where clauses in Swift 2.0 solves. That way, you can extend Array only and exactly insofar as its element type adopts Comparable (or even Int), thus guaranteeing that > is defined.
extension Array where Element : Comparable {
// ... your function involving > goes here
}