The documentation for Set's insert method states:
Return Value
(true, newMember) if newMember was not contained in the set. If an element equal to newMember was already contained in the set, the method returns (false, oldMember), where oldMember is the element that was equal to newMember.
In some cases, oldMember may be distinguishable from newMember by identity comparison or some other means.
What are the cases when oldMember might be distinguishable from newMember they are talking about? Any example?
Reason for the question: I have a Set that contains objects that are not necessarily Hashable. I had the idea to use Unmanaged.passUnretained(object).toOpaque().hashValue to get a hash-value for just any object (basically from its address) in a wrapper struct, however the above bit in the documentation makes me wary.
If you have some way to tell two instances apart even when they compare equal by their implementation of Equatable's == operator, then you can use this to compare the element to insert with the memberAfterInsert returned by insert(_:).
A prime example of this, as the documentation says, is the identity comparison (===). Two object instances that compare equal may not necessarily be the exact same instance (i.e have the same address).
For example:
class Foo : Hashable {
static func ==(lhs: Foo, rhs: Foo) -> Bool {
return lhs.foo == rhs.foo
}
var hashValue: Int {
return foo
}
let foo: Int
init(_ foo: Int) {
self.foo = foo
}
}
var set : Set = [Foo(10)]
let toInsert = Foo(10)
let result = set.insert(toInsert) // the element toInsert will compare equal to the element
// already in the set, thus the element already in the
// already in the set will be returned.
print(result) // (false, Foo) – false, as element was already in set
print(result.memberAfterInsert == toInsert) // true – obviously they were equal.
print(result.memberAfterInsert === toInsert) // false – but they weren't the same instance.
In your case of using a hash value based on the identity, as long as your Equatable implementation also relies on the identity, you won't break the contract of Hashable (instances that compare equal with == must have the same hashValue) – although obviously you won't be able to use the identity to differentiate the memberAfterInsert with the element you are trying to insert.
However it's worth noting that having a Set where the Element's equality implementation is solely based on identity may cause surprising results for objects that look like they should be equivalent. For example, it would be perfectly fine to have a set of [Foo(5), Foo(5), Foo(5), Foo(5)], given that the objects are different instances.
Related
I have a global var notes: Set<Note> that contains notes initialized with downloaded data.
In the code below, does Swift know to skip the initialization of my Note object if notes already contains it?
for dictionary in downloadedNoteDictionaries {
let note = Note(dictionary: dictionary)
notes.insert(note)
}
I'm wondering because my app downloads dozens of notes per request and initializing a Note object seems rather computationally expensive.
If the answer to my question is no, then how could I improve my code's performance?
My Note class—which I just realized should probably be a struct instead—has the property let id: Int64 as its sole essential component, but apparently, you can't access an element of a set by its hash value? I don't want to use Set's instance method first(where:) because it has a complexity of O(n), and notes could contain millions of Note objects.
You cannot rely on Swift to eliminate the construction of a new Note in your code. Your Set needs to ask the Note for its hashValue, and may need to call == with your Note as an argument. Those computations require the Note object. Possibly if Swift can inline everything, it can notice that your hashValue and == depend only on the id property, but it is certainly not guaranteed to notice or to act on that information.
It sounds like you should be using an [Int64: Note] instead of a Set<Note>.
No, Swift will not avoid creating the new Note object. The problem here is trying to determine if an object already exists in a set. In order to check if an object already exists in the set, you must have some way to identify this object consistently and have that identification persist across future reads and writes to that set. Assuming we want to adopt Swift's hashing enhancements which deprecates the old methods for having to manually provide a hashValue for any object conforming to the Hashable, we should not use a Set as a solution to this problem. The reason is because Swift's new recommended hashing methods use a random seed to generate hashes for added security. Depending on hash values to identify an element in a set alone would therefore not be possible.
It seems that your method of identifying these objects are by an id. I would do as Rob suggests and use a dictionary where the keys are the id. This would help you answer existential questions in order avoid instantiating a Note object.
If you need the resulting dictionary as a Set, you can still create a new Set from this resulting dictionary sequence.
It is possible to pull out a particular element from a set as long as you know how to identify it, and the operations would be O(1). You would use these two methods:
https://developer.apple.com/documentation/swift/set/2996833-firstindex
https://developer.apple.com/documentation/swift/set/2830801-subscript
Here is an example that I was able to run in a playground. However, this assumes that you have a way to identify in your data / model the id of the Note object beforehand in order to avoid creating the Note object. In this case, I am assuming that a Note shares an id with a Dictionary it holds:
import UIKit
struct DictionaryThatNoteUses {
var id: String
init(id: String = UUID().uuidString) {
self.id = id
}
}
struct Note: Hashable {
let dictionary: DictionaryThatNoteUses
var id: String {
return dictionary.id
}
func hash(into hasher: inout Hasher) {
hasher.combine(id)
}
static func == (lhs: Note, rhs: Note) -> Bool {
return lhs.id == rhs.id
}
}
var downloadedNoteDictionaries: [DictionaryThatNoteUses] = [
DictionaryThatNoteUses(),
DictionaryThatNoteUses(),
DictionaryThatNoteUses()
]
var notesDictionary: [String: Note] = [:]
var notesSet: Set<Note> = []
// Add a dictionary with the same id as the first dictionary
downloadedNoteDictionaries.append(downloadedNoteDictionaries.first!) // so now there are four Dictionary objects in this array
func createDictionaryOfNotes() {
func avoidCreatingDuplicateNotesObject(with id: String) {
print("avoided creating duplicate notes object with id \(id)") // prints once because of the duplicated note at the end matching the first, so a new Note object is not instantiated
}
downloadedNoteDictionaries.forEach {
guard notesDictionary[$0.id] == nil else { return avoidCreatingDuplicateNotesObject(with: $0.id) }
let note = Note(dictionary: $0)
notesDictionary[note.id] = note
}
}
createDictionaryOfNotes()
// Obtain a set for set operations on those unique Note objects
notesSet = Set<Note>(notesDictionary.values)
print("number of items in dictionary = \(notesDictionary.count), number of items in set = \(notesSet.count)") // prints 3 and 3 because the 4th object was a duplicate
// Grabbing a specific element from the set
// Let's test with the second Note object from the notesDictionary
let secondNotesObjectFromDictionary = notesDictionary.values[notesDictionary.values.index(notesDictionary.values.startIndex, offsetBy: 1)]
let thirdNotesObjectFromDictionary = notesDictionary.values[notesDictionary.values.index(notesDictionary.values.startIndex, offsetBy: 2)]
if let secondNotesObjectIndexInSet = notesSet.firstIndex(of: secondNotesObjectFromDictionary) {
print("do the two objects match: \(notesSet[secondNotesObjectIndexInSet] == secondNotesObjectFromDictionary)") // prints true
print("does the third object from dictionary match the second object from the set: \(thirdNotesObjectFromDictionary == notesSet[secondNotesObjectIndexInSet])") // prints false
}
when i read swift documentation i can not understand what that mean?.
A type must be hashable in order to be stored in a set—that is, the type must provide a way to compute a hash value for itself. A hash value is an Int value that is the same for all objects that compare equally, such that if a == b, it follows that a.hashValue == b.hashValue.
Sets are designed for performance. The contains(_:) method on Set is O(1) complexity meaning it takes a constant amount of time to perform no matter the size of the set. contains(_:) on an Array is O(n) meaning the time to determine if the array contains an element increases as the size of the array increases.
How does a Set do that? It uses a hashValue for the items of the Set and contains an internal dictionary like structure that maps the hashValue to the list of items with that hashValue. This makes testing equality of complex structures (like String) very quick because Set first tests if two values have the same hashValue. If the hashValues are different, it doesn't need to check the contents of the structs themselves.
To test if the Set contains a value, it is necessary to only look up the hashValue in the dictionary and then compare the item to the values that match.
So, for items to be contained in a Set, it is important to provide a hashing function that:
Is the same if two structs/objects are equal. (absolute requirement)
Computes a wide range of hashValues so that the items are widely distributed and don't require falling back to the slower check for equality. (for good performance)
Here is an example of a Hashable struct that is appropriate for storage in a Set:
struct Option: Hashable, CustomStringConvertible {
var title: String
var key: String
var value: Int
var description: String { return "{title: \"\(title)\", key: \"\(key)\", value: \(value)}" }
func hash(into hasher: inout Hasher) {
hasher.combine(title)
hasher.combine(key)
}
static func ==(lhs: Option, rhs: Option) -> Bool {
return lhs.title == rhs.title && lhs.key == rhs.key
}
}
Note: In this example, only the title and key properties are considered for equality. Two structs can be equal even if they have different value properties. The hash(into:) function likewise only considers the title and key properties.
I was thinking about how Swift ensures uniqueness for Set because I have turned one of my obj from Equatable to Hashable for free and so I came up with this simple Playground
struct SimpleStruct: Hashable {
let string: String
let number: Int
static func == (lhs: SimpleStruct, rhs: SimpleStruct) -> Bool {
let areEqual = lhs.string == rhs.string
print(lhs, rhs, areEqual)
return areEqual
}
}
var set = Set<SimpleStruct>()
let first = SimpleStruct(string: "a", number: 2)
set.insert(first)
So my first question was:
Will the static func == method be called anytime I insert a new obj inside the set?
My question comes from this thought:
For Equatable obj, in order to make this decision, the only way to ensure two obj are the same is to ask the result of static func ==.
For Hashable obj, a faster way is to compare hashValues... but, like in my case, the default implementation will use both string and number, in contrast with == logic.
So, in order to test how Set behaves, I have just added a print statement.
I have figured out that sometimes I got the print statement, sometimes no. Like sometimes hashValue isn't enough in order to make this decision ... So the method hasn't been called every time.
Weird...
So I've tried to add two objects that are equal and wondering what will be the result of set.contains
let second = SimpleStruct(string: "a", number: 3)
print(first == second) // returns true
set.contains(second)
And wonders of wonders, launching a couple of times the playground, I got different results and this might cause unpredictable results ...
Adding
var hashValue: Int {
return string.hashValue
}
it gets rid of any unexpected results but my doubt is:
Why, without the custom hashValue implementation, == sometimes gets called and sometimes it doesn't?
Should Apple avoid this kind of unexpected behaviours?
The synthesized implementation of the Hashable requirement uses all stored
properties of a struct, in your case string and number. Your implementation
of == is only based on the string:
let first = SimpleStruct(string: "a", number: 2)
let second = SimpleStruct(string: "a", number: 3)
print(first == second) // true
print(first.hashValue == second.hashValue) // false
This is a violation of a requirement of the Hashable protocol:
Two instances that are equal must feed the same values to Hasher in hash(into:), in the same order.
and causes the undefined behavior. (And since hash values are randomized
since Swift 4.2, the behavior can be different in each program run.)
What probably happens in your test is that the hash value of second is used to determine the “bucket” of the set in which the value
would be stored. That may or may not be the same bucket in which first is stored. – But that is an implementation detail: Undefined behavior is undefined behavior, it can cause unexpected results or even
runtime errors.
Implementing
var hashValue: Int {
return string.hashValue
}
or alternatively (starting with Swift 4.2)
func hash(into hasher: inout Hasher) {
hasher.combine(string)
}
fixes the rule violation, and therefore makes your code behave as expected.
I'd like to create a generic find() typically used in functional programming. In functional programming you don't work with array indices and for loops. You filter. The way it works is that if you have a list of say
["apple", "banana", "cherry"]
and you want to find banana then you assign the array indices to the list elements by creating tuples
[(1, "apple"), (2, "banana"), (3, "cherry")]
Now you can filter down to "banana" and return the index value.
I was trying to create a generic function for this but I get an error. What's wrong with this syntax?
func findInGenericIndexedList<T>(indexedList: [(index: Int, value: T)], element: (index: Int, value: T)) -> Int? {
let found = indexedList.filter { // ERROR: Cannot invoke 'filter' with an argument list of type '((_) -> _)'
element.value === $0.value
}
if let definiteFound = found.first {
return definiteFound.index
}
return nil
}
UPDATE 1: I would like to use the above solution as opposed to using find() (will be deprecated) or in Swift 2.0 indexOf() because I'm trying to follow the Functional Programming paradigm, relying on general functions and not class methods.
The minimum change required to make this work would be to make T conform to Equatable and use the == operator.
func findInGenericIndexedList<T:Equatable>(indexedList: [(index: Int, value: T)], element: (index: Int, value: T)) -> Int? {
let found = indexedList.filter {
element.value == $0.value
}
if let definiteFound = found.first {
return definiteFound.index
}
return nil
}
It doesn't really make sense to use === here because are usually going to be applying this to value types (especially if you are following functional paradigms) for which this is never true.
Beyond this I spent some time thinking about the problem and here is what I would do:
extension Array where Element : Equatable {
func find(element:Array.Generator.Element) -> Int? {
let indexedList = lazy(self.enumerate())
let found = indexedList.filter {
element == $1
}
let definiteFound = found.prefix(1)
return definiteFound.generate().next()?.index
}
}
Protocol extension on Array because it makes the syntax neater, lazy sequence to avoid checking every element, 0 indexed.
Here are a couple of thoughts.
I would still prefer to use the identity operator '===', because in my array I may have multiple items of the same value.
The identity operator === only works for reference types, like classes. It will never work for value types, like Strings or Ints, or structs, etc. You might take a look at the difference between value types and reference types, especially if you are interested in functional programming, which eschews reference types almost completely. When you are working with value types, there is only equality (==) - there is no identity. Two instances of the String "bananas" will never refer to the same identical object. They will always refer to two different Strings, though their values might be equal.
I want to delete the exact item that I passed to the function. Is it impossible to do this?
If you are working with value types, like Strings, then yes, it is impossible. There is no such thing as two different Strings that are the exact same item. Two Strings are always are always different objects, for the reasons stated above.
Note that if you work only with classes, and not value types, then you could use the === operator, but this would defeat much of what you are trying to do.
What this boils down to is that if you have an array of (index, value) tuples that looks like this:
[(0, "bananas"), (1, "apples"), (2, "oranges"), (3, "bananas")]
And you write a function that looks for tuples where the value is "bananas", you have a couple of choices. You can filter it and look for the first tuple in the array that has the value "bananas" and return the index of that tuple. In the above case it would return 0. Or, you could return all of the indexes in the form of an array, like this: [0, 3]. Or I suppose you could return some other arbitrary subset of the results, like the last index, or the first-and-last indexes, etc., but those all seem a little silly. The Swift standard library opts for returning the index of the first item that matches the search criteria for precisely this reason. None of the other options make a whole lot of sense.
But putting it back into the context of your question, none of the tuples that you find with the value of "bananas" are going to be the exact (identical) instance of "bananas" that you passed in to your search function. No two value types are ever identical. They may be equal, but they are never identical.
One more note - mainly for clarification of what you are even trying to do. In your first attempt at writing this function, you appear to already know the index of the item you are searching for. You pass it in to the function as a parameter, right here:
// ----------vvvvv
func findInGenericIndexedList<T>(indexedList: [(index: Int, value: T)], element: (index: Int, value: T)) -> Int?
Just out of curiosity, is this a typo? Or do you actually know the index of the tuple that you are searching for? Because if you already know what it is, well... you don't need to search for it :)
TLDR
My custom structure implements the Hashable Protocol. However, when hash collisions occur while inserting keys in a Dictionary, they are not automatically handled. How do I overcome this problem?
Background
I had previously asked this question How to implement the Hashable Protocol in Swift for an Int array (a custom string struct). Later I added my own answer, which seemed to be working.
However, recently I have detected a subtle problem with hashValue collisions when using a Dictionary.
Most basic example
I have simplified the code down as far as I can to the following example.
Custom structure
struct MyStructure: Hashable {
var id: Int
init(id: Int) {
self.id = id
}
var hashValue: Int {
get {
// contrived to produce a hashValue collision for id=1 and id=2
if id == 1 {
return 2
}
return id
}
}
}
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.hashValue == rhs.hashValue
}
Note the global function to overload the equality operator (==) in order to conform to the Equatable Protocol, which is required by the Hashable Protocol.
Subtle Dictionary key problem
If I create a Dictionary with MyStructure as the key
var dictionary = [MyStructure : String]()
let ok = MyStructure(id: 0) // hashValue = 0
let collision1 = MyStructure(id: 1) // hashValue = 2
let collision2 = MyStructure(id: 2) // hashValue = 2
dictionary[ok] = "some text"
dictionary[collision1] = "other text"
dictionary[collision2] = "more text"
print(dictionary) // [MyStructure(id: 2): more text, MyStructure(id: 0): some text]
print(dictionary.count) // 2
the equal hash values cause the collision1 key to be overwritten by the collision2 key. There is no warning. If such a collision only happened once in a dictionary with 100 keys, then it could easily be missed. (It took me quite a while to notice this problem.)
Obvious problem with Dictionary literal
If I repeat this with a dictionary literal, though, the problem becomes much more obvious because a fatal error is thrown.
let ok = MyStructure(id: 0) // hashValue = 0
let collision1 = MyStructure(id: 1) // hashValue = 2
let collision2 = MyStructure(id: 2) // hashValue = 2
let dictionaryLiteral = [
ok : "some text",
collision1 : "other text",
collision2 : "more text"
]
// fatal error: Dictionary literal contains duplicate keys
Question
I was under the impression that it was not necessary for hashValue to always return a unique value. For example, Mattt Thompson says,
One of the most common misconceptions about implementing a custom hash
function comes from ... thinking that hash values must be distinct.
And the respected SO user #Gaffa says that one way to handle hash collisions is to
Consider hash codes to be non-unique, and use an equality comparer for
the actual data to determine uniqueness.
In my opinion, the question Do swift hashable protocol hash functions need to return unique values? has not been adequately answered at the time of this writing.
After reading the Swift Dictionary question How are hash collisions handled?, I assumed that Swift automatically handled hash collisions with Dictionary. But apparently that is not true if I am using a custom class or structure.
This comment makes me think the answer is in how the Equatable protocol is implemented, but I am not sure how I should change it.
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.hashValue == rhs.hashValue
}
Is this function called for every dictionary key lookup or only when there is a hash collision? (Update: see this question)
What should I do to determine uniqueness when (and only when) a hash collision occurs?
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.hashValue == rhs.hashValue
}
Note the global function to overload the equality operator (==) in order to conform to the Equatable Protocol, which is required by the Hashable Protocol.
Your problem is an incorrect equality implementation.
A hash table (such as a Swift Dictionary or Set) requires separate equality and hash implementations.
hash gets you close to the object you're looking for; equality gets you the exact object you're looking for.
Your code uses the same implementation for hash and equality, and this will guarantee a collision.
To fix the problem, implement equality to match exact object values (however your model defines equality). E.g.:
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.id == rhs.id
}
I think you have all the pieces of the puzzle you need -- you just need to put them together. You have a bunch of great sources.
Hash collisions are okay. If a hash collision occurs, objects will be checked for equality instead (only against the objects with matching hashes). For this reason, objects' Equatable conformance needs to be based on something other than hashValue, unless you are certain that hashes cannot collide.
This is the exact reason that objects that conform to Hashable must also conform to Equatable. Swift needs a more domain-specific comparison method for when hashing doesn't cut it.
In that same NSHipster article, you can see how Mattt implements isEqual: versus hash in his example Person class. Specifically, he has an isEqualToPerson: method that checks against other properties of a person (birthdate, full name) to determine equality.
- (BOOL)isEqualToPerson:(Person *)person {
if (!person) {
return NO;
}
BOOL haveEqualNames = (!self.name && !person.name) || [self.name isEqualToString:person.name];
BOOL haveEqualBirthdays = (!self.birthday && !person.birthday) || [self.birthday isEqualToDate:person.birthday];
return haveEqualNames && haveEqualBirthdays;
}
He does not use a hash value when checking for equality - he uses properties specific to his person class.
Likewise, Swift does not let you simply use a Hashable object as a dictionary key -- implicitly, by protocol inheritance -- keys must conform to Equatable as well. For standard library Swift types this has already been taken care of, but for your custom types and class, you must create your own == implementation. This is why Swift does not automatically handle dictionary collisions with custom types - you must implement Equatable yourself!
As a parting thought, Mattt also states that you can often just do an identity check to make sure your two objects are at different memory address, and thus different objects. In Swift, that would like like this:
if person1 === person2 {
// ...
}
There is no guarantee here that person1 and person2 have different properties, just that they occupy separate space in memory. Conversely, in the earlier isEqualToPerson: method, there is no guarantee that two people with the same names and birthdates are actually same people. Thus, you have to consider what makes sense for you specific object type. Again, another reason that Swift does not implement Equatable for you on custom types.
the equal hash values cause the collision1 key to be overwritten by
the collision2 key. There is no warning. If such a collision only
happened once in a dictionary with 100 keys, then it could easily be
missed.
Hash collision has nothing to do with it. (Hash collisions never affect the result, only the performance.) It is working exactly as documented.
Dictionary operations work on equality (==) of keys. Dictionaries do not contain duplicate keys (meaning keys that are equal). When you set a value with a key, it overwrites any entry containing an equal key. When you get an entry with a subscript, it finds a value with a key that is equal to, not necessarily the same as, the thing you gave. And so on.
collision1 and collision2 are equal (==), based on the way you defined the == operator. Therefore, setting an entry with key collision2 must overwrite any entry with key collision1.
P.S. The same exact thing applies with dictionaries in other languages. For example, in Cocoa, NSDictionary does not allow duplicate keys, i.e. keys that are isEqual:. In Java, Maps do not allow duplicate keys, i.e. keys that are .equals().
You can see my comments on this page's answers and this answer. I think all answers are still written in a VERY confusing way.
tl;dr 0) you don't need to write the implementation isEqual ie == between the hashValues. 1) Only provide/return hashValue. 2) just implement Equatable as you normally would
0) To conform to hashable you must have a computed value named hashValue and give it an appropriate value. Unlike equatable protocol, the comparison of hashValues is already there. You DON'T need to write:
func ==(lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.hashValue == rhs.hashValue
// Snippet A
}
1) Then it uses the hashValue to check to see if for that hashValue's index (calculated by its modulo against the array's count) the key being looked exists or not. It looks within the array of key/value pairs of that index.
2) However as a fail safe ie in case there are matching hashes you fall back to the regular == func. (Logically you need it because of a fail safe. But you also you need it because Hashable protocol conforms to Equatable and therefore you must write an implementation for ==. Otherwise you would get a compiler error)
func == (lhs: MyStructure, rhs: MyStructure) -> Bool {
return lhs.id == rhs.id
//Snippet B
}
Conclusion:
You must include Snippet B, exclude Snippet A, while also having a hashValue property