BigQuery: Hashing a String Doesn't Match CityHash - hash

Question

Trying to get my external CityHash to return the same value as BigQuery Hash().
Here are the values I'm trying to match:
The only hashed string that matches is a blank string.
In the BigQuery Query Reference, it mentions that it uses the CityHash libarary. I've tried using multiple external libraries for CityHash, and they're all consistent with each other, but not with BigQuery Hash()
Here is an example of CityHash in Go (Golang):
package main
import (
"fmt"
"bitbucket.org/creachadair/cityhash"
)
func main() {
var bytesToHash = []byte("mystringtohash")
myHash := int64(cityhash.Hash64(bytesToHash))
fmt.Printf("Hashed version of '%s': %d\n", bytesToHash, myHash)
bytesToHash = []byte("")
myHash = int64(cityhash.Hash64(bytesToHash))
fmt.Printf("Hashed version of '%s': %d\n", bytesToHash, myHash)
}
Here is the output from my program:
Hashed version of 'mystringtohash': -6615946700494525143
Hashed version of '1234': 882600748797058222
Hashed version of '': -7286425919675154353
Is BigQuery doing something special with the string before hashing it?

Answer 1

OK, I spent some time going through the code, and here is what I think happened.
BigQuery's implementation of CityHash is based on code in version 1.0.3 (can be still downloaded from here https://code.google.com/p/cityhash/downloads/detail?name=cityhash-1.0.3.tar.gz)
The golang implementation you used seems to be a port of version 1.1.1 (can be downloaded from here https://code.google.com/p/cityhash/downloads/detail?name=cityhash-1.1.1.tar.gz)
Unfortunately, these versions seem to be incompatible since version 1.1, as noted in README (emphasis is mine):
CityHash v1.1, October 22, 2012
Add CityHash32(), intended for 32-bit platforms.
Change existing functions to improve their hash quality and/or speed. > Most
of the changes were minor, but CityHashCrc* was substantially reworked
(and made perhaps 10% slower, unfortunately).
Improve README.
I am not sure what is the right thing to do here, maybe BigQuery should update its implementation to match version 1.1.1, or maybe it will be a breaking change to existing users who rely on it. But at least we know what is going on now.