I want to be able to switch between hash algorithms depending on caller input, for example, implement a function:
func GenericHash(dat []byte, hash unint) (string, error) { ... }
where hash is the algorithm type as specified by crypto.Hash.
I'm not sure how to write this function, in particular, where the import statements should go. If I include all the import statements for algorithms that I will use at the top, go complains that they're imported and not used. Is there anyway to import on demand?
What you need to do is import the packages for their side effects only (i.e. use the blank identifier when importing the packages). This means that the imported packages' init functions will be executed, but you will not be able to access any of their exported members directly.
Here is one way you could solve your problem:
import (
"errors"
"encoding/hex"
"crypto"
_ "crypto/md5"
_ "crypto/sha1"
// import more hash packages
)
func GenericHash(dat []byte, hash crypto.Hash) (string, error) {
if !hash.Available() {
return "", errors.New("hash unavailable")
}
h := hash.New()
return hex.EncodeToString(h.Sum(dat)), nil
}
Related
I've done some searching and can't seem to find a solution for my below issue. I'm fairly new to swift so working through some issues. I have the below function which is just a stub containing the signature of the function and a return value just to get it to compile.
The test code is what I've been given, I did not write this and unfortunately cannot alter it.
My issues is that when I run it, it says that the test that calls this function has an "Ambiguous call to member '=='". I cannot alter the test code. Whatever the issue is must be in my function signature i'm assuming but could use some help.
Function That I am writing (That I assume contains the issue):
func contains(target: StringOrInt, compare: Character ) -> Bool {
return false
} // contains
Test that calls the function (I'm not allowed to edit this and did not write it):
func test_contains_cons_2_strings_3() {
let list1 = MyList.cons("foo", MyList.cons("bar", MyList.empty))
assertEquals(testName: "test_contains_cons_2_strings_3",
expected: true,
received: list1.contains(target: "bar", compare: ==))//Error is on this line '=='
} // test_contains_cons_2_strings_3
Error:
main.swift:807:67: error: ambiguous reference to member '=='
received: list1.contains(target: "foo", compare: ==))
Also note that "StringOrInt" is a protocol that I've defined that acts as an extension on both Int and String. This is done because the test code (Which i did not write and cannot edit) passes both strings and ints to this same variable and function.
Thanks advance for any help, I really appreciate it!
I think you want to compare two strings by passing "==" operator and return a Boolean value.If so you can use the below method,
func myList(_ compare:((String, String) -> Bool)) -> Bool {
return compare("foo","bar")
}
myList(==)
I was able to figure this out using the below. I implemented an enum of generic type and the nan extension on that enum. I then used this generic type within the contains function to represent multiple types instead of StringOrInt. Thanks everyone for the comments and the help.
enum implemented:
indirect enum my_list<A> {
case cons(A, my_list<A>)
case empty
}
I then implemented in extension for this enum "extension my_list" and placed the contains function within the extension.
contains function signature/stub:
func contains(target: A, compare:((A, A) -> Bool))
-> Bool {
return true
}// contains
I'm using the GameplayKit entity component system to develop a game.
I have a number of GKComponent. Most of the instance functions require that self.entity is not nil, but it is an optional, as a component may be initialised before being attached to an entity.
If I "do things properly", I understand I should use guard (or if) to check that self.entity is not nil, for each function where I require it. This adds a chunk of extra code for each function, and in addition a function may require the entity has specific components to function.
Problem statement:
For some functions in a GKComponent, I want to specify that the function requires self.entity is not nil, and has some specific components, without needing a number of guard or if statements which bulks up my code.
Things I've tried:
I tried creating an extension to GKComponent.
The extension contains a function which throws an error if the entity is nil, returns true if it is not nil.
import Foundation
import GameplayKit
extension GKComponent {
enum ComponentExtensionError: Error {
case hasNoEntity
}
func requiresEntity() throws -> Bool {
if (self.entity != nil) {
return true
} else {
throw ComponentExtensionError.hasNoEntity
}
}
}
GKComponent function:
func setTileWalls () {
try? self.requiresEntity()
# Some stuff
}
This compiles, but doesn't solve my problem, as I still have to access the entity as self.entity?.somefunc().
I figured now I know that self.entity is not nil, I could proceed to set it as unwrapped...
func setTileWalls () {
try? self.requiresEntity()
self.entity = self.entity!
# Some stuff
}
But alas, you cannot SET self.entity.
I considered I could in stead modify the extension to return the entity, and then do something like...
func stTileWalls () {
guard let entity = self.requiresEntity() else { throw }
}
but then I'd need to modify each functions code to use entity rather than self.entity. I don't feel like this is optimal solution, but maybe an acceptable one.
My ideal solution would be to be able to call a function which checks that self has an entity, and some components, making them non optional for the scope, and throwing an error if any are nil.
Why? I'm frustrated by entity and any component always being optional (when I know it shouldn't be). If I understand correctly, "doing things properly" requires that I guard and throw (or return maybe?). Syntactically I feel it looks ugly and I feel like I'm missing something in terms of fixing this in an elegent way with Swift.
There is no easy solution to solve this issue if you want a safe solution that requires less boilerplate code than unwrapping an optional.
Throwing an error instead of returning nil is definitely a bad idea, since handling an error requires even more boilerplate code than safely unwrapping an optional.
You cannot override existing variables types, so you cannot make GKComponent.entity non-optional either.
Use guard statements for optional binding, that way you can minimise the boilerplate code for unwrapping the value.
I'm writing a RESTful API in Golang, which also has a gRPC api. The API connects to a MongoDB database, and uses structs to map out entities. I also have a .proto definition which matches like for like the struct I'm using for MongoDB.
I just wondered if there was a way to share, or re-use the .proto defined code for the MongoDB calls also. I've noticed the strucs protoc generates has json tags for each field, but obviously there aren't bson tags etc.
I have something like...
// Menu -
type Menu struct {
ID bson.ObjectId `json:"id" bson"_id"`
Name string `json:"name" bson:"name"`
Description string `json:"description" bson:"description"`
Mixers []mixers.Mixer `json:"mixers" bson:"mixers"`
Sections []sections.Section `json:"sections" bson:"sections"`
}
But then I also have protoc generated code...
type Menu struct {
Id string `protobuf:"bytes,1,opt,name=id" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name" json:"name,omitempty"`
Description string `protobuf:"bytes,3,opt,name=description" json:"description,omitempty"`
Mixers []*Mixer `protobuf:"bytes,4,rep,name=mixers" json:"mixers,omitempty"`
Sections []*Section `protobuf:"bytes,5,rep,name=sections" json:"sections,omitempty"`
}
Currently I'm having to convert between the two structs depending what I'm doing. Which is tedious and I'm probably quite a considerable performance hit. So is there a better way of converting between the two, or re-using one of them for both tasks?
Having lived with this same issue, there's a couple methods of solving it. They fall into two general methods:
Use the same data type
Use two different struct types and map between them
If you want to use the same data type, you'll have to modify the code generation
You can use something like gogoprotobuf which has an extension to add tags. This should give you bson tags in your structs.
You could also post-process your generated files, either with regular expressions or something more complicated involving the go abstract syntax tree.
If you choose to map between them:
Use reflection. You can write a package that will take two structs and try to take the values from one and apply it to another. You'll have to deal with edge cases (slight naming differences, which types are equivalent, etc), but you'll have better control over edge cases if they ever come up.
Use JSON as an intermediary. As long as the generated json tags match, this will be a quick coding exercise and the performance hit of serializing and deserializing might be acceptable if this isn't in a tight loop in your code.
Hand-write or codegen mapping functions. Depending on how many structs you have, you could write out a bunch of functions that translate between the two.
At my workplace, we ended up doing a bit of all of them: forking the protoc generator to do some custom tags, a reflection based structs overlay package for mapping between arbitrary structs, and some hand-written ones in more performance sensitive or less automatable mappings.
I have played with it and have a working example with:
github.com/gogo/protobuf v1.3.1
go.mongodb.org/mongo-driver v1.4.0
google.golang.org/grpc v1.31.0
First of all I would like to share my proto/contract/example.proto file:
syntax = "proto2";
package protobson;
import "gogoproto/gogo.proto";
option (gogoproto.sizer_all) = true;
option (gogoproto.marshaler_all) = true;
option (gogoproto.unmarshaler_all) = true;
option go_package = "gitlab.com/8bitlife/proto/go/protobson";
service Service {
rpc SayHi(Hi) returns (Hi) {}
}
message Hi {
required bytes id = 1 [(gogoproto.customtype) = "gitlab.com/8bitlife/protobson/custom.BSONObjectID", (gogoproto.nullable) = false, (gogoproto.moretags) = "bson:\"_id\""] ;
required int64 limit = 2 [(gogoproto.nullable) = false, (gogoproto.moretags) = "bson:\"limit\""] ;
}
It contains a simple gRPC service Service that has SayHi method with request type Hi. It includes a set of options: gogoproto.sizer_all, gogoproto.marshaler_all, gogoproto.unmarshaler_all. Their meaning you can find at extensions page. The Hi itself contains two fields:
id that has additional options specified: gogoproto.customtype and gogoproto.moretags
limit with only gogoproto.moretags option
BSONObjectID used in gogoproto.customtype for id field is a custom type that I defined as custom/objectid.go:
package custom
import (
"go.mongodb.org/mongo-driver/bson/bsontype"
"go.mongodb.org/mongo-driver/bson/primitive"
)
type BSONObjectID primitive.ObjectID
func (u BSONObjectID) Marshal() ([]byte, error) {
return u[:], nil
}
func (u BSONObjectID) MarshalTo(data []byte) (int, error) {
return copy(data, (u)[:]), nil
}
func (u *BSONObjectID) Unmarshal(d []byte) error {
copy((*u)[:], d)
return nil
}
func (u *BSONObjectID) Size() int {
return len(*u)
}
func (u *BSONObjectID) UnmarshalBSONValue(t bsontype.Type, d []byte) error {
copy(u[:], d)
return nil
}
func (u BSONObjectID) MarshalBSONValue() (bsontype.Type, []byte, error) {
return bsontype.ObjectID, u[:], nil
}
It is needed because we need to define a custom marshaling and un-marshaling methods for both: protocol buffers and mongodb driver. This allows us to use this type as an object identifier in mongodb. And to "explaine" it to mongodb driver I marked it with a bson tag by using (gogoproto.moretags) = "bson:\"_id\"" option in proto file.
To generate source code from the proto file I used:
protoc \
--plugin=/Users/pstrokov/go/bin/protoc-gen-gogo \
--plugin=/Users/pstrokov/go/bin/protoc-gen-go \
-I=/Users/pstrokov/Workspace/protobson/proto/contract \
-I=/Users/pstrokov/go/pkg/mod/github.com/gogo/protobuf#v1.3.1 \
--gogo_out=plugins=grpc:. \
example.proto
I have tested it on my MacOS with running MongoDB instance: docker run --name mongo -d -p 27017:27017 mongo:
package main
import (
"context"
"log"
"net"
"time"
"gitlab.com/8bitlife/protobson/gitlab.com/8bitlife/proto/go/protobson"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"google.golang.org/grpc"
)
type hiServer struct {
mgoClient *mongo.Client
}
func (s *hiServer) SayHi(ctx context.Context, hi *protobson.Hi) (*protobson.Hi, error) {
collection := s.mgoClient.Database("local").Collection("bonjourno")
res, err := collection.InsertOne(ctx, bson.M{"limit": hi.Limit})
if err != nil { panic(err) }
log.Println("generated _id", res.InsertedID)
out := &protobson.Hi{}
if err := collection.FindOne(ctx, bson.M{"_id": res.InsertedID}).Decode(out); err != nil { return nil, err }
log.Println("found", out.String())
return out, nil
}
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
lis, err := net.Listen("tcp", "localhost:0")
if err != nil { log.Fatalf("failed to listen: %v", err) }
clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
clientOptions.SetServerSelectionTimeout(time.Second)
client, err := mongo.Connect(ctx, clientOptions)
if err != nil { log.Fatal(err) }
if err := client.Ping(ctx, nil); err != nil { log.Fatal(err) }
grpcServer := grpc.NewServer()
protobson.RegisterServiceServer(grpcServer, &hiServer{mgoClient: client})
go grpcServer.Serve(lis); defer grpcServer.Stop()
conn, err := grpc.Dial(lis.Addr().String(), grpc.WithInsecure())
if err != nil { log.Fatal(err) }; defer conn.Close()
hiClient := protobson.NewServiceClient(conn)
response, err := hiClient.SayHi(ctx, &protobson.Hi{Limit: 99})
if err != nil { log.Fatal(err) }
if response.Limit != 99 { log.Fatal("unexpected limit", response.Limit) }
if response.Id.Size() == 0 { log.Fatal("expected a valid ID of the new entity") }
log.Println(response.String())
}
Sorry for the formatting of the last code snippet :)
I hope this can help.
I'm in the process of testing and may provide code shortly, (ping me if you don't see it and you want it) but https://godoc.org/go.mongodb.org/mongo-driver/bson/bsoncodec looks like the ticket. protoc will make your structs and you don't have to mess with customizing them. Then you can customize the mongo-driver to do the mapping of certain types for you and it looks like their library for this is pretty good.
This is great because if I use the protogen structs then I'd like that to my application core / domain layer. I don't want to be concerned about mongoDB compatibility over there.
So right now, it seems to me that #Liyan Chang 's answer saying
If you want to use the same data type, you'll have to modify the code generation
doesn't necessarily have to be the case. Because you can opt to use 1 datatype.
You can use one generated type and account for seemingly whatever you need to in terms of getting and setting data to the DB with this codec system.
See https://stackoverflow.com/a/59561699/8546258 - the bson struct tags are not an end all be all. looks like codec can totally help with this.
See https://stackoverflow.com/a/58985629/8546258 fo a nice write up about codecs in general.
Please keep in mind these codecs were released in 1.3 of the mongodb go driver. I found this which directed me there: https://developer.mongodb.com/community/forums/t/mgo-setbson-to-mongo-golang-driver/2340/2?u=yehuda_makarov
I am having difficulty understanding how to set an interface value that has been passed as a pointer. I am trying to accomplish something along the lines of this:
import "fmt"
var Stuff map[string]interface{}
func main() {
var num int
Stuff["key"] = 9001
get("key", &num)
fmt.Println("num:", num)
}
func get(k string, v interface{}) {
*v = Stuff[k]
}
What would I have to do to make my program output be
num: 9001
Edit: is there a possible catch-all solution using reflect?
You can emulate the AppEngine datastore interface using reflect; usually I say minimize reflection, but you (and AppEngine and other ORMs) have no other great option here to present the interface you want. For something emulating Get you:
get a reflect.Value with ValueOf()
get the type of the thing you want to create
create it with reflect.Zero
optionally fill in some data with reflect.Field(), etc.
use reflect.Indirect() and Value.Set() to set the original through the pointer.
A trivial example that just zeroes a struct through a pointer is at http://play.golang.org/p/g7dNlrG_vr and copied here:
package main
import (
"fmt"
"reflect"
)
func main() {
i := 1
clear(&i)
fmt.Println(i)
}
func clear(dst interface{}) {
// ValueOf to enter reflect-land
dstPtrValue := reflect.ValueOf(dst)
// need the type to create a value
dstPtrType := dstPtrValue.Type()
// *T -> T, crashes if not a ptr
dstType := dstPtrType.Elem()
// the *dst in *dst = zero
dstValue := reflect.Indirect(dstPtrValue)
// the zero in *dst = zero
zeroValue := reflect.Zero(dstType)
// the = in *dst = 0
dstValue.Set(zeroValue)
}
For emulating GetMulti you need more steps to work with the slice. An example is at http://play.golang.org/p/G_6jit2t-2 and below:
package main
import (
"fmt"
"reflect"
)
func main() {
s := []int{}
getMultiZeroes(&s, 10)
fmt.Println(s)
}
func getMultiZeroes(slicePtrIface interface{}, howMany int) {
// enter `reflect`-land
slicePtrValue := reflect.ValueOf(slicePtrIface)
// get the type
slicePtrType := slicePtrValue.Type()
// navigate from `*[]T` to `T`
sliceElemType := slicePtrType.Elem().Elem() // crashes if input type not `*[]T`
// we'll need this to Append() to
sliceValue := reflect.Indirect(slicePtrValue)
// and this to Append()
sliceElemValue := reflect.Zero(sliceElemType)
// append requested number of zeroes
for i := 0; i < howMany; i++ {
// s := append(s, v)
sliceValue.Set(reflect.Append(sliceValue, sliceElemValue))
}
}
In live code (as opposed to testing like you're doing), it'd be faster to use a type switch (as Martin suggested) so that specialized native code runs for each type; that might also be handy if you have different behavior by type. An example for GetMulti is at http://play.golang.org/p/q-9WyUqv6P and below:
package main
import "fmt"
func main() {
s := []int{}
getZeroes(&s)
fmt.Println(s)
fails := []float32{}
getZeroes(&fails)
}
func getZeroes(slicePtrIface interface{}) {
switch sp := slicePtrIface.(type) {
case *[]int:
(*sp) = append((*sp), 0, 0)
case *[]string:
(*sp) = append((*sp), "", "")
default:
panic(fmt.Sprintf("getZeroes: passed type %T, which is not a pointer to a slice of a supported type", slicePtrIface))
}
}
You could even trivially combine the two; write custom code for common types and call the slow reflect-based version in the default case. Demo at http://play.golang.org/p/6qw52B7eC3 (not copying because it's a such a simple stitching together of the two above).
There happened to be another recent question on how to make a value to pass to GetMulti, rather than emulating the GetMulti itself, if that comes up.
More for general reference than to answer this:
"Go lacks pass by reference" is useful to know, but also needs some elaboration. Go has pointers, and other types like slices that contain pointers to data. The sense in which there isn't "pass by reference" is just that Go will never change a value argument (int, struct) into a pointer implicitly. C++ reference arguments do exactly that: C++ void f(i int&) { i++; } changes i in the caller without the caller explicitly passing in a pointer at the callsite. func (i int) { i++ } doesn't.
In Go, you can look at the types passed to a function call and tell what data it can change. With C++ reference arguments or some languages' "pass by reference" semantics, any call might change locals; you can't tell without looking up the declarations.
For purposes of avoiding unnecessary copying of data, there are already pointers in the implementations of slice, string, map, interface, and channel values. Of those types, pointers, slices, and maps will actually let you modify data through them. Also, like in C++, Go's this-like receiver parameter can be a pointer without an explicit & in the calling code. There's more about this in Russ Cox's godata post and this summary on when you need a pointer or not.
The Go Programming Language Specification
Calls
In a function call, the function value and arguments are evaluated in
the usual order. After they are evaluated, the parameters of the call
are passed by value to the function and the called function begins
execution. The return parameters of the function are passed by value
back to the calling function when the function returns.
In Go everything is passed by value; nothing is passed by reference. Therefore, pass a pointer. For example,
package main
import "fmt"
var Stuff map[string]interface{}
func main() {
Stuff = make(map[string]interface{})
Stuff["key"] = 9001
var value interface{}
get("key", &value)
num := value.(int)
fmt.Println("num:", num)
}
func get(k string, v interface{}) {
*v.(*interface{}) = Stuff[k]
}
Output:
num: 9001
First: There is absolutely no concept of "pass by reference" in Go. There isn't. What you can do is pass around a pointer. This pointer value is passed by value as everything in Go is passed by value.
Second: Instead of passing in a pointer to an interface and modify the pointees value (doable but ugly) you could return the value (much nicer).
Third: It cannot (i.e. not without reflection or unsafe) be done without type assertions.
And you should never (in the sense of "no until you mastered Go and interfaces") use pointer to interface.
Fifth: If your solution requires interface{} you might be doing something wrong. Are you sure your entities are not describable by some (non empty) interface?
That said, something like that works.
func main() {
var num int
Stuff["key"] = 9001
num = get("key").(int)
}
func get(k string) interface{}{
return Stuff[k]
}
Martin Gallagher solution works perfectly, but as he said, you can't use generics in Golang, so code looks a bit ugly. I guess another solution is to use always interface{} as the type and then cast (or check the type) in your program. Something like this: http://play.golang.org/p/0o20jToXHV
http://play.golang.org/p/kx5HvEiOm9
Without generics you will have to implement the switch {} for each of your supported types.
For example:
hash("HelloWorld") = 1234567
Is there any built-in function could do this ?
Thanks.
The hash package is helpful for this. Note it's an abstraction over specific hash implementations. Some ready made are found in the package subdirectories.
Example:
package main
import (
"fmt"
"hash/fnv"
)
func hash(s string) uint32 {
h := fnv.New32a()
h.Write([]byte(s))
return h.Sum32()
}
func main() {
fmt.Println(hash("HelloWorld"))
fmt.Println(hash("HelloWorld."))
}
(Also here)
Output:
926844193
107706013
Here is a function you could use to generate a hash number:
// FNV32a hashes using fnv32a algorithm
func FNV32a(text string) uint32 {
algorithm := fnv.New32a()
algorithm.Write([]byte(text))
return algorithm.Sum32()
}
I put together a group of those utility hash functions here: https://github.com/shomali11/util
You will find FNV32, FNV32a, FNV64, FNV64a, MD5, SHA1, SHA256 and SHA512