I am using go-couchbase to update data to couchbase, however, I have problem in how to use the callback function.
The function Update requires me to pass a callback function in which should be UpdateFunc
func (b *Bucket) Update(k string, exp int, callback UpdateFunc) error
So that's what I have done
First, I declared a type UpdateFunc:
type UpdateFunc func(current []byte) (updated []byte, err error)
Then in the code, I add the following lines:
fn := UpdateFunc{func(0){}}
and then call the Update function:
bucket.Update("12345", 0, fn()}
But the Go returns the following error:
syntax error: unexpected literal 0, expecting ) for this line fn := UpdateFunc{func(0){}}
So what I am doing wrong? So how can I make the callback function work ?
additional information
Thanks all of your suggestion. Now I can run the callback function as follows:
myfunc := func(current []byte)(updated []byte, err error) {return updated, err }
myb.Update("key123", 1, myfunc)
However, when I run the Update function of the bucket. I checked the couch database. The document with the key of "key123" was disappeared. It seems the Update does not update the value but delete it. What happened?
You need to create a function that matches the couchbase.UpdateFunc signature then pass it to bucket.Update.
For example:
fn := func(current []byte) (updated []byte, err error) {
updated = make([]byte, len(current))
copy(updated, current)
//modify updated
return
}
....
bucket.Update("12345",0,fn)
Notice that to pass a function you just pass fn not fn(), that would actually call the function right away and pass the return value of it.
I highly recommend stopping everything you're doing and reading Effective Go and all the posts on Go's blog starting with First Class Functions in Go.
Same answer I posted on the go-nuts list, just in case:
You don't have to define the UpdateFunc type by yourself, it's already defined here:
https://github.com/couchbaselabs/go-couchbase/blob/master/client.go#L608
just define the function normally and pass it as an argument or pass an anonymous function as in other languages.
here's a simple example:
http://play.golang.org/p/YOKxRtQqmU
Related
I am trying to open a connection to a postgres database using pgx and I am getting the following error:
./dbservice.go:12:26: too many arguments in call to "github.com/jackc/pgx".Connect
have (context.Context, string)
want ("github.com/jackc/pgx".ConnConfig)
./dbservice.go:13:18: too many arguments in call to conn.Close
have (context.Context)
want ()
./dbservice.go:21:44: cannot use context.Background() (type context.Context) as type string in argument to conn.Query
I am not sure what the error is asking me to do here. pgx.Connect works when I call it from the main file but here it doesn't work. Here's the code:
func initNodes(nodes *[]Node, searchNodes *[]SearchNode, storageNodes *[]StorageNode) error {
conn, err := pgx.Connect(context.Background(), DATABATE_URL)
defer conn.Close(context.Background())
if err != nil {
fmt.Printf("Connection failed: %v\n", err)
os.Exit(-1)
}
...
func main() {
a:= Arbiter{}
a.init()
}
Any ideas?
Most likely you are importing the v3 pgx API in dbservice.go, but the v4 API in your “main file”. The Connect function in github.com/jackc/pgx/v4 accepts the two arguments you're passing. The Connect function in v3 accepts a pkg.ConnConfig instead.
So, check your import statements in dbservice.go: if you intend to be using the v4 API, import it as "github.com/jackc/pgx/v4".
(sorry this question turned out longer than I had thought...)
I'm using Go and MongoDB with the mgo driver. I'm trying to persist and retrieve different structs (implementing a common interface) in the same MongoDB collection. I'm coming from the Java world (where this is very easily done with Spring with literally no config) and I'm having a hard time doing something similar with Go.
I have read every last related article or post or StackExchange question I could find, but still I haven't found a complete solution. This includes:
Unstructured MongoDB collections with mgo
How do you create a new instance of a struct from its type at run time in Go?
Golang reflect: Get Type representation from name?
Here's a simplified setup I use for testing. Suppose two structs S1 and S2, implementing a common interface I. S2 has an implicit field of type S1, meaning that structure-wise S2 embeds a S1 value, and that type-wise S2 implements I.
type I interface {
f1()
}
type S1 struct {
X int
}
type S2 struct {
S1
Y int
}
func (*S1) f1() {
fmt.Println("f1")
}
Now I can save an instance of S1 or S2 easily using mgo.Collection.Insert(), but to properly populate a value using mgo.Collection.Find().One() for example, I need to pass a pointer to an existing value of S1 or S2, which means I already know the type of the object I want to read!!
I want to be able to retrieve a document from the MongoDB collection without knowing if it's a S1, or a S2, or in fact any object that implements I.
Here's where I am so far: instead of directly saving the object I want to persist, I save a Wrapper struct that holds the MongoDB id, an identifier of the type, and the actual value. The type identifier is a concatenation of packageName + "." + typeName, and it is used to lookup the type in a type registry, since there is no native mechanism to map from a type name to a Type object in Go. This means I need to register the types that I want to be able to persist and retrieve, but I could live with that. Here's how it goes:
typeregistry.Register(reflect.TypeOf((*S1)(nil)).Elem())
typeregistry.Register(reflect.TypeOf((*S2)(nil)).Elem())
Here's the code for the type registry:
var types map[string]reflect.Type
func init() {
types = make(map[string]reflect.Type)
}
func Register(t reflect.Type) {
key := GetKey(t)
types[key] = t
}
func GetKey(t reflect.Type) string {
key := t.PkgPath() + "." + t.Name()
return key
}
func GetType(key string) reflect.Type {
t := types[key]
return t
}
The code for saving an object is quite straightforward:
func save(coll *mgo.Collection, s I) (bson.ObjectId, error) {
t := reflect.TypeOf(s)
wrapper := Wrapper{
Id: bson.NewObjectId(),
TypeKey: typeregistry.GetKey(t),
Val: s,
}
return wrapper.Id, coll.Insert(wrapper)
}
The code for retrieving an object is a bit more tricky:
func getById(coll *mgo.Collection, id interface{}) (*I, error) {
// read wrapper
wrapper := Wrapper{}
err := coll.Find(bson.M{"_id": id}).One(&wrapper)
if err != nil {
return nil, err
}
// obtain Type from registry
t := typeregistry.GetType(wrapper.TypeKey)
// get a pointer to a new value of this type
pt := reflect.New(t)
// FIXME populate value using wrapper.Val (type bson.M)
// HOW ???
// return the value as *I
i := pt.Elem().Interface().(I)
return &i, nil
}
This partially works as the returned object is typed correctly, but what i can't figure out is how to populate the value pt with the data retrieved from MongoDB which is stored in wrapper.Val as a bson.M.
I have tried the following but it doesn't work:
m := wrapper.Val.(bson.M)
bsonBytes, _ := bson.Marshal(m)
bson.Unmarshal(bsonBytes, pt)
So basically the remaining problem is: how to populate an unknown structure from a bson.M value? I'm sure there has to be an easy solution...
Thanks in advance for any help.
Here's a Github gist with all the code: https://gist.github.com/ogerardin/5aa272f69563475ba9d7b3194b12ae57
First, you should always check returned errors, always. bson.Marshal() and bson.Unmarshal() return errors which you don't check. Doing so reveals why it doesn't work:
unmarshal can't deal with struct values. Use a pointer
pt is of type reflect.Value (which happens to be a struct), not something you should pass to bson.Unmarshal(). You should pass e.g. a pointer to a struct value you want to unmarshal into (which will be wrapped in an interface{} value). So call Value.Interface() on the value returned by reflect.New():
pt := reflect.New(t).Interface()
You can pass this to bson.Unmarshal():
bsonBytes, err := bson.Marshal(m)
if err != nil {
panic(err)
}
if err = bson.Unmarshal(bsonBytes, pt); err != nil {
panic(err)
}
(In your real code you want to do something else than panic, this is just to show you should always check errors!)
Also note that it is possible to directly convert maps to structs (directly meaning without marshaling and unmarshaling). You may implement it by hand or use a ready 3rd party lib. For details, see Converting map to struct
Also note that there are more clever ways to solve what you want to do. You could store the type in the ID itself, so if you have the ID, you can construct a value of the type to unmarshal into the query result, so you could skip this whole process. It would be a lot more simple and a lot more efficient.
For example you could use the following ID structure:
<type>-<id>
For example:
my.package.S1-123
When fetching / loading this document, you could use reflection to create a value of my.package.S1, and unmarshal into that directly (pass that to Query.One()).
As per #icza 's comments, here's a modified version of getById() that actually works:
func getById(coll *mgo.Collection, id interface{}) (*I, error) {
// read wrapper
wrapper := Wrapper{}
err := coll.Find(bson.M{"_id": id}).One(&wrapper)
if err != nil {
return nil, err
}
// obtain Type from registry
t := typeregistry.GetType(wrapper.TypeKey)
// get a pointer to a new value of this type
pt := reflect.New(t)
// populate value using wrapper.Val
err = mapstructure.Decode(wrapper.V, pt.Interface())
if err != nil {
return nil, err
}
// return the value as *I
i := pt.Elem().Interface().(I)
return &i, nil
}
Conversion from bson.M to the struct is handled by https://github.com/mitchellh/mapstructure instead of marshalling-unmarshaling.
In the current project we use Go and MongoDB via mgo driver.
For every entity we have to implement DAO for CRUD operations, and it's basically copy-paste, e.g.
func (thisDao ClusterDao) FindAll() ([]*entity.User, error) {
session, collection := thisDao.getCollection()
defer session.Close()
result := []*entity.User{} //create a new empty slice to return
q := bson.M{}
err := collection.Find(q).All(&result)
return result, err
}
For every other entity it's all the same but the result type.
Since Go has no generics, how could we avoid code duplication?
I've tried to pass the result interface{} param instead of creating it in the method, and call the method like this:
dao.FindAll([]*entity.User{})
but the collection.Find().All() method need a slice as the input, not just interface:
[restful] recover from panic situation: - result argument must be a slice address
/usr/local/go/src/runtime/asm_amd64.s:514
/usr/local/go/src/runtime/panic.go:489
/home/dds/gopath/src/gopkg.in/mgo.v2/session.go:3791
/home/dds/gopath/src/gopkg.in/mgo.v2/session.go:3818
Then I tried to make this param result []interface{}, but in that case it's impossible to pass []*entity.User{}:
cannot use []*entity.User literal (type []*entity.User) as type []interface {} in argument to thisDao.GenericDao.FindAll
Any idea how could I implement generic DAO in Go?
You should be able to pass a result interface{} to your FindAll function and just pass it along to mgo's Query.All method since the argument's would have the same type.
func (thisDao ClusterDao) FindAll(result interface{}) error {
session, collection := thisDao.getCollection()
defer session.Close()
q := bson.M{}
// just pass result as is, don't do & here
// because that would be a pointer to the interface not
// to the underlying slice, which mgo probably doesn't like
return collection.Find(q).All(result)
}
// ...
users := []*entity.User{}
if err := dao.FindAll(&users); err != nil { // pass pointer to slice here
panic(err)
}
log.Println(users)
I will be giving the following as the command to run my go program.
go run app.go 3001-3005
What this is supposed to do is, run my go Rest api on server ports 3001 to 3005.
This part of my main function which handles this argument.
func main() {
ipfile := os.Args[1:]
s := strings.Split(ipfile, "-")
mux := routes.New()
mux.Put("/:key1/:value1", PutData)
mux.Get("/profile/:key1", GetSingleData)
mux.Get("/profile", GetData)
http.Handle("/", mix)
Here I will run a for loop and replace the first argument with s[i].
http.ListenAndServe(":3000", nil)
}
I get the following output:
cannot use ipfile (type []string) as type string in argument to strings.Split
What datatype does os.args return?
I tried converting it to string and then splitting. Does not work.
Please let me know what is wrong?
Like the error says, ipfile is a []string. The [1:] slice operation is going to return a slice, even if there's only 1 element.
After checking that os.Args has the enough elements, use:
ipfile := os.Args[1]
s := strings.Split(ipfile, "-")
I am having difficulty understanding how to set an interface value that has been passed as a pointer. I am trying to accomplish something along the lines of this:
import "fmt"
var Stuff map[string]interface{}
func main() {
var num int
Stuff["key"] = 9001
get("key", &num)
fmt.Println("num:", num)
}
func get(k string, v interface{}) {
*v = Stuff[k]
}
What would I have to do to make my program output be
num: 9001
Edit: is there a possible catch-all solution using reflect?
You can emulate the AppEngine datastore interface using reflect; usually I say minimize reflection, but you (and AppEngine and other ORMs) have no other great option here to present the interface you want. For something emulating Get you:
get a reflect.Value with ValueOf()
get the type of the thing you want to create
create it with reflect.Zero
optionally fill in some data with reflect.Field(), etc.
use reflect.Indirect() and Value.Set() to set the original through the pointer.
A trivial example that just zeroes a struct through a pointer is at http://play.golang.org/p/g7dNlrG_vr and copied here:
package main
import (
"fmt"
"reflect"
)
func main() {
i := 1
clear(&i)
fmt.Println(i)
}
func clear(dst interface{}) {
// ValueOf to enter reflect-land
dstPtrValue := reflect.ValueOf(dst)
// need the type to create a value
dstPtrType := dstPtrValue.Type()
// *T -> T, crashes if not a ptr
dstType := dstPtrType.Elem()
// the *dst in *dst = zero
dstValue := reflect.Indirect(dstPtrValue)
// the zero in *dst = zero
zeroValue := reflect.Zero(dstType)
// the = in *dst = 0
dstValue.Set(zeroValue)
}
For emulating GetMulti you need more steps to work with the slice. An example is at http://play.golang.org/p/G_6jit2t-2 and below:
package main
import (
"fmt"
"reflect"
)
func main() {
s := []int{}
getMultiZeroes(&s, 10)
fmt.Println(s)
}
func getMultiZeroes(slicePtrIface interface{}, howMany int) {
// enter `reflect`-land
slicePtrValue := reflect.ValueOf(slicePtrIface)
// get the type
slicePtrType := slicePtrValue.Type()
// navigate from `*[]T` to `T`
sliceElemType := slicePtrType.Elem().Elem() // crashes if input type not `*[]T`
// we'll need this to Append() to
sliceValue := reflect.Indirect(slicePtrValue)
// and this to Append()
sliceElemValue := reflect.Zero(sliceElemType)
// append requested number of zeroes
for i := 0; i < howMany; i++ {
// s := append(s, v)
sliceValue.Set(reflect.Append(sliceValue, sliceElemValue))
}
}
In live code (as opposed to testing like you're doing), it'd be faster to use a type switch (as Martin suggested) so that specialized native code runs for each type; that might also be handy if you have different behavior by type. An example for GetMulti is at http://play.golang.org/p/q-9WyUqv6P and below:
package main
import "fmt"
func main() {
s := []int{}
getZeroes(&s)
fmt.Println(s)
fails := []float32{}
getZeroes(&fails)
}
func getZeroes(slicePtrIface interface{}) {
switch sp := slicePtrIface.(type) {
case *[]int:
(*sp) = append((*sp), 0, 0)
case *[]string:
(*sp) = append((*sp), "", "")
default:
panic(fmt.Sprintf("getZeroes: passed type %T, which is not a pointer to a slice of a supported type", slicePtrIface))
}
}
You could even trivially combine the two; write custom code for common types and call the slow reflect-based version in the default case. Demo at http://play.golang.org/p/6qw52B7eC3 (not copying because it's a such a simple stitching together of the two above).
There happened to be another recent question on how to make a value to pass to GetMulti, rather than emulating the GetMulti itself, if that comes up.
More for general reference than to answer this:
"Go lacks pass by reference" is useful to know, but also needs some elaboration. Go has pointers, and other types like slices that contain pointers to data. The sense in which there isn't "pass by reference" is just that Go will never change a value argument (int, struct) into a pointer implicitly. C++ reference arguments do exactly that: C++ void f(i int&) { i++; } changes i in the caller without the caller explicitly passing in a pointer at the callsite. func (i int) { i++ } doesn't.
In Go, you can look at the types passed to a function call and tell what data it can change. With C++ reference arguments or some languages' "pass by reference" semantics, any call might change locals; you can't tell without looking up the declarations.
For purposes of avoiding unnecessary copying of data, there are already pointers in the implementations of slice, string, map, interface, and channel values. Of those types, pointers, slices, and maps will actually let you modify data through them. Also, like in C++, Go's this-like receiver parameter can be a pointer without an explicit & in the calling code. There's more about this in Russ Cox's godata post and this summary on when you need a pointer or not.
The Go Programming Language Specification
Calls
In a function call, the function value and arguments are evaluated in
the usual order. After they are evaluated, the parameters of the call
are passed by value to the function and the called function begins
execution. The return parameters of the function are passed by value
back to the calling function when the function returns.
In Go everything is passed by value; nothing is passed by reference. Therefore, pass a pointer. For example,
package main
import "fmt"
var Stuff map[string]interface{}
func main() {
Stuff = make(map[string]interface{})
Stuff["key"] = 9001
var value interface{}
get("key", &value)
num := value.(int)
fmt.Println("num:", num)
}
func get(k string, v interface{}) {
*v.(*interface{}) = Stuff[k]
}
Output:
num: 9001
First: There is absolutely no concept of "pass by reference" in Go. There isn't. What you can do is pass around a pointer. This pointer value is passed by value as everything in Go is passed by value.
Second: Instead of passing in a pointer to an interface and modify the pointees value (doable but ugly) you could return the value (much nicer).
Third: It cannot (i.e. not without reflection or unsafe) be done without type assertions.
And you should never (in the sense of "no until you mastered Go and interfaces") use pointer to interface.
Fifth: If your solution requires interface{} you might be doing something wrong. Are you sure your entities are not describable by some (non empty) interface?
That said, something like that works.
func main() {
var num int
Stuff["key"] = 9001
num = get("key").(int)
}
func get(k string) interface{}{
return Stuff[k]
}
Martin Gallagher solution works perfectly, but as he said, you can't use generics in Golang, so code looks a bit ugly. I guess another solution is to use always interface{} as the type and then cast (or check the type) in your program. Something like this: http://play.golang.org/p/0o20jToXHV
http://play.golang.org/p/kx5HvEiOm9
Without generics you will have to implement the switch {} for each of your supported types.