I want to build a microservice that uses jrpc2 and mongodb to manage small IoT devices. These are the libraries in question:
https://godoc.org/bitbucket.org/creachadair/jrpc2
https://godoc.org/github.com/globalsign/mgo
The problem is, being rather new to Golang in general, I'm not sure how to combine these things together. Here's the code I have:
func DeviceAdd(ctx context.Context) (map[string]string, error) {
m := make(map[string]string)
m["token"] = "0xdeadbeef"
return m, nil
}
func DeviceBootstrap(ctx context.Context, params map[string]string) (map[string]string, error) {
m := make(map[string]string)
m["entered_token"] = params["token"]
return m, nil
}
...
func NewServer() *jrpc2.Server {
assigner := jrpc2.MapAssigner{
"device_add": jrpc2.NewHandler(DeviceAdd),
"device_bootstrap": jrpc2.NewHandler(DeviceBootstrap),
"device_update": jrpc2.NewHandler(DeviceUpdate),
"device_get_status": jrpc2.NewHandler(DeviceGetStatus),
}
srv := jrpc2.NewServer(assigner, nil)
return srv
}
Basically, you write some basic functions that accept a context.Context or any other JSON-RPC params, and jrpc2.NewHandler(function) will attach them to the jrpc2.Server.
The problem is, I cannot change the function handler signatures to put a mgo.Session in there. Or should I make DeviceAdd/DeviceBootstrap methods of a struct, which has access to a mgo.Session?
According to another StackOverflow question, it seems I should make DeviceAdd/DeviceBootstrap methods of a struct that has an mgo.Session, but I'm really not sure if I understood it correctly, or if jrpc2.NewHandler will accept the newly converted methods, or if it's the right thing to do. Also, Golang's context.Context seems like the right place to put such a thing. Help!
A request can be handled by any value that implements the jrpc2.Handler interface. Perhaps the simplest way to adapt an mgo.Session to this interface would be to wrap it in a struct, e.g.,
type SessionHandler struct {
mgo.Session
}
func (s SessionHandler) Handle(ctx context.Context, req *jrpc2.Request) (interface{}, error) {
// whatever you want your handler to do, using the s.Session
// as a delegate.
}
Then you can write something like:
a := handler.Map{"DoAThing": SessionHandler{s}}
to plug it into your server.
Related
I have the following (simplified) struct:
type newAppRegister struct {
SomeFlag *bool `json:"someflag" validate:"nonnil"`
ComputeLevel string `json:"compute-level" validate:"computelevelvalidator"`
}
And computelevelvalidator is some validation function.
I want that if SomeFlag is false, then ComputeLevel will be required and run his validation function.
A possible solution is to create a static variable, and set it in a custom validation function of SomeFlag, so for example:
var someFlag bool
func someFlagValidator(v interface{}, param string) error {
st := reflect.ValueOf(v)
if st.Kind() != reflect.Bool {
return fmt.Errorf("must be bool")
}
someFlag = st.Bool()
return nil
}
And then in computelevelvalidator it's possible to condition based on his value.
But since this is a REST API, and it may receive plenty of calls per second, I'm not sure if using this static variable will work (I'm worried about race conditions - but I'm not quite sure about it).
I'm using go1.11 and validation package gopkg.in/validator.v2 (version gopkg.in/validator.v2 v2.0.0-20190827175613-1a84e0480e5b).
So first up: Yes, using this static (global) variable will be a problem for concurrent use/access. It's not the way to go.
If your validation logic is contextual (like you said: if the flag is false, then certain restrictions apply on another field), then implementing a simple Validate function is pretty straightforward. I know the package you use supports this type of thing, but an external package will almost always be designed to be fairly generic. At the very least you'll end up performing type assertions. Last I checked, you still need an explicit call for the validation validator.Validate(), so why not move that to a method on your type?
type newAppRegister struct {} // your type
func (n newAppRegister) Validate() error {
if err := validator.Validate(n); err != nil {
return err
}
// at this point, we now the flag field isn't nit, because it passed validation
if !*n.SomeFlag {
// validate ComputeLevel here
}
return nil
}
No need for type assertions, let alone reflection. With just these two fields, I'd even argue there's no need for the validator package at all. You can get the same thing done with just standard JSON tags:
type Foo struct {
SomeFlag *bool `json:"some_flag,omitempty"`
ComputeLevel string `json:"compute_level"`
}
func (f Foo) Validate() error {
if f.SomeFlag == nil {
return ErrSomeFlagRequired
}
if !*f.SomeFlag {
// validate ComputeLevel
}
return nil
}
It's fairly straightforward to use:
f := Foo{}
if err := json.Unmarshal([]byte(payload), &f); err != nil {
// some shady JSON was submitted
}
if err := f.Validate(); err != nil {
// JSON was technically valid, but payload made no sense
}
// handle valid request
When doing forms with fields i want to send if there is a change i often do
let initialOrChangedName = Signal.merge(
nameChanged.signal,
self.viewDidLoadProperty.signal
.map { _ in nil }
)
where
private let nameChangedProperty = MutableProperty<String?>(nil)
private let viewDidLoadProperty = MutableProperty(())
to get a signal that has fired once on load, so i can use it in a combineLatest when user taps a button that will fire a web request with the form value to server. Since this signal merges it will give all values that change after the initial value, allowing me to send the newest value when user taps the submit button
Usage for this is usually something like
Signal.combineLatest(intialOrChangedName, initialOrChangedAge)
.sample(on:sendButtonTappedProperty.signal)
if values sent nil, i just dont include them in the web request, but i do get the other values if some of them was changed by user.
Since this is a fairly common pattern, i want to generalize it to a single function, for example
let initialOrChangedName = nameChanged.initialOrChangedState(on: viewDidLoadProperty)
I've tried writing it
extension MutableProperty where Value: OptionalProtocol {
public func initialOrChangedState(on viewDidLoadProperty: MutableProperty<Void>) -> Signal<Value?, Error> {
return Signal.merge(self.signal.map(Optional.init),
viewDidLoadProperty.signal.map { _ in nil})
}
}
Which looks good on paper, but will return String?? for the example given, and does not work.
I've also tried writing it as a static function on Signal, but with no luck.
I guess it should be something like this:
extension MutableProperty where Value: OptionalProtocol {
public func initialOrChangedState(on viewDidLoadProperty: MutableProperty<Void>) -> Signal<Value?, Error> {
return self.signal.map({Optional<Value>($0)}).merge(with: viewDidLoadProperty.signal.map({_ in nil}))
}
}
But, whats the point of using viewDidLoadProperty? Actually, if you subscribe to your signal at the end of viewDidLoad(), you don't even need such a property and as a result you wont need that merge() thing and you wont need to extend MutableProperty protocol.
So, all you need to do is something like:
submitButton.reactive.controlEvents(.touchUpInside).observer(on: UIScheduler()).observeValues({ _ in
readInputs()
launchRequest()
})
I might be misunderstanding so forgive me, if I am. But I think something like this might help. I have this method in a library I’ve written.
public func to<T>(_ value: T) -> Signal<T, Error> {
self.map { _ in value }
}
which allows you to do something like this
let voidProperty = MutableProperty(())
let nilStringSignal: Signal<String?, Never> = voidProperty.signal.to(nil)
So then maybe your case could be something like this, which leans a bit on type inference
nameChanged.signal.merge(with: self.viewDidLoadProperty.signal.to(nil))
I know maybe that’s not quite as concise as you want. Working with generics like optionals in signals can sometimes make the type wrangling a bit frustrating 😅
I'm just getting started with Golang and writing my first test suite.
I have a background in Rails, which has fantastic support for testing tools (Rspec, Cucumber, etc..), so I'm approaching my golang tests with a similar mindset (not sure if that's the right or wrong thing to do)
I have a User data model (basically a struct) that reads records from a users table in postgres and stores an array of them. (Essentially a really simple version of what ActiveRecord does in the Rails world)
I'd like to write a test that checks if the routine correctly reads from the DB and builds the models.
In almost every test suite I'll be connecting to the DB so I have a helper named establish_db_connection. Where can I place this so that it's centrally available to all my tests?
Building off #1 - is there an equivalent of a before block or some setup/teardown method where I can establish a connection before every test?
Lastly, how do I handle fixtures? Right now before each test I call a clear_db function that resets all tables and inserts some static data rows. I'd love to move away from fixtures and use factories to build data as needed (very similar to FactoryGirl in Rails), but not sure how common that is in Golang.
Is the built-in go test framework the best approach, or are there better alternatives?
Go is based on strong package management, meaning a namespace is treated as one single file. If establish_db_connection is used within a single test package, it can begin with a lowercase letter to signify a private instance and use it in the test file with the same package as the code being tested (Note that naming convention in Go is establishDBConnection).
However, most of the time, as in data/sql, you will want to obtain a DB connection once and keep that around until the test is finished (more like a factory and injection pattern).
There is none in the standard testing package. If you like BDD, Goconvey use scopes to define fixtures and a reset function for teardown.
You can use factory and dependency injections in your testing. I think that's pretty idiomatic.
A few includes Goconvey, Ginkgo and Testify They all have pros and cons of their own. The first two often end up with too many nested scopes, but Goconvey has a great browser-based real-time testing server which can be used with Go standard testing.
Since there's no global variables/functions in Go, you might design your project in interface-delegate pattern to help with importing functions cross packages and avoiding cyclic imports when dealing with cross-package testing.
mypackage
type DBOptions struct {
Name, Credentials string
}
func aFunc(db *sql.DB) error {
// do something
return nil
}
func bFunc(db *sql.DB) int, error {
// do something
return 0, nil
}
func establishConn(opts *DBOptions) (*sql.DB, error) {
db, err := sql.Open(opts.Name, opts.Credentials)
if err != nil {
return nil, err
}
return db, nil
}
func destroyConn(conn *sql.DB) {
conn.Close()
}
// test file
mypackage
import "testing"
var myOpt = &DBOptions{
Name: "mysql",
Credentials: "user:password#tcp(127.0.0.1:3306)/hello",
}
var conn, _ = establishConn(myOpt)
func TestAFunc(t *testing.T) {
err := aFunc(conn)
if err != nil {
t.Error(err)
}
}
func TestBFunc(t *testing.T) {
err := aFunc(conn)
if err != nil {
t.Error(err)
}
}
// use `conn` in other tests ...
destroyConn(conn)
About test fixture library similar to FactoryGirl in Rails, There are some choices in go.
those two libraries got stars most.
https://github.com/bluele/factory-go
https://github.com/go-testfixtures/testfixtures
And I also implemented test-fixture library which is type-safe, DRY, and flexible compared to the above libraries!
https://github.com/k-yomo/fixtory
on fixtures: consider passing functions in your testcases:
package main
import "testing"
type testcase struct {
scenario string
before func(string)
after func()
input string
expOutput string
}
var state = ""
func setup(s string) {
state = s
}
func nilSetup(s string) {}
func reset() {
state = ""
}
func execute(s string) string {
return state
}
func TestSetupTeardown(t *testing.T) {
tcs := []testcase{
{
scenario: "blank output when initial state is wrong",
before: nilSetup,
after: reset,
input: "foo",
expOutput: "",
},
{
scenario: "correct output when initial state is right",
before: setup,
after: reset,
input: "foo",
expOutput: "foo",
},
}
for _, tc := range tcs {
tc.before(tc.input)
if out := execute(tc.input); out != tc.expOutput {
t.Fatal(tc.scenario)
}
tc.after()
}
}
I built a tiny utility library to make it easy to create reusable fixtures to tests in go. Check out https://github.com/houqp/gtest to see if it solves your problem.
Here is a quick example on how to create a database transaction fixture for each test in a test group:
type TransactionFixture struct{}
// Construct can take other fixtures as input parameter as well
func (s TransactionFixture) Construct(t *testing.T, fixtures struct{}) (*sqlx.Tx, *sqlx.Tx) {
tx := // create db transaction here
return tx, tx
}
func (s TransactionFixture) Destruct(t *testing.T, tx *sqlx.Tx) {
tx.Rollback()
}
func init() {
// register and make fixture available to all tests
gtest.MustRegisterFixture(
"Transaction", &TransactionFixture{}, gtest.ScopeSubTest)
}
// begin of test definition
type SampleTests struct{}
func (s *SampleTests) Setup(t *testing.T) {
// you can create/initialize DB in this method
// DB instance can also be implemented as a fixture and get injected into Transanction fixture.
}
func (s *SampleTests) Teardown(t *testing.T) {
// you can clean up all DB resources in this method
}
func (s *SampleTests) BeforeEach(t *testing.T) {}
func (s *SampleTests) AfterEach(t *testing.T) {}
func (s *SampleTests) SubTestFoo(t *testing.T, fixtures struct {
Tx sqlx.Tx `fixture:"Transaction"`
}) {
// transaction is available as fixtures.Tx in this test
}
func TestSampleTests(t *testing.T) {
gtest.RunSubTests(t, &SampleTests{})
}
See https://godoc.org/github.com/houqp/gtest and https://github.com/houqp/gtest/blob/master/example_test.go for more advanced and uptodate examples.
I'm new to GOLANG - I would like to reduce the complexity of the handlers in my GO API. I'm coming from a Node.js background!
At the moment a route handler looks like this:
func getCards(c web.C, w http.ResponseWriter, r *http.Request) {
session := *MongoConnection().CreateSession()
defer session.Close()
collection := session.DB(db).C(cardsCollection)
result := []Card{}
err := collection.Find(bson.M{}).All(&result)
if err != nil {
panic(err)
}
w.Header().Set("Content-Type", "application/json")
encoder := json.NewEncoder(w)
encoder.Encode(result)
}
What I would like to do is return a collection for use without having to do this part:
session := *MongoConnection().CreateSession()
defer session.Close()
collection := session.DB(db).C(cardsCollection)
Instead I would like to do something like
collection := Card.Collection()
And have it create the session etc, Is this possible?
Why don't you have the session created in your main function, and pass it to the packages that need it. This is obviously missing a ton of stuff but the general idea would be
package main
//imports
func main() {
session := *MongoConnection().CreateSession()
defer session.Close()
Card.SetSession(session)
//other stuff
log.Fatal(http.ListenAndServe(":80", nil))
}
Then in Card
package Card
var session *mgo.Session
func SetSession(s *mgo.Session) {
session = s
}
func (c *Card) Collection() *mgo.Collection {
return session.DB(db).C(cardsCollection)
}
What you are describing is a factory pattern. But there are caveats to deal with.
Defer is local to scope it is called. So in case you would put the defer into a factory method, the session would be closed basically when it is returned.
Simply defining just one session und reuse it all over the place is pretty much the same as using a single SQL database connection and not a pool - it scales horribly.
Here is what I tend to do: I have my collections as global, and do the following at the beginning of my method
// Create a func-local session
myCollection := globalCollection.With( globalCollection.Database.Session.Copy() )
// Close it on exiting the scope
defer myCollection.Database.Session.Close()
Of course, you could externalize the first line into a factory, but that would not really declutter the code or make it more readable.
I am trying to write a general purpose wrapper for subscriptions, something like:
type Subscriber interface{
Subscribe(addr string) chan interface{}
}
Suppose there is a library I want to use which has a subscribe method in it, but which uses a chan library.Object. I would like to be able to do something like:
func (s *mySubscriber) Subscribe(addr string) chan interface{}{
ch := make(chan library.Object)
library.Subscribe(addr, ch)
return chan interface{}(ch)
}
Currently, I don't believe such a cast is possible. And I don't want to modify the underlying library, since the wrapper should be agnostic to library implementations.
I've seen Is there a way to cast Structs for sending over a channel, but in that case the application can be modified to suit the need. Here, it can't. Is this possible? Is there a better way?
One solution is to pass in a general purpose channel into Subscribe, and to wait indefinetely on chan library.Object and fire anything that comes through on my general channel, but I didn't particularly like having to introduce another channel just to get around the type cast.
No, you can't do this with just a cast. You have to use an extra channel, as you have already considered. Fortunately, there is a helper library for this already (disclaimer: I wrote it). You want the Wrap function.
Documentation: https://godoc.org/github.com/eapache/channels#Wrap
Code: https://github.com/eapache/channels/
For anyone else that stumbles on this issue and wants some inline code:
// wrap a timeout channel in a generic interface channel
func makeDefaultTimeoutChan() <-chan interface{} {
channel := make(chan interface{})
go func() {
<-time.After(30 * time.Second)
channel <- struct{}{}
}()
return channel
}
// usage
func main() {
resultChannel := doOtherThingReturningAsync()
cancel := makeDefaultTimeoutChan()
select {
case <-cancel:
fmt.Println("cancelled!")
case results := <-resultChannel:
fmt.Printf("got result: %#v\n", results)
}
}