I have a global variable that I am trying to use across two different functions, and unable to figure out why the following code is not working...
package main
import (
"github.com/ant0ine/go-json-rest/rest"
"log"
"net"
"net/http"
)
type Message struct {
Body string
}
var api rest.Api
func hostLookup(w rest.ResponseWriter, req *rest.Request) {
ip, err := net.LookupIP(req.PathParam("host"))
if err != nil {
rest.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.WriteJson(&ip)
}
func foo() {
api := rest.NewApi()
api.Use(rest.DefaultDevStack...)
router, err := rest.MakeRouter(
&rest.Route{"GET", "/lookup/#host", hostLookup},
)
if err != nil {
log.Fatal(err)
}
api.SetApp(router)
}
func bar() {
log.Fatal(http.ListenAndServe(":8080", api.MakeHandler()))
}
func main() {
foo()
bar()
}
The above code does not work... the HTTP server does not route the request to the hostLookup function.
However - if I move the following line from bar()
log.Fatal(http.ListenAndServe(":8080", api.MakeHandler()))
to the end of function foo(), then it works correctly
What am I doing wrong?
Your problem is two fold...
For one, you declare
var api rest.Api
but the rest.New() returns a *rest.Api
func NewApi() *Api {
Secondly, in your foo() function, you are creating a local variable called api instead of using your package variable.
Instead of
api := rest.NewApi()
It should be
api = rest.NewApi()
So, the fix is to add a * before rest.Api as in var api *rest.Api and remove a colon from the setting of api as in api = rest.NewApi()
Related
I'm fetching a document from MongoDB and passing it into function transform, e.g.
var doc map[string]interface{}
err := collection.FindOne(context.TODO(), filter).Decode(&doc)
result := transform(doc)
I want to write unit tests for transform, but I'm not sure how to mock a response from MongoDB. Ideally I want to set something like this up:
func TestTransform(t *testing.T) {
byt := []byte(`
{"hello": "world",
"message": "apple"}
`)
var doc map[string]interface{}
>>> Some method here to Decode byt into doc like the code above <<<
out := transform(doc)
expected := ...
if diff := deep.Equal(expected, out); diff != nil {
t.Error(diff)
}
}
One way would be to json.Unmarshal into doc, but this sometimes gives different results. For example, if the document in MongoDB has an array in it, then that array is decoded into doc as a bson.A type not []interface{} type.
A member from my team recently found out there is a hidden gem inside the official MongoDB driver for GO: https://pkg.go.dev/go.mongodb.org/mongo-driver#v1.9.1/mongo/integration/mtest. Although the package is in experimental mode and there is no backward compatibility guaranteed for it, it can help you to perform unit testing, at least with this version of the driver.
You can check this cool article with plenty of examples of how to use it: https://medium.com/#victor.neuret/mocking-the-official-mongo-golang-driver-5aad5b226a78. Additionally, here is the repository with the code samples for this article: https://github.com/victorneuret/mongo-go-driver-mock.
So, based in your example and the samples from the article I think you could try something like the following (of course, you might need to tweak and experiment with this):
func TestTransform(t *testing.T) {
mt := mtest.New(t, mtest.NewOptions().ClientType(mtest.Mock))
defer mt.Close()
mt.Run("find & transform", func(mt *mtest.T) {
myollection = mt.Coll
expected := myStructure{...}
mt.AddMockResponses(mtest.CreateCursorResponse(1, "foo.bar", mtest.FirstBatch, bson.D{
{"_id", expected.ID},
{"field-1", expected.Field1},
{"field-2", expected.Field2},
}))
response, err := myFindFunction(expected.ID)
if err != nil {
t.Error(err)
}
out := transform(response)
if diff := deep.Equal(expected, out); diff != nil {
t.Error(diff)
}
})
}
Alternatively, you can perform a more real testing and in an automated way via integration testing with Docker containers. There are a few good packages that could help you with this:
https://github.com/ory/dockertest
https://github.com/testcontainers/testcontainers-go
I have followed this approach with dockertest library to automate a full integration testing environment that could be setUp and tearDown via the go test -v -run Integration command. See a full example here: https://github.com/AnhellO/learn-dockertest/tree/master/mongo.
Hope this helps.
The best solution to write testable could would be to extract your code to a DAO or Data-Repository. You would define an interface which would return what you need. This way, you can just used a Mocked Version for testing.
// repository.go
type ISomeRepository interface {
Get(string) (*SomeModel, error)
}
type SomeRepository struct { ... }
func (r *SomeRepository) Get(id string) (*SomeModel, error) {
// Handling a real repository access and returning your Object
}
When you need to mock it, just create a Mock-Struct and implement the interface:
// repository_test.go
type SomeMockRepository struct { ... }
func (r *SomeRepository) Get(id string) (*SomeModel, error) {
return &SomeModel{...}, nil
}
func TestSomething() {
// You can use your mock as ISomeRepository
var repo *ISomeRepository
repo = &SomeMockRepository{}
someModel, err := repo.Get("123")
}
This is best used with some kind of dependency-injection, so passing this repository as ISomeRepository into the function.
Using monkey library to hook any function from mongo driver.
For example:
func insert(collection *mongo.Collection) (int, error) {
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
u := User{
Name: "kevin",
Age: 20,
}
res, err := collection.InsertOne(ctx, u)
if err != nil {
log.Printf("error: %v", err)
return 0, err
}
id := res.InsertedID.(int)
return id, nil
}
func TestInsert(t *testing.T) {
var c *mongo.Collection
var guard *monkey.PatchGuard
guard = monkey.PatchInstanceMethod(reflect.TypeOf(c), "InsertOne",
func(c *mongo.Collection, ctx context.Context, document interface{}, opts ...*options.InsertOneOptions) (*mongo.InsertOneResult, error) {
guard.Unpatch()
defer guard.Restore()
log.Printf("record: %+v, collection: %s, database: %s", document, c.Name(), c.Database().Name())
res := &mongo.InsertOneResult{
InsertedID: 100,
}
return res, nil
})
collection := client.Database("db").Collection("person")
id, err := insert(collection)
require.NoError(t, err)
assert.Equal(t, id, 100)
}
I wrote a small web service to learn unit testing. There is one endpoint to get data with a three-letter string. My code runs fine. The right query is http://localhost:8000/iata/thu, with the last bit thu being the three-letter string. I can get the correct data with it. I can also successfully get 404 with wrong ones. Then I wrote the test. It fails and dumps the entire database.
The SQLite3 database, main.go, and main_test.go are in the same directory.
Here's main_test.go:
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestIata(t *testing.T) {
// "thu" is the three-letter code.
// I also tried "http://localhost:8000/iata/thu"
req, err := http.NewRequest("GET", "/iata/thu", nil)
if err != nil {
t.Fatal(err)
}
rr := httptest.NewRecorder()
handler := http.HandlerFunc(iata)
handler.ServeHTTP(rr, req)
if status := rr.Code; status != http.StatusOK {
t.Errorf("handler returned wrong status code: got %v want %v",
status, http.StatusOK)
}
expected := `[{"airport_id":"10","name":"Thule Air Base","city":"Thule","country":"Greenland","iata":"THU","icao":"BGTL","latitude":"76.5311965942","longitude":"-68.7032012939","altitude":"251","timezone":"-4","dst":"E","tz_db":"America/Thule","type":"airport","source":"OurAirports"}]`
if rr.Body.String() != expected {
t.Errorf("handler returned unexpected body: got %v want %v",
rr.Body.String(), expected)
}
}
Here's main.go:
package main
import (
"database/sql"
"encoding/json"
"log"
"net/http"
"github.com/gorilla/mux"
_ "github.com/mattn/go-sqlite3"
)
type datum struct {
AirportID string `json:"airport_id,omitempty"`
...
}
func check(err error) {
...
}
// Accesses the database and gets relevant rows.
func getRows(column string, searchTerm string) *sql.Rows {
db, err := sql.Open("sqlite3", "airports.db")
check(err)
stmt := `SELECT * FROM airports WHERE ` + column + ` LIKE ? COLLATE NOCASE;`
rows, err := db.Query(stmt, `%`+searchTerm+`%`)
check(err)
return rows
}
// Processes the data into a slice so it can be sent out as JSON.
func processData(rows *sql.Rows) []datum {
data := []datum{}
// For each row, insert data into a datum instance and then append to data slice.
for rows.Next() {
datum := datum{}
rows.Scan(&datum.AirportID,
...)
data = append(data, datum)
}
rows.Close()
return data
}
// Uses the above code to get data from the database, process it, and send it.
func getAndSendData(w http.ResponseWriter, r *http.Request, searchType string) {
params := mux.Vars(r)
searchTerm := params[searchType]
datum := getRows(searchType, searchTerm)
processed := processData(datum)
if len(processed) == 0 {
http.Error(w, "Data not found.", 404)
return
}
json.NewEncoder(w).Encode(processed)
}
func main() {
router := mux.NewRouter()
router.HandleFunc("/iata/{iata}", iata).Methods("GET")
log.Fatal(http.ListenAndServe(":8000", router))
}
func iata(w http.ResponseWriter, r *http.Request) {
searchType := "iata"
getAndSendData(w, r, searchType)
}
Running the test gets the entire database dumped in the result:
=== RUN TestIata
--- FAIL: TestIata (0.21s)
main_test.go:46: handler returned unexpected body: got [ENTIRE DATABASE DUMPED HERE] want [CORRECT DATA HERE]
FAIL
exit status 1
I've looked through a lot of tutorials, such as this one, which I feel is pretty clear. As far as I can tell, my test code is correct. I've also tried running main.go before doing the test. But that shouldn't matter, right?
What am I missing?
I am trying to write a unit test using gotests and gomock to my restful service written in golang using gorilla but service fails to get variables from the url
Here is my request
req, err := http.NewRequest("GET", "product/5b5758f9931653c36bcaf0a0", nil)
actual endpoint is product/{id}
when I step into my service at the below code
params := mux.Vars(req)
params map is empty when it should have id key mapped to 5b5758f9931653c36bcaf0a0
Strange part is endpoint works fine from post man.
May I know whats wrong with the request?
This solved the issue
req = mux.SetURLVars(req, map[string]string{"id": "5b5758f9931653c36bcaf0a0"})
Since you're using GET requests, you can use the http.Get function, it works as expected:
package main
import (
"fmt"
"net/http"
"github.com/gorilla/mux"
)
func handle(w http.ResponseWriter, r *http.Request) {
params := mux.Vars(r)
fmt.Println(params)
}
func main() {
m := mux.NewRouter()
m.HandleFunc("/products/{id}", handle)
http.Handle("/", m)
go func() {
http.ListenAndServe(":8080", nil)
}()
_, err := http.Get("http://localhost:8080/products/765")
// Handle Error
}
If you really want to use http.NewRequest, that function doesn't actually execute the request so here is what you would need:
req, err := http.NewRequest("GET", "product/5b5758f9931653c36bcaf0a0", nil)
client := &http.Client{}
client.Do(req)
Create the mux Router in a separate function in the source code and call that directly in your test.
In Source code:
func Router() *mux.Router {
r := mux.NewRouter()
r.HandleFunc("/product/{id}", productHandler)
return r
}
func main() {
http.Handle("/", Router())
}
In Test:
func TestProductHandler(t *testing.T) {
r := http.NewRequest("GET", "product/5b5758f9931653c36bcaf0a0", nil)
w := httptest.NewRecorder()
Router().ServeHTTP(w, r)
}
Found the related solution in one of the google groups forums.
https://groups.google.com/forum/#!msg/golang-nuts/Xs-Ho1feGyg/xg5amXHsM_oJ
Searching SO for Generic REST API Golang gives 0 results. Searching Google gives 2 results. So this question is maybe not correctly formulated or it is impossible to achieve in Golang.
My goal is to avoid repeating similar code over and over again. So I am trying to make the code in Golang as generic as possible. Write once, use many.
This is my first attempt to create a generic REST API for select in Golang. The code below gives almost what I want:
But the result is presented in the Terminal. I have no idea how to redirect the result to the browser.
package main
import (
"fmt"
"log"
"net/http"
"database/sql"
"time"
_ "github.com/lib/pq"
)
var db *sql.DB
func main() {
Connect()
http.HandleFunc("/", Query)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func Connect() {
const (
host = "127.0.0.1"
port = 5432
user = "test"
password = "test"
dbname = "Test")
login := fmt.Sprintf("host=%s port=%d user=%s "+"password=%s dbname=%s sslmode=require", host, port, user, password, dbname)
var err error
db, err = sql.Open("postgres", login)
if err != nil {
log.Fatalln(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
func Query(w http.ResponseWriter, r *http.Request) {
var query string
switch r.URL.String() {
case "/getuser":
query = "select * from getuser()"
case "/getco":
query = "select * from getco()"
case "/etc"
query = "select * from etc"
default:
query = ""
}
var err error
var rows *sql.Rows
rows, err = db.Query(query)
if err != nil {
http.Error(w, http.StatusText(500), 500)
return
}
defer rows.Close()
cols, err := rows.Columns()
vals := make([]interface{}, len(cols))
for i := 0; i < len(cols); i++ {
vals[i] = new(interface{})
if i != 0 {
fmt.Print("\t")
}
fmt.Print(cols[i])
}
fmt.Println()
for rows.Next() {
err = rows.Scan(vals...)
if err != nil {
fmt.Println(err)
continue
}
for i := 0; i < len(vals); i++ {
if i != 0 {
fmt.Print("\t")
}
printValue(vals[i].(*interface{}))
}
fmt.Println()
}
func printValue(pval *interface{}) {
switch v := (*pval).(type) {
case nil:
fmt.Print("NULL")
case bool:
if v {
fmt.Print("1")
} else {
fmt.Print("0")
}
case []byte:
fmt.Print(string(v))
case time.Time:
fmt.Print(v.Format("2006-01-02"))
default:
fmt.Print(v)
}
}
Every attempt to write to the browser gives various type of errors:
fmt.Printf("%s\n", vals...)
My questions are
How do I redirect the result to the browser?
Is there any better way to achieve this? (reuse generic code)
My recommendation would be to look at using existing packages like "mux" for calling REST APIs in browser. As a quick demo how you would do it as as follows:
your restapi.go cound have APIs as follows:
func SampleAPI(w http.ResponseWriter, r *http.Request) { //Assuming this is a POST request
var example SomeSruct
_ = json.NewDecoder(r.Body).Decode(&example) //Decode the POST body
result := someLogicFunction(example) //call your generic function
json.NewEncoder(w).Encode(result) //encode the result to pass it back to browser
}
Now say you write a main.go and you are using mux package here is an example of how you would call this
main.go
func main() {
router := mux.NewRouter()
router.HandleFunc("/testFunc",restapi.SampleAPI).Methods("POST") //This creates the route for your http request
handler := cros.Default().Handler(router) //You will need this if you plan to deploy it in a server and call it externally for testing locally you don't need this
log.Fatal(http.ListenAndServe(":8080", handler)) //Port that the router is listening to
}
Now note that you will have to import the "github.com/gorilla/mux" and the "github.com/rs/cors" packages to use these but this way you can create REST APIs whic can be accessed by te browser. Similarly you could create a GET method and use parameters which you can grab in your function and perform any logical step.
If you build and install the above code you can POST to localhost:8080/testFunc over http using any web app and get results i your browser. If you had a GET request you could directly type the Url in the browser and see the result.
write response with appropriahe HTTP hearers && status code
import "net/http"
func writeResponse(w http.ResponseWriter, contents []byte) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(http.StatusOK)
fmt.Fprintln(w, contents)
}
sounds a little unclear, sorry
I want to close an mgo session after it' fully streamed to the client. At first, I thought this might work, but it seems the defer just waits until the the func begins to return or something.
func (c App) OpenFile(fileId string) revel.Result {
// convert string to bson.ObjectId
objId := bson.ObjectIdHex(fileId)
file, session := OpenFile(objId)
defer session.Close()
// memfile io.Reader, filename string, delivery ContentDisposition, modtime time.Time
var now = time.Now()
return c.RenderBinary(file, "filename", "inline", now)
}
I have no idea what your OpenFile function doing but I assume it using Mongo's GridFS to access the file. You are right defer session.Close() is called right before the return. If you use file Revel closes it automatically but in your case, you also want to close the session.
The only one way around it is to make custom Reader that can be used by Revel in RenderBinary. Something like:
package main
import (
"io"
"os"
"gopkg.in/mgo.v2"
)
type MongoReader struct {
io.Reader
File *os.File
Session *mgo.Session
}
func (mr *MongoReader) Read(p []byte) (n int, e error) {
return mr.File.Read(p)
}
func (mr *MongoReader) Close() {
mr.File.Close()
mr.Session.Close()
}
func main() {
mr := MongoReader{}
mr.Session, mr.File = OpenFile("abababa")
return c.RenderBinary(mr, "filename", "inline", now)
}
I didn't test it on Revel but as code is quite simple I think it should work. Maybe you will need some os.File specific methods.