I'm new in Go and unit test. I build a samll side projecy called "urlshortener" using Go with Gorm, mux and postgresql.
There is a qeustion annoying me after search many articles.
To make the question clean, I delete some irrelevant code like connect db, .env, etc
My code is below(main.go):
package main
type Url struct {
ID uint `gorm:"primaryKey"` // used for shortUrl index
Url string `gorm:"unique"` // prevent duplicate url
ExpireAt string
ShortUrl string
}
var db *gorm.DB
var err error
func main() {
// gain access to database by getting .env
...
// database connection string
...
// make migrations to the dbif they have not already been created
db.AutoMigrate(&Url{})
// API routes
router := mux.NewRouter()
router.HandleFunc("/{id}", getURL).Methods("GET")
router.HandleFunc("/api/v1/urls", createURL).Methods("POST")
router.HandleFunc("/create/urls", createURLs).Methods("POST")
// Listener
http.ListenAndServe(":80", router)
// close connection to db when main func finishes
defer db.Close()
}
Now I'm building unit test for getURL function, which is a GET method to get data from my postgresql database called urlshortener and the table name is urls.
Here is getURL function code:
func getURL(w http.ResponseWriter, r *http.Request) {
params := mux.Vars(r)
var url Url
err := db.Find(&url, params["id"]).Error
if err != nil {
w.WriteHeader(http.StatusNotFound)
} else {
w.WriteHeader(http.StatusOK)
json.NewEncoder(w).Encode(url.Url)
}
}
This is work fine with my database. See curl command below:
I know that the unit test is not for mock data, and it aim to test a function/method is stable or not. Although I import mux and net/http for conncetion, but I think the unit test on it should be "SQL syntax". So I decide to focus on testing if gorm return the right value to the test function.
In this case, db.Find will return a *gorm.DB struct which should be exactly same with second line. (see docs https://gorm.io/docs/query.html)
db.Find(&url, params["id"])
SELECT * FROM urls WHICH id=<input_number>
My question is how to write a unit test on it for check the SQL syntax is correct or not in this case (gorm+mux)? I've check some articles, but most of them are testing the http connect status but not for SQL.
And my function do not have the return value, or I need to rewrite the function to have a return value before I can test it?
below is the test structure in my mind:
func TestGetURL(t *testing.T) {
//set const answer for this test
//set up the mock sql connection
//call getURL()
//check if equal with answer using assert
}
Update
According to #Emin Laletovic answer
Now I have a prototype of my testGetURL. Now I have new questions on it.
func TestGetURL(t *testing.T) {
//set const answer for this test
testQuery := `SELECT * FROM "urls" WHERE id=1`
id := 1
//set up the mock sql connection
testDB, mock, err := sqlmock.New()
if err != nil {
panic("sqlmock.New() occurs an error")
}
// uses "gorm.io/driver/postgres" library
dialector := postgres.New(postgres.Config{
DSN: "sqlmock_db_0",
DriverName: "postgres",
Conn: testDB,
PreferSimpleProtocol: true,
})
db, err = gorm.Open(dialector, &gorm.Config{})
if err != nil {
panic("Cannot open stub database")
}
//mock the db.Find function
rows := sqlmock.NewRows([]string{"id", "url", "expire_at", "short_url"}).
AddRow(1, "http://somelongurl.com", "some_date", "http://shorturl.com")
mock.ExpectQuery(regexp.QuoteMeta(testQuery)).
WillReturnRows(rows).WithArgs(id)
//create response writer and request for testing
mockedRequest, _ := http.NewRequest("GET", "/1", nil)
mockedWriter := httptest.NewRecorder()
//call getURL()
getURL(mockedWriter, mockedRequest)
//check values in mockedWriter using assert
}
In the code, I mock the request and respone with http, httptest libs.
I run the test, but it seems that the getURL function in main.go cannot receive the args I pass in, see the pic below.
when db.find called, mock.ExpectQuery receive it and start to compare it, so far so good.
db.Find(&url, params["id"])
mock.ExpectQuery(regexp.QuoteMeta(testQuery)).WillReturnRows(rows).WithArgs(id)
According to the testing log, it shows that when db.Find triggerd, it only excute SELECT * FROM "urls" but not I expected SELECT * FROM "urls" WHERE "urls"."id" = $1.
But when I test db.Find on local with postman and log the SQL syntax out, it can be excute properly. see pic below.
In summary, I think the problem is the responeWriter/request I put in getURL(mockedWriter, mockedRequest) are wrong, and it leads that getURL(w http.ResponseWriter, r *http.Request) cannot work as we expect.
Please let me know if I missing anything~
Any idea or way to rewrite the code would be help, thank you!
If you just want to test the SQL string that db.Find returns, you can use the DryRun feature (per documentation).
stmt := db.Session(&Session{DryRun: true}).Find(&url, params["id"]).Statement
stmt.SQL.String() //returns SQL query string without the param value
stmt.Vars // contains an array of input params
However, to write a test for the getURL function, you could use sqlmock to mock the results that would be returned when executing the db.Find call.
func TestGetURL(t *testing.T) {
//set const answer for this test
testQuery := "SELECT * FROM `urls` WHERE `id` = $1"
id := 1
//create response writer and request for testing
//set up the mock sql connection
testDB, mock, err := sqlmock.New()
//handle error
// uses "gorm.io/driver/postgres" library
dialector := postgres.New(postgres.Config{
DSN: "sqlmock_db_0",
DriverName: "postgres",
Conn: testDB,
PreferSimpleProtocol: true,
})
db, err = gorm.Open(dialector, &gorm.Config{})
//handle error
//mock the db.Find function
rows := sqlmock.NewRows([]string{"id", "url", "expire_at", "short_url"}).
AddRow(1, "http://somelongurl.com", "some_date", "http://shorturl.com")
mock.ExpectQuery(regexp.QuoteMeta(testQuery)).
WillReturnRows(rows).WithArgs(id)
//call getURL()
getUrl(mockedWriter, &mockedRequest)
//check values in mockedWriter using assert
}
This Post and Emin Laletovic are really helps me alot.
I think I get the answer to this qeustion.
Let's recap this questioon. First, I'm using gorm for postgresql and mux for http services and build a CRUD service.
I need to write a unit test to check if my database syntax is correct (we assuming that the connection is statusOK), so we focus on how to write a unit test for SQL syntax.
But the handler function in main.go don't have return value, so we need to use mock-sql/ ExpectQuery(), this function will be triggered when the db.Find() inside getURL(). By doing this, we dont have to return a value to check if it match our target or not.
The problem I met in Update is fixed by This Post, building an unit test with mux, but that post is focusing on status check and return value.
I set the const answer for this test, the id variable is what we expect to get. Noticed that $1 I don't know how to change it, and I've try many times to rewrite but SQL syntax is still return $1, maybe it is some kind of constraint I dont know.
//set const answer for this test
testQuery := `SELECT * FROM "urls" WHERE "urls"."id" = $1`
id := "1"
I set the value pass into the getURL() by doint this
//set the value send into the function
vars := map[string]string{
"id": "1",
}
//create response writer and request for testing
mockedWriter := httptest.NewRecorder()
mockedRequest := httptest.NewRequest("GET", "/{id}", nil)
mockedRequest = mux.SetURLVars(mockedRequest, vars)
Finally, we call mock.ExpectationsWereMet() to check if anything went wrong.
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("SQL syntax is not match: %s", err)
}
Below is my test code:
func TestGetURL(t *testing.T) {
//set const answer for this test
testQuery := `SELECT * FROM "urls" WHERE "urls"."id" = $1`
id := "1"
//set up the mock sql connection
testDB, mock, err := sqlmock.New()
if err != nil {
panic("sqlmock.New() occurs an error")
}
// uses "gorm.io/driver/postgres" library
dialector := postgres.New(postgres.Config{
DSN: "sqlmock_db_0",
DriverName: "postgres",
Conn: testDB,
PreferSimpleProtocol: true,
})
db, err = gorm.Open(dialector, &gorm.Config{})
if err != nil {
panic("Cannot open stub database")
}
//mock the db.Find function
rows := sqlmock.NewRows([]string{"id", "url", "expire_at", "short_url"}).
AddRow(1, "url", "date", "shorurl")
//try to match the real SQL syntax we get and testQuery
mock.ExpectQuery(regexp.QuoteMeta(testQuery)).WillReturnRows(rows).WithArgs(id)
//set the value send into the function
vars := map[string]string{
"id": "1",
}
//create response writer and request for testing
mockedWriter := httptest.NewRecorder()
mockedRequest := httptest.NewRequest("GET", "/{id}", nil)
mockedRequest = mux.SetURLVars(mockedRequest, vars)
//call getURL()
getURL(mockedWriter, mockedRequest)
//check result in mockedWriter mocksql built function
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("SQL syntax is not match: %s", err)
}
}
And I run two tests with args(1, 1) and args(1, 2), and it works fine. see pic below(please ignore the chinese words)
I am writing a web app in go and using the GORM for my ORM. I need to be able to retrieve all the metrics of a certain user and return it via JSON to be displayed on the front end. The query seems to run successfully but I only see a memory address when printing the results and receive an error when trying to cast the results the standard way.
Here is my current code
func DisplayData(w http.ResponseWriter, r *http.Request) {
//Get the data from the database
var metric models.Metric
results := db.Where("user_id = ?", "1").Find(&metric)
//Write a json response
w.WriteHeader(http.StatusCreated)
w.Header().Set("Content-Type", "application/json")
resp := make(map[string]string)
resp["message"] = results
jsonResp, err := json.Marshal(resp)
if err != nil {
log.Fatalf("Error happened in JSON marshal. Err: %s", err)
}
w.Write(jsonResp)
return
}
This results in the error
controllers/statsCont.go:116:18: cannot use results (type *gorm.DB) as type string in assignment
note: module requires Go 1.17
When I try to cast by surrounding result in string() it gives the following error.
controllers/statsCont.go:116:26: cannot convert results (type *gorm.DB) to type string
note: module requires Go 1.17
As stated by #BaytaDarell the query result is added in the variable passed inside Find method
The return value of Find is different in the context it is called in case when it is called with db type the return type is (tx *DB) and when called with associations type the return type is Error
To solve the issue remove below lines
resp := make(map[string]string)
resp["message"] = results
And update it has
resp := map[string]interface{}{"message": metric}
I have something like this and I would like to check if email already exist in DB:
func RegisterUser(c *gin) {
var user models.User
if err := c.ShouldBindJSON(&user); err != nil {
c.JSON(http.StatusBadRequest, gin.H{
"messAge": err.Error(),
"data": "",
})
return
}
// **I TRIED SOEMTHING LIKE THIS**
err := database.DB.Find(&user.Email).Error
if err != nil {
c.JSON(401, gin.H{"MESSAGE": "Email ALREADY exist",
return
}
// **but is not working, because ANY mail it give me error**
if !strings.Contains(user.Email, "#") {
c.JSON(400, gin.H{"MESSAGE": utils.ErrEmailWrong})
return
}
if len(user.Password) < 4 {
c.JSON(400, gin.H{"MESSAGE": utils.ErrPasswordLength})
return
}
database.DB.Create(&user)
c.JSON(200, gin.H{
"MESSAGE": "CREATED",
})
}
With this code, every time is telling me that : Email already exist, only works for the first time.
plase read the document:
https://gorm.io/docs/query.html
var userFind models.User
database.DB.Where("email = ?", user.Email).First(&userFind)
Since, your struct object is not a slice. You should use ErrRecordNotFound.
Note : ErrRecordNotFound only works with First, Last, Take which is expected to return some result. And RecordNotFound is removed in V2.
if err != nil {
if errors.Is(err, gorm.ErrRecordNotFound){
c.JSON(401, gin.H{"MESSAGE": "Email Not Found",
return
}
c.JSON(401, gin.H{"MESSAGE": "Your Message",
return
}
OR
If you want to avoid the ErrRecordNotFound error, you could use Find like db.Limit(1).Find(&user), the Find method accepts both struct and slice data. And check like this :
result.RowsAffected // returns count of records found
For better understanding refer the link here : https://gorm.io/docs/v2_release_note.html#ErrRecordNotFound and https://gorm.io/docs/query.html
And, If you want to add record in DB though email exist then you should remove unique constraint and also check the error while creating the record. If record successfully created then return success response else return the appropriate error message.
you should validate the input after binding and before db queries
alter your email column to be unique
try to insert the validated data to db
if success => 200 (there was no similar email)
if err => check err code
for example:
func IsUniqueContraintViolation(err error) bool {
if pgError, ok := err.(*pgconn.PgError); ok && errors.Is(err, pgError) {
if pgError.Code == "23505" {
return true
}
}
return false
}
For more Information, you should look GoDoc pg lib and Possible Error Codes
and then, then you can return a suitable error code
btw. hopefully you don't save clear passwords to db :D
ITS FINALLY WORKING ! Thanks for everyone thats answered !
I would like to handle a postgresql unique constraint error by accessing the error code. I have gone through these articles here and here and also gone through the documentation here, but still don't quite understand how to implement this and access the errorcode returned by the db. It seems only the Message field method was implemented:
func (err Error) Error() string {
return "pq: " + err.Message
}
If I want to access the SQLSTATE code, Do I implement something like:
func (err *Error) Error() string {
return err.Code
}
and assume the Error struct as defined here will be available.
I have tried something in my handler like this:
sqlInsert := INSERT INTO usrtable (usrCode, teamName, email, phone) VALUES ($1,$2,$3,$4)
_, err := db.Exec(sqlInsert, Data.UsrCode, Data.Teamname, Data.Email, Data.Phone)
if err != nil {
switch err {
case errorCodeNames["23505"]:
// Return web page identifying field and advising user what to do.
return
This returns undefined which makes sense since errorCodeNames is not exported but I am stumped regarding how to achieve this.
You could do something like this if you are using Postgres:
if pgErr, ok := err.(*pq.Error); ok {
if pgErr.Code == "23505" {
//handle duplicate insert
}
}
I have a small Go program which uses a a postgresql db. In it there is a query which can return no rows, and the code I'm using to deal with this isn't working correctly.
// Get the karma value for nick from the database.
func getKarma(nick string, db *sql.DB) string {
var karma int
err := db.QueryRow("SELECT SUM(delta) FROM karma WHERE nick = $1", nick).Scan(&karma)
var karmaStr string
switch {
case err == sql.ErrNoRows:
karmaStr = fmt.Sprintf("%s has no karma.", nick)
case err != nil:
log.Fatal(err)
default:
karmaStr = fmt.Sprintf("Karma for %s is %d.", nick, karma)
}
return karmaStr
}
This logic is taken directly from the Go documentation. When there are no rows corresponding to nick, the following error occurs:
2016/07/24 19:37:07 sql: Scan error on column index 0: converting driver.Value type <nil> ("<nil>") to a int: invalid syntax
I must be doing something stupid - clues appreciated.
I believe your issue is that you're getting a NULL value back from the database, which go translates into nil. However, you're scanning into an integer, which has no concept of nil. One thing you can do is scan into a type that implements the sql.Scanner interface (and can handle NULL values), e.g., sql.NullInt64.
In the example code in the documentation, I'd assume they have a NOT NULL constraint on the username column. I think the reason for this is because they didn't want to lead people to believe that you have to use NULL-able types across the board.
I reworked the code to get the results I wanted.
// Get the karma value for nick from the database.
func getKarma(nick string, db *sql.DB) string {
var karma int
rows, err := db.Query("SELECT SUM(delta) FROM karma WHERE nick = $1", nick)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
karmaStr := fmt.Sprintf("%s has no karma.", nick)
if rows.Next() {
rows.Scan(&karma)
karmaStr = fmt.Sprintf("Karma for %s is %d.", nick, karma)
}
return karmaStr
}
Tempted to submit a documentation patch of some sort to the database/sql package.