I'm learning Go and am trying to create an api endpoint that has an 'fields' parameter. When I try to scan the sqlx resulting rows it into a struct,however the fields omitted by the user are being returned as as an empty string. Is there a way that I can change the struct to reflect only the fields that the user passed? I don't think I want to use omitempty in case for example user_name is an empty string.
type User struct {
Id int `db:"id"`
UserName string `db:"user_name"`
}
func GetUsers(w http.ResponseWriter,r *http.Request,db *sqlx.DB) {
acceptedFields := map[string]bool {
"id":true,
"user_name":true,
}
var requestedFields string = "id"
if r.URL.Query().Get("fields") != ""{
requestedFields = r.URL.Query().Get("fields");
}
for _, requestedField := range strings.Split(requestedFields,",") {
if !acceptedFields[requestedField] {
http.Error(w, fmt.Sprintf("Unknown Field '%s'",requestedField), http.StatusBadRequest)
}
}
users := []User{}
err := db.Select(&users,fmt.Sprintf("SELECT %s FROM users",requestedFields));
if err != nil {
log.Fatal(err)
}
response, _ := json.Marshal(users)
fmt.Fprintf(w,string(response))
}
Resulting Endpoint Output
/users?fields=id => [{"Id":12,"UserName":""}]
Desired Endpoint Output
/users?fields=id => [{"Id":12}]
Also using sql.NullString results in this:
[{"Id":12,"UserName":{"String":"","Valid":false}}]
Thanks to mkorpriva here is a solution
type User struct {
Id int `db:"id"`
UserName *string `db:"user_name" json:",omitempty"`
}
Related
I defined this types in GoLang:
type Comment struct {
Id int
User string
Email string
Date string
Comment string
}
type Post struct {
Id int
Special bool
Title string
Date string
Content string
Image string
Comments []Comment
}
I need to know how to modify this code:
OpenDB()
rows, _ := cn.Query(`SELECT id, date, title, special, content, image
FROM posts ORDER BY date DESC LIMIT $1
OFFSET $2`, fmt.Sprint(limit), fmt.Sprint(offset))
posts := []Post{}
for rows.Next() {
post := Post{}
e := rows.Scan(&post.Id, &post.Date, &post.Title,
&post.Special, &post.Content, &post.Image)
if e != nil {
panic(e)
}
posts = append(posts, post)
}
To allow reading comments. And also, how I can modify:
OpenDB()
_, e = cn.Exec(`INSERT INTO
posts(date, title, special, content, image)
VALUES ($1, $2, $3, $4, $5)`, date, title, special, content, image)
if e != nil {
panic(e)
}
defer CloseDB()
To allow writting an empty array of comments.
Finally I would be grateful if someone tell me how can I write single comments into an existing post.
This depends on your database schema.
If Comment has its own table you will have to loop over all comments in a Post and insert them after you have have inserted the Post. cn.Exec should return a Result which can be used to get the last inserted ID like:
result, err := cn.Exec(...)
if err != nil {
panic(err)
}
id, err := result.LastInsertId()
if err != nil {
panic(err)
}
You can now use the ID of your post as a foreign key.
If you are using a JSON column to store your comments you should define a custom type as alias for []Comment and make the type implement sql.Scanner and driver.Valuer.
type Comment struct {
Id int
User string
Email string
Date string
Comment string
}
type Comments []Comment
// Make the Comments type implement the driver.Valuer interface. This method
// simply returns the JSON-encoded representation of the struct.
func (c Comments) Value() (driver.Value, error) {
return json.Marshal(c)
}
// Make the Comments type implement the sql.Scanner interface. This method
// simply decodes a JSON-encoded value into the struct fields.
func (c *Comments) Scan(value interface{}) error {
var b []byte
switch t := value.(type) {
case []byte:
b = t
case string:
b = string(t)
default:
return errors.New("unknown type")
}
return json.Unmarshal(b, &c)
}
I want to insert to database from CSV file using gorm AutoMigrate and while inserting I want to avoid duplicate entry. How Can I achieve this? Please check the attached code.
type User struct {
gorm.Model
ID int64 `csv:"_" db:"id"`
FirstName string `csv:"First name" db:"first_name"`
LastName string `csv:"Last name" db:"last_name"`
Emails string `csv:"Emails" db:"emails"`
}
func main() {
file, err := os.Open(os.Args[1])
defer file.Close()
users := []User{}
err = gocsv.Unmarshal(file, &users)
db, err := gorm.Open(postgres.Open("host=xxx.xx.x.x user=database password=password dbname=database port=5432 sslmode=disable"))
err = db.AutoMigrate(&User{})
if err != nil {
panic(err)
}
result := db.Create(users)
if result.Error != nil {
panic(result.Error)
}
}
Example: Consider the below data
FIrst name
Last name
Emails
First
Name
first#example.com
Second
Name
second#example.com
Third
Name
Forth
Name
first#example.com
If we pass the above data, the first 3 rows should insert into the database i.e. we have to avoid duplicate email entries to the database. Thanks.
Note: If the email is empty then the row should be inserted into the database.
You have to sanitize "users" after err = gocsv.Unmarshal(file, &users)
Somethink like
func sanytize(arr []User) []User {
users := []User{}
mail := []string{}
for _, a := range arr {
if !contains(mail, a.Emails){
users = append(users, a)
}
mail = append(mail, a.Emails)
}
return users
}
func contains(arr []string, str string) bool {
for _, a := range arr {
if a == str {
return true
}
}
return false
}
....
err = gocsv.Unmarshal(file, &users)
users = sanytize(users)
I try to read and write and delete data from a Go application with the official mongodb driver for go (go.mongodb.org/mongo-driver).
Here is my struct I want to use:
Contact struct {
ID xid.ID `json:"contact_id" bson:"contact_id"`
SurName string `json:"surname" bson:"surname"`
PreName string `json:"prename" bson:"prename"`
}
// xid is https://github.com/rs/xid
I omit code to add to the collection as this is working find.
I can get a list of contacts with a specific contact_id using the following code (abbreviated):
filter := bson.D{}
cursor, err := contactCollection.Find(nil, filter)
for cur.Next(context.TODO()) {
...
}
This works and returns the documents. I thought about doing the same for delete or a matched get:
// delete - abbreviated
filter := bson.M{"contact_id": id}
result, _ := contactCollection.DeleteMany(nil, filter)
// result.DeletedCount is always 0, err is nil
if err != nil {
sendError(c, err) // helper function
return
}
c.JSON(200, gin.H{
"ok": true,
"message": fmt.Sprintf("deleted %d patients", result.DeletedCount),
}) // will be called, it is part of a webservice done with gin
// get complete
func Get(c *gin.Context) {
defer c.Done()
id := c.Param("id")
filter := bson.M{"contact_id": id}
cur, err := contactCollection.Find(nil, filter)
if err != nil {
sendError(c, err) // helper function
return
} // no error
contacts := make([]types.Contact, 0)
for cur.Next(context.TODO()) { // nothing returned
// create a value into which the single document can be decoded
var elem types.Contact
err := cur.Decode(&elem)
if err != nil {
sendError(c, err) // helper function
return
}
contacts = append(contacts, elem)
}
c.JSON(200, contacts)
}
Why does the same filter does not work on delete?
Edit: Insert code looks like this:
_, _ = contactCollection.InsertOne(context.TODO(), Contact{
ID: "abcdefg",
SurName: "Demo",
PreName: "on stackoverflow",
})
Contact.ID is of type xid.ID, which is a byte array:
type ID [rawLen]byte
So the insert code you provided where you use a string literal to specify the value for the ID field would be a compile-time error:
_, _ = contactCollection.InsertOne(context.TODO(), Contact{
ID: "abcdefg",
SurName: "Demo",
PreName: "on stackoverflow",
})
Later in your comments you clarified that the above insert code was just an example, and not how you actually do it. In your real code you unmarshal the contact (or its ID field) from a request.
xid.ID has its own unmarshaling logic, which might interpret the input data differently, and might result in an ID representing a different string value than your input. ID.UnmarshalJSON() defines how the string ID will be converted to xid.ID:
func (id *ID) UnmarshalJSON(b []byte) error {
s := string(b)
if s == "null" {
*id = nilID
return nil
}
return id.UnmarshalText(b[1 : len(b)-1])
}
As you can see, the first byte is cut off, and ID.UnmarshalText() does even more "magic" on it (check the source if you're interested).
All-in-all, to avoid such "transformations" happen in the background without your knowledge, use a simple string type for your ID, and do necessary conversions yourself wherever you need to store / transmit your ID.
For the ID Field, you should use the primitive.ObjectID provided by the bson package.
"go.mongodb.org/mongo-driver/bson/primitive"
ID primitive.ObjectID `json:"_id" bson:"_id"`
I'm retrieving the data from the database when the user hit the url like http://localhost:8080/api/v1/customer?keyword=dhiman then it search for the data in the collection if there is any field matches then it will retrieve that data. if the user entered the short url like http://localhost:8080/api/v1/customer?keyword=dhi then it also retrieve the data which matches like that how I'll solve this problem. I tried the code for this like following:-
Struct of the customer
type Customer struct {
Id int `json:"id" bson:"_id"`
FirstName string `json:"first_name" bson:"first_name"`
LastName string `json:"last_name" bson:"last_name"`
Email string `json:"email" bson:"email"`
PhoneNumber string `json:"phone_number" bson:"phone_number"`
}
type Customers []Customer
Functions
func GetCustomers(c *gin.Context) {
value := c.Query("keyword")
fmt.Println(value)
response := ResponseControllerList{}
conditions := bson.M{"last_name":value}
data, err := models.GetCustomerListing(conditions)
if err != nil {
response = ResponseControllerList{
config.FailureCode,
config.FailureFlag,
config.FailureMsg,
nil,
nil,
}
} else {
response = ResponseControllerList{
config.SuccessFlag,
config.SuccessFlag,
config.SuccessMsg,
data,
// dataCount,
nil,
}
}
GetResponseList(c, response)
}
GetCustomerListing function in models page:-
func GetCustomerListing(customerQuery interface{}) (result Customers, err error) {
mongoSession := config.ConnectDb()
sessionCopy := mongoSession.Copy()
defer sessionCopy.Close()
getCollection := mongoSession.DB(config.Database).C(config.CustomerCollection)
err = getCollection.Find(customerQuery).Select(bson.M{"password": 0}).All(&result) //.Skip(skip).Limit(limit)
if err != nil {
return result, err
}
return result, nil
}
collection images
I got the answer it is done by using the $or in mongodb.
In the monogdb there is a operator called or $or it checks the value with all the fields and produce result.
There is a bson.RegExis used. Because it will matches or checks the data similar to it receives from the user.
There is change in condition. The condition is:-
conditions := bson.M{"$or": []bson.M{
bson.M{"first_name": bson.RegEx{value,""}},
bson.M{"last_name": bson.RegEx{value,""}},
bson.M{"email": bson.RegEx{value,""}},
bson.M{"phone_number": bson.RegEx{value,""}},
}}
there is change in the query
Now I'm doing:
sess := mongodb.DB("mybase").C("mycollection")
var users []struct {
Username string `bson:"username"`
}
err = sess.Find(nil).Select(bson.M{"username": 1, "_id": 0}).All(&users)
if err != nil {
fmt.Println(err)
}
var myUsers []string
for _, user := range users{
myUsers = append(myUsers, user.Username)
}
Is there a more effective way to get slice with usernames from Find (or another search function) directly, without struct and range loop?
The result of a MongoDB find() is always a list of documents. So if you want a list of values, you have to convert it manually just as you did.
Using a custom type (derived from string)
Also note that if you would create your own type (derived from string), you could override its unmarshaling logic, and "extract" just the username from the document.
This is how it could look like:
type Username string
func (u *Username) SetBSON(raw bson.Raw) (err error) {
doc := bson.M{}
if err = raw.Unmarshal(&doc); err != nil {
return
}
*u = Username(doc["username"].(string))
return
}
And then querying the usernames into a slice:
c := mongodb.DB("mybase").C("mycollection") // Obtain collection
var uns []Username
err = c.Find(nil).Select(bson.M{"username": 1, "_id": 0}).All(&uns)
if err != nil {
fmt.Println(err)
}
fmt.Println(uns)
Note that []Username is not the same as []string, so this may or may not be sufficient to you. Should you need a user name as a value of string instead of Username when processing the result, you can simply convert a Username to string.
Using Query.Iter()
Another way to avoid the slice copying would be to call Query.Iter(), iterate over the results and extract and store the username manually, similarly how the above custom unmarshaling logic does.
This is how it could look like:
var uns []string
it := c.Find(nil).Select(bson.M{"username": 1, "_id": 0}).Iter()
defer it.Close()
for doc := (bson.M{}); it.Next(&doc); {
uns = append(uns, doc["username"].(string))
}
if err := it.Err(); err != nil {
fmt.Println(err)
}
fmt.Println(uns)
I don't see what could be more effective than a simple range loop with appends. Without all the Mongo stuff your code basically is this and that's exactly how I would do this.
package main
import (
"fmt"
)
type User struct {
Username string
}
func main() {
var users []User
users = append(users, User{"John"}, User{"Jane"}, User{"Jim"}, User{"Jean"})
fmt.Println(users)
// Interesting part starts here.
var myUsers []string
for _, user := range users {
myUsers = append(myUsers, user.Username)
}
// Interesting part ends here.
fmt.Println(myUsers)
}
https://play.golang.com/p/qCwENmemn-R