Go: Unable to unmarshal json value - postgresql

Question

I have jsonb value stored in postgres columns which I need to convert to object to send the response
I am unable to convert JSON -> struct using json.Unmarshal method
type Result struct {
Id int
Expertise string // because json string
Languages string // because json string
}
// saving dd query data to result struct
e, _ := json.Marshal(data.Expertise)
l, _ := json.Marshal(data.Languages)
row := r.db.QueryRow(`
INSERT INTO filters (user_id, expertise, languages)
VALUES ($1, $2, $3)
ON CONFLICT (user_id) DO
UPDATE SET expertise=$2, languages=$3
RETURNING id, expertise, languages;
`, userid, e, l)
var res Result
err := row.Scan(&res.Id, &res.Expertise, &res.Languages)
Then I take this the Expertise and Languages field and UNMARSHAL THEM
// helper method
func Unmarshal(value string) interface{} {
var obj interface{}
err := json.Unmarshal([]byte(value), &obj)
return obj
}
type CreateFilterResponse struct {
Id int `json:"id"`
Expertise interface{} `json:"expertise"`
Languages interface{} `json:"languages"`
}
response := dto.CreateFilterResponse{
Id: res.Id,
Expertise: Unmarshal(res.Expertise), // 👈 not decoding, still json
Languages: Unmarshal(res.Languages), // 👈 not decoding, still json
}
I would really appreciate the help, I need the Expertise and Languages to be {} and [] respectively
This is what I am getting in response:
{"id":5,"expertise":"[{\"role\":1,\"experience\":5},
{\"role\":3,\"experience\":4}]","languages":"[1,2,3]"}
Note:
Expertise is jsonb column : []{role: int, experience: int}
Languages is jsonb column : []int

Answer 1

Thanks to #Blackgreen comment, the suggestion worked.
As he suggested "I believe the issue is that you are scanning the jsonb bytes into string, which causes it to be interpreted literally. Then when you pass a string into json.Unmarshal it stays a string. Change the type of Expertise and Language to []byte (or json.RawMessage)"
I did the same and it works, here is the code:
// for parsing jsonb coming from db query
type Result struct {
Id int
Expertise json.RawMessage // change
Languages json.RawMessage // change
}
// the database query itself
e, _ := json.Marshal(filters.Expertise)
l, _ := json.Marshal(filters.Languages)
row := r.db.QueryRow(`
INSERT INTO filters (user_id, expertise, languages)
VALUES ($1, $2, $3)
ON CONFLICT (user_id) DO
UPDATE SET expertise=$2, languages=$3
RETURNING id, expertise, languages;
`, userid, e, l)
var res Result
err := row.Scan(&res.Id, &res.Expertise, &res.Languages) // scanning to result
if err != nil {
fmt.Println("Unable to create a new filter ==>", err)
return nil, err
}
Then I Unmarshalled the expertise and languages jsonb values using a helper method and created a Response struct for the client:
// response struct type
type CreateFilterResponse struct {
Id int `json:"id"`
Expertise interface{} `json:"expertise"`
Languages interface{} `json:"languages"`
}
// final response struct
response := dto.CreateFilterResponse{
Id: res.Id,
Expertise: Unmarshal(res.Expertise),
Languages: Unmarshal(res.Languages),
}
// helper method
func Unmarshal(value []byte) interface{} {
var obj interface{}
json.Unmarshal(value, &obj)
return obj
}
This works for now. I still need to find an easy way to do this as this is too much boilerplate code to perform a simple task. like in JS/TS it can be done using single line of code JSON.parse(value)

Answer 2

The problem is in the json output. It is encoding your JSON twice, making it difficult to read and deserialize the json in your structure.
You can start by organizing your structures as follows:
type Expertise struct {
Role int `json:"role"`
Experience int `json:"Experience"`
}
type CreateFilterResponse struct {
Id int `json:"id"`
Expert []Expertise `json:"expertise"`
Languages []int `json:"languages"`
}
I made a cleaner method just to exemplify and facilitate understanding. It will remove the unnecessary quotes and escapes so you can convert your string to []bytes and unmarshal your structure.
func jsonCleaner(quoted string) []byte{
quoted = strings.ReplaceAll(quoted, "\n", "")
quoted = strings.ReplaceAll(quoted, "\\", "")
quoted = strings.ReplaceAll(quoted, "]\"", "]")
quoted = strings.ReplaceAll(quoted, "\"[", "[")
dataBytes := []byte(quoted)
fmt.Println(dataBytes)
return dataBytes
}
Your json message would go from:
{"id":5,"expertise":"[{\"role\":1,\"experience\":5},{\"role\":3,\"experience\":4}]","languages":"[1,2,3]"}
To:
{"id":5,"expertise":[{"role":1,"experience":5},"role":3,"experience":4}],"languages":[1,2,3]}