Convert JSON.RawMessage to JSON - postgresql

I am using gqlgen, sqlx and pgx. And I'm trying to use a custom scalar for sqlx's types.JSONText.
I have this attributes jsonb field in items table.
-- migrations/001_up.sql
CREATE TABLE IF NOT EXISTS items (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
quantity INT NOT NULL,
attributes JSONB
);
I have these model structs:
// graph/model/item.go
type Item struct {
ID string `json:"id,omitempty" db:"id,omitempty"`
Quantity int `json:"quantity" db:"quantity"`
Attributes *Attributes `json:"attributes,omitempty" db:"attributes,omitempty"`
}
type Attributes types.JSONText
I have this graphql schema:
// graph/schema.graphql
type Item {
id: ID!
quantity: Int!
attributes: Attributes
}
scalar Attributes
I can successfully inserted into database, but got error at retrieving.
| id | quantity | attributes |
|---------------|----------|------------------------------------|
| 031e1489-... | 100 | {"size": "medium", "color": "red"} |
This is the log I got from db query:
>> items.Db: &{
031e1489-02c9-46d3-924d-6a2edf1ca3ba // id
100 // quantity
0xc000430600 // attributes
}
I tried to marshal attributes scalar:
// graph/model/item.go
...
func (a *Attributes) MarshalGQL(w io.Writer) {
b, _ := json.Marshal(a)
w.Write(b)
}
// Unmarshal here
...
Add custom scalar type in gqlgen.yml:
...
Attributes:
model:
- github.com/my-api/graph/model.Attributes
But I got the string instead of json:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": "eyJjb2xvciI6ICJyZWQifQ==",
}
}
}
The desired output is:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": {
"size": "medium",
"color": "red",
}
}
}
}
What I am doing wrong?
Here are my attempts:
If I removed pointer from Attributes in Item struct, gqlgen throws error:
go generate ./...
go: finding module for package github.com/my-api/graph/generated
generating core failed: type.gotpl: template: type.gotpl:49:28: executing "type.gotpl" at <$type.Elem.GO>: nil pointer evaluating *config.TypeReference.GOexit status 1
graph/resolver.go:3: running "go": exit status 1
make: *** [Makefile:2: gengql] Error 1
This returns the desired result, but I don't know how to use it with real data:
func (a *Attributes) MarshalGQL(w io.Writer) {
raw := json.RawMessage(`{"foo":"bar"}`)
j, _ := json.Marshal(&raw)
s := string(j)
w.Write([]byte(s))
}
Query result:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": {
"foo": "bar"
}
}
}
}

Attributes is defined using types.JSONText which is defined using json.RawMessage which is defined using []byte. This means that the underlying type of all 4 types, including []byte, is []byte, which in turn means that all of the 4 types can be converted to []byte.
Hence it should be enough to do this:
func (a *Attributes) MarshalGQL(w io.Writer) {
w.Write([]byte(*a))
}

Related

`find_one` does not give ObjectId in proper format

As the title states, here is the following code.
let users_coll = db
.database("foo")
.collection::<bson::oid::ObjectId>("users");
let user_id = users_coll
.find_one(
doc! { "email": &account.email },
mongodb::options::FindOneOptions::builder()
.projection(doc! { "_id": 1i32 })
.build(),
)
.await?
.unwrap();
But it fails at ? operator with the following mongodb::error::Error,
Error { kind: BsonDeserialization(DeserializationError { message: "expected map containing extended-JSON formatted ObjectId, instead found { \"_id\": ObjectId(\"62af199df4a16d3ea6056536\") }" }), labels: {}, wire_version: None, source: None }
And it is right. Given ObjectId should be in this format,
{
"_id": {
"$oid": "62af199df4a16d3ea6056536"
}
}
But I do not know how to handle this. Any help is appreciated.
Have a good day!
Your users collection isn't a collection of ObjectIds, it's actually a collection of documents which each contain an ObjectId. To let Rust know what to do with those, you should create a struct which represents the document, or at least the parts which you care about getting back from your query, and tell your collection to de-serialize into that struct:
use mongodb::bson::oid::ObjectId;
use serde::{Serialize, Deserialize};
#[derive(Debug, Default, Serialize, Deserialize)]
struct User {
_id: ObjectId,
}
#[tokio::main]
async fn main() {
let users_coll = db
.database("foo")
.collection::<User>("users");
let user_id: ObjectId = users_coll
.find_one(
doc! { "email": &account.email },
mongodb::options::FindOneOptions::builder()
.projection(doc! { "_id": 1i32 })
.build(),
)
.await?
.unwrap()
._id;
}
By default, the BSON fields have to match the struct fields exactly (_id in this case), but I'm pretty sure serde has a way to change that if you don't like the leading underscore.

Access Data Object by ID

I have the following structs that defined according to the SwiftUI example in Apple.
struct Cat: Hashable, Codable, Identifiable {
let id: Int
let name: String
let audio: [Int]? // list of audio of cat sound
}
struct CatAudio: Hashable, Codable, Identifiable {
let id: Int
let filename: String
}
I then would like to access the audio and then deliver in the view.
I have json data like this:
[
{
"id": 55,
"name": "meow",
"audio": [6,5]
},
{
"id": 3,
"name": "meowmeow",
"audio": [2]
}
]
AudioData.json
[
{
"id": 5,
"filename": "5.wav"
},
{
"id": 2,
"filename": "2.wav"
},
{
"id": 6,
"filename": "6.wav"
}
]
The json files loaded successfully.
#Published var cats: [Cat] = load("CatData.json")
#Published var catAudios: [CatAudio] = load("CatAudio.json")
I then tried to get an audio object from my environment model data:
#EnvironmentObject var modelData: ModelData
and then I want to get an the corresponding audio object of the cat. but I failed to do so as I do not know how to use the "Id" to get it.
Example:
Assume that I got the cat object from my model:
let cat = modelData.cats[0]
I then want to get its audio data according to the id stored in the audio list of it
let catAudio = modelData.catAudios[cat.audio[0]!] // exception here
I found that it is because the array order may not be consistent with the "Id". I want to make use of the "Id" instead of the Array order to get the item.
How can I do it?
========
I have tried to write a function to get the list of CatAudio of a cat.
I have also make audio non-optional.
func getAudio(cat: Cat) -> [CatAudio] {
return cat.audio.compactMap{id in catAudio.filter { $0.id==id } }
}
But it complains and said that cannot convert the value of type [CatAudio] to closure result type 'CatAudio'
I got confused with that.
You need to use high-order functions to match the id values in the audio array with the elements in the CatAudio array
Assuming the 2 arrays and a selected Cat object
var cats: [Cat] = ...
var catAudios: [CatAudio] = ...
let cat = cats[0]
To select one audio for a one id
if let firstId = cat.audio?.first {
let audio = catAudios.first { $0.id == firstId}
}
To get an array of all CatAudio for the cat object
if let array = cat.audio {
let values = array.compactMap { id in catAudios.filter { $0.id == id }}
}
The code would be simpler if the audio array wasn't optional, any reason for it to be declared optional?

Using MongoDB Projection

I have the following structure in my database:
{
"_id": {
"$oid": "5fc4fc68fcd604bac9f61f71"
},
"init": {
"fullname": "Besikta Anibula",
"parts": [
"Besikta",
"Anibula"
],
"alt": "Besikta Ani."
},
"industry": "E-Commerce"
}
I´m trying to just access the init object and write the results to a structured variable in Go:
var InputData struct {
Fullname string `bson:"fullname" json:"fullname"`
Parts []string`bson:"parts" json:"parts"`
Alt string `bson:"alt" json:"alt"`
}
collectionRESULTS.FindOne(ctx, options.FindOne().SetProjection(bson.M{"init": 1})).Decode(&InputData)
js, _ := json.Marshal(InputData)
fmt.Fprint(writer, string(js))
But the result is empty:
{"fullname":"","parts":null,"alt":""}
It is working when not using a projection like:
var InputData struct {
Ident primitive.ObjectID `bson:"_id" json:"id"`
}
collectionRESULTS.FindOne(ctx, bson.M{}).Decode(&InputData)
js, _ := json.Marshal(InputData)
fmt.Fprint(writer, string(js))
Result as expected:
{"id":"5fc4fc68fcd604bac9f61f71"}
You set the projection to only retrieve the init property of the result documents, and then you try to unmarshal into a struct value that does not have any matching field for the init value, so nothing will be set in that struct value.
You must use a value that has an init field, like this wrapper Result struct:
type Result struct {
Init InputData `bson:"init"`
}
type InputData struct {
Fullname string `bson:"fullname" json:"fullname"`
Parts []string `bson:"parts" json:"parts"`
Alt string `bson:"alt" json:"alt"`
}
Use it like this:
var result Result
err := collectionRESULTS.FindOne(ctx, bson.M{}, options.FindOne().
SetProjection(bson.M{"init": 1})).Decode(&result)
if err != nil {
// handle error
}

Get names of all keys in the collection using Go

I'd like to get the names of all the keys in a MongoDB collection.
For example, from this:
"Id": ObjectId("5f5a010d431c4519dcda0e3d")
"title": "App"
"query": ""
"db": ""
"widgettype": ""
"tablename": "active_instance"
fields:Object
user:"name",
key:"passcode"
"status": "active"
"inlibrary": ""
"createdts": 1599733804
Using "gopkg.in/mgo.v2" and "gopkg.in/mgo.v2/bson" packages.
err := mongodbSession.DB(dbName).C(collectionName).Find(bson.M{}).One(&result)
var keyset []string
for index, _ := range result {
fmt.Printf("%+v\n", index)
keyset = append(keyset, index)
}
fmt.Println(keyset)
getting output as this
[_id title query db widgettype status fields inlibrary createdts ]
child key is not being featched that is user and key.
Embedded documents will appear as another bson.M values inside your result, so you have to use a recursion to also traverse those.
Here's how you can do that:
func getKeys(m bson.M) (keys []string) {
for k, v := range m {
keys = append(keys, k)
if m2, ok := v.(bson.M); ok {
keys = append(keys, getKeys(m2)...)
}
}
return
}
Example using it:
m := bson.M{"Id": bson.ObjectId("5f5a010d431c4519dcda0e3d"),
"title": "App",
"query": "",
"db": "",
"widgettype": "",
"tablename": "active_instance",
"fields": bson.M{
"user": "name",
"key": "passcode",
},
"status": "active",
"inlibrary": "",
"createdts": 1599733804,
}
keys := getKeys(m)
fmt.Println(keys)
Which will output (try it on the Go Playground):
[db widgettype createdts inlibrary _id title query tablename
fields user key status]
If you look at the results, user and key are included, but it's not possible to tell if they were fields of the document or that of an embedded document.
You may choose to prefix fields of embedded documents with the field name of the embedded document field itself, e.g. to get fields.user and fields.key.
This is how you can do that:
func getKeys(m bson.M) (keys []string) {
for k, v := range m {
keys = append(keys, k)
if m2, ok := v.(bson.M); ok {
for _, k2 := range getKeys(m2) {
keys = append(keys, k+"."+k2)
}
}
}
return
}
Which would output (try it on the Go Playground):
[createdts title query db status inlibrary _id widgettype tablename
fields fields.user fields.key]
Also note that the above solutions do not handle arrays. If you have arrays, you should also iterate over them, and if they contain another array or object, you should do the same (recursively). It's an exercise for you to extend it to handle arrays too.

postgres, go-gorm - can't preload data

I am trying to preload some data when querying for a row on my postgresql with gorm
I have the following types defined in go with a belongs to association:
type MyObject struct {
ID uint `grom:"column:id" json:"id"`
Name string `gorm:"column:name" json:"name"`
OwnerID uint `gorm:"column:owner_id" json:"owner_id"`
Owner Owner `json:"owner"`
...
}
type Owner struct {
ID uint `gorm:"column:id" json:"id"`
Name string `gorm:"column:name" json:"name"`
Config json.RawMessage `gorm:"column:config" sql:"type:json" json:"config"`
Components []uint8 `gorm:"column:components;type:integer[]" json:"components"`
}
and the tables in the postgres db with a row as follows
my_schema.my_object
id | name | owner_id | ...
----|-----------|-----------|-----
0 | testobj | 0 | ...
my_schema.owner
id | name | config | components
----|-----------|-----------|-----------
0 | testowner | <jsonb> | {0,1,2,3}
I am running the following query with gorm:
object := &models.MyObject{}
result := ls.Table("my_schema.my_object").
Preload("Owner").
Where("id = ?", "0").
First(object)
but as a result, in object i get the following struct:
{"id": 0, "name": "testobj", "owner_id":0, "owner": {"id": 0, "name": "", "config": nil, "components": nil}}
I get no errors or warnings of any kind
the foreignkey relation was specified in the database creation sql script
I was able to achieve what I wanted by implementing the TableName() method for each of my models as such:
type MyObject struct {
ID uint `grom:"column:id" json:"id"`
OwnerID uint `gorm:"column:owner_id" json:"owner_id"`
Owner Owner `json:"owner"`
...
}
func (MyObject) TableName() {
return "my_schema.my_object"
}
type Owner struct {
ID uint `gorm:"column:id" json:"id"`
...
}
func (Owner) TableName() {
return "my_schema.owner"
}
Then, I can then use it in a query like the following:
myObject := &models.MyObject{}
result := ls.Model(myObject).
Where("id = ?", id). //assume any valid value for id
First(myObject).
Related(&myObject.Owner)
return myObject, processResult(result)
in the end, when making a request for MyObject to my server I get all the data of the related Owner object in the database:
$ curl "http://localhost:8080/api/objetcs/related/1" | jq // passing 1 as object id value in url params
{
"id": "1", // taken from UrlParam
"name": "testobj",
"owner": {
"id": "1", // using foreign key owner_id
"name": "testowner",
"config": configJson // json object taken from row in my_schema.owner table in the DB
}
}
}