I'm trying to map data from DB ( Mongo ) to slice in go , and everythin works fine if I'm returning simple []string but if I change type to []*models.Organization that code returns slice of same elements.
func (os *OrganizationService) GetAll() ([]*models.Organization, error) {
var organizations []*models.Organization
results := os.MongoClient.Collection("organizations").Find(bson.M{})
organization := &models.Organization{}
for results.Next(organization) {
fmt.Println(organization)
organizations = append(organizations, organization)
}
return organizations, nil
}
I expect output [{ Name: "someOrg", ID: "someId" },{ Name: "someOrg2", ID: "someID }, ... ] , but actual output is [{ Name: "someOrg", ID: "someId" },{ Name: "someOrg", ID: "someId" }, ... ]
I'm using bongo package.
The application appends the same value of organization on every iteration through the loop. Fix by creating a new value inside the loop.
func (os *OrganizationService) GetAll() ([]*models.Organization, error) {
var organizations []*models.Organization
results := os.MongoClient.Collection("organizations").Find(bson.M{})
organization := &models.Organization{}
for results.Next(organization) {
fmt.Println(organization)
organizations = append(organizations, organization)
organization = &models.Organization{} // new value for next iteration
}
return organizations, nil
}
Related
I get the following error: runtime error: invalid memory address or nil pointer dereference goroutine 23 [running].
If I remove config.NewConfig().Repo.InsertLicense(license) in this function I don't get an error. That means this line causes the error in this function
func generateAvailableLicenses() {
license := entity.License{
Key: generateRandomLicenseKey(),
IsUsed: entity.AVAILABLE,
CreatedAt: time.Now().Format("2006-01-02"),
ExpiresAt: time.Now().AddDate(0, 0, 1).Format("2006-01-02"),
CompanyId: "1",
CompanyName: "Company 1",
TeamId: "1",
TeamName: "Team 1",
}
availableLicenses = append(availableLicenses, license)
config.NewConfig().Repo.InsertLicense(license) // error line
}
That's everything for my config:
type Config struct {
WebPort string
Repo repository.Repository
}
func NewConfig() *Config {
return &Config{
WebPort: ":8080",
}
}
func (app *Config) SetupPostgresRepository(conn *gorm.DB) {
db := postgres.NewPostgresRepository(conn)
app.Repo = db
}
Repository
type PostgresRepository struct {
Conn *gorm.DB
}
var db *gorm.DB
func NewPostgresRepository(pool *gorm.DB) *PostgresRepository {
db = pool
return &PostgresRepository{
Conn: pool,
}
}
func (u *PostgresRepository) InsertLicense(license entity.License) error {
result := db.Create(&license)
if result.Error != nil {
return result.Error
}
return nil
}
Tech: Go, gorm, Postgresql, docker
This could potentially trying to access a value which is still not yet initialised.
Makes sure to initialise and fill all repository values before using it.
you can use func init() if application architecture allows.
Mongo 4.4 and respective Golang driver are used. Database’s replica set is being run locally at localhost:27017, localhost:27020. I’ve also tried using Atlas’s sandbox cluster which gave me the same results.
According to Mongo's documentation when handling insertion of a new document fullDocument field of event data is supposed to contain newly inserted document which for some reason is not the case for me. ns field where database and collection name are supposed to be and documentKey where affected document _id is stored are empty as well. operationType field contains correct operation type. In another test it appeared that update operations do not appear in a change stream at all.
It used to work as it should but now it doesn't. Why does it happen and what am I doing wrong?
Code
// ds is the connection to discord, required for doing stuff inside handlers
func iterateChangeStream(stream *mongo.ChangeStream, ds *discordgo.Session, ctx context.Context, cancel context.CancelFunc) {
defer stream.Close(ctx)
defer cancel() // for graceful crashing
for stream.Next(ctx) {
var event bson.M
err := stream.Decode(&event)
if err != nil {
log.Print(errors.Errorf("Failed to decode event: %w\n", err))
return
}
rv := reflect.ValueOf(event["operationType"]) // getting operation type
opType, ok := rv.Interface().(string)
if !ok {
log.Print("String expected in operationType\n")
return
}
// event["fullDocument"] will be empty even when handling insertion
// models.Player is a struct representing a document of the collection
// I'm watching over
doc, ok := event["fullDocument"].(models.Player)
if !ok {
log.Print("Failed to convert document into Player type")
return
}
handlerCtx := context.WithValue(ctx, "doc", doc)
// handlerToEvent maps operationType to respective handler
go handlerToEvent[opType](ds, handlerCtx, cancel)
}
}
func WatchEvents(ds *discordgo.Session, ctx context.Context, cancel context.CancelFunc) {
pipeline := mongo.Pipeline{
bson.D{{
"$match",
bson.D{{
"$or", bson.A{
bson.D{{"operationType", "insert"}}, // !!!
bson.D{{"operationType", "delete"}},
bson.D{{"operationType", "invalidate"}},
},
}},
}},
}
// mongo instance is initialized on program startup and stored in a global variable
opts := options.ChangeStream().SetFullDocument(options.UpdateLookup)
stream, err := db.Instance.Collection.Watch(ctx, pipeline, opts)
if err != nil {
log.Panic(err)
}
defer stream.Close(ctx)
iterateChangeStream(stream, ds, ctx, cancel)
}
My issue might be related to this, except that it consistently occurs on insertion instead ocuring sometimes on updates.
If you know how to enable change stream optimization feature flag mentioned inside link above, let me know.
Feel free to ask for more clarifications.
The question was answered here.
TLDR
You need to create the following structure to unmarshal event into:
type CSEvent struct {
OperationType string `bson:"operationType"`
FullDocument models.Player `bson:"fullDocument"`
}
var event CSEvent
err := stream.Decode(&event)
event will contain a copy of the inserted document.
From sample events that I see from this link we can see that fullDocument exists only on operationType: 'insert'.
{
_id: { _data: '825DE67A42000000072B022C0100296E5A10046BBC1C6A9CBB4B6E9CA9447925E693EF46645F696400645DE67A42113EA7DE6472E7680004' },
operationType: 'insert',
clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 7, high_: 1575385666 },
fullDocument: {
_id: 5de67a42113ea7de6472e768,
name: 'Sydney Harbour Home',
bedrooms: 4,
bathrooms: 2.5,
address: { market: 'Sydney', country: 'Australia' } },
ns: { db: 'sample_airbnb', coll: 'listingsAndReviews' },
documentKey: { _id: 5de67a42113ea7de6472e768 }
}
{
_id: { _data: '825DE67A42000000082B022C0100296E5A10046BBC1C6A9CBB4B6E9CA9447925E693EF46645F696400645DE67A42113EA7DE6472E7680004' },
operationType: 'delete',
clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 8, high_: 1575385666 },
ns: { db: 'sample_airbnb', coll: 'listingsAndReviews' },
documentKey: { _id: 5de67a42113ea7de6472e768 }
}
So I recommend You
to limit Your $match to insert
or add if statement to operationType.
if opType == "insert" {
doc, ok := event["fullDocument"].(models.Player)
if !ok {
log.Print("Failed to convert document into Player type")
return
}
handlerCtx := context.WithValue(ctx, "doc", doc)
// handlerToEvent maps operationType to respective handler
go handlerToEvent[opType](ds, handlerCtx, cancel)
return
}
or make sure You're getting document using id of document from event["documentKey"]["_id"] and call playersCollection.findOne({_id: event["documentKey"]["_id"]})
I am trying to update my addresses into my person data:
if _, ok := update["addressName"]; ok {
request = bson.M{"addresses": bson.M{"addressName": update["addressName"]}}
} else {
request = update
}
_, err = people.UpdateOne(context.TODO(), filter, bson.M{"$set": request})
this doesn't create an object in the array,
I want to have a result like this:
{
"updateAt": TIME_NOW
"addresses": [
{"addressName": "ONLY", default: true},
{"addressName": "ONLY", default: true}
]
}
how is the correct way to request for an object in array with MongoDB Driver?
You are $setting the addresses to an array containing a single element. Either you have to set the addresses to an array containing all the elements you need, or you have to add to that array using $push:
_, err = people.UpdateOne(context.TODO(), filter, bson.M{"$push":bson.M{"addresses":bson.M{ address info }})
I'm having trouble finding objects in a nested array. I need to find home/away within league array which has an events array.
Example JSON:
{
"sportId":4,
"last":266178326,
"league":[
{
"id":423,
"name":"Germany - Bundesliga",
"events":[
{
"id":1125584543,
"starts":"2020-06-07T17:00:00Z",
"home":"SC Rasta Vechta",
"away":"EnBW Ludwigsburg",
"rotNum":"2601",
"liveStatus":0,
"status":"I",
"parlayRestriction":0,
"altTeaser":false,
"resultingUnit":"Regular"
},
{
"id":1125585441,
"starts":"2020-06-10T18:30:00Z",
"home":"Ratiopharm Ulm",
"away":"Crailsheim Merlins",
"rotNum":"2617",
"liveStatus":0,
"status":"I",
"parlayRestriction":0,
"altTeaser":false,
"resultingUnit":"Regular"
}
]
},
{
"id":268,
"name":"ABA - Adriatic League",
"events":[
{
"id":1122419811,
"starts":"2020-05-07T19:34:00Z",
"home":"Test 1(Do Not Wager)",
"away":"Test 2(Do Not Wager)",
"rotNum":"999998",
"liveStatus":0,
"status":"I",
"parlayRestriction":1,
"altTeaser":false,
"resultingUnit":"Regular"
}
]
},
{
"id":487,
"name":"NBA",
"events":[
{
"id":1120192519,
"starts":"2020-05-01T17:00:00Z",
"home":"Test Team B",
"away":"Test Team A",
"rotNum":"123",
"liveStatus":0,
"status":"O",
"parlayRestriction":0,
"altTeaser":false,
"resultingUnit":"Regular"
}
]
}
]
}
For example finding the league name "Germany - Bundesliga" I solved it by doing
// retrieve league by searching in the fixture collection
func FindLeagueFixture(name string) (pinnacle.Fixtures, pinnacle.League, error) {
var fixtures []pinnacle.Fixtures
err := db.C(FIXTURES).Find(
bson.M{"league.name": bson.RegEx{
Pattern: name,
Options: "i",
}}).All(&fixtures)
if err != nil {
return pinnacle.Fixtures{}, pinnacle.League{}, err
}
But now I have to event home/away names within league events. For example, finding "SC Rasta Vechta". What's the best way to handle this?
I've tried something like (No regex usage yet, since I'm having trouble already. Only trying count, not doing the whole unmarshaling for now)
// retrieve sport team by searching in the fixture collection
func FindHomeOrAwayFixture(name string) (pinnacle.Fixtures, pinnacle.League, error) {
var fixtures []pinnacle.Fixtures
// find home
c, err := db.C(FIXTURES).Find(
bson.M{"league": bson.M{"$elemMatch": bson.M{"home": name}}}).Count()
if err != nil {
return pinnacle.Fixtures{}, pinnacle.League{}, err
}
fmt.Println(c)
}
I’m writing a code for receiving a data from mongodb in golang.
My code is as below:
type DataContent struct {
Create time.Time `bson:"create"`
Desc string `bson:"desc"`
}
type Data struct {
Id bson.ObjectId `bson:"_id,omitempty"`
Desc string `bson:"desc"`
Content DataContent `bson:"content"`
}
func get() error {
result := []Data{}
coll := session.DB(“”).C(“aaa”)
project := bson.M{"$project": bson.M{"_id": 1, "desc": 1, "content": 1 }}
err := coll.Pipe([]bson.M{project}).All(&result)
if err != nil {
return err
}
data, err := json.Marshal(result)
fmt.Printf("#####\n%s\n#####\n", string(data))
return nil
}
Result of execution is as below:
#####
[{"Id":"58133f92cf4abf18c834750d", "Desc”:”reg1\n”,"Content":{"Create":"0001-01-01T00:00:00Z","Desc":""}},
{"Id":"58134bbbcf4abf18c8347513", "Desc”:”reg2\n”,”Content":{"Create":"0001-01-01T00:00:00Z","Desc”:””}}]
#####
Values for subfield of “Content” was not come.
I’ve run same process via terminal also. Result is as below:
> db.aaa.aggregate([{$project: { _id:1, desc:1, content:1}}])
[{"Id":"58133f92cf4abf18c834750d", "Desc”:”reg1\n”,"Content":{"Create":ISODate("2016-10-28T13:13:13.520Z"),"Desc”:”aaa”}},
{"Id":"58134bbbcf4abf18c8347513", "Desc”:”reg2\n”,”Content":{"Create":ISODate("2016-10-28T13:09:32.810Z"),"Desc”:””}}]
Does anyone let me know how to get subfield-value via pipe function?
In addition, “Content” has following structure.
Content : [
{
Create : ISODate(“…”),
Desc : “…”
},
{
Create : ISODate(“…”),
Desc : “…”
}
]
Your Content structure is not a single value but an array of values. This means it cannot be loaded into the field Data.Content which is a single value of type DataContent.
Change your Data.Content field to a slice type ([]DataContent) and it will work:
type Data struct {
Id bson.ObjectId `bson:"_id,omitempty"`
Desc string `bson:"desc"`
Content []DataContent `bson:"content"`
}