Ensure MongoDB expires data at dynamic time intervals and calls are idempotent - mongodb

Question

I am using MongoDB to save user-generated links in storage. The user can state how long they want the URL to be saved before it is expired.Every user id is unique too.
Ideally, I would like my requests to be idempotent. I would like to make as many calls without having to check if there was an expiry value on the last call.
My code below seems to give me:
"Index with name: creationtime_1 already exists with different options" or
index does not exist.
This is my first run with MongoDB and I would appreciate any insights.I think I might have redundant checks on my code too but I can't figure out how else to do it
```
//mongo settings
sessionTTL := mgo.Index{
Key: []string{"creationtime"},
Unique: false,
DropDups: false,
Background: true,
ExpireAfter: time.Hour * time.Duration(expires)} // Expire in expire time
// START MONGODB
session, err := mgo.Dial(tokenMongo)
if err != nil {
return "", err
}
defer session.Close()
//session.SetSafe(&mgo.Safe{})
// Optional. Switch the session to a monotonic behavior.
id := uuid.NewV4().String()
thistime := time.Now().Local()
// find index
err = session.DB("tokenresults").C("tokenurl").Find(bson.M{"id": id}).One(&result)
if err == nil{
//Drop old values if exist // cant drop if empty
if err := session.DB("tokenresults").C("tokenurl").DropIndex("creationtime"); err != nil {
return "", err
}
}
//add stuff
c := session.DB("tokenresults").C("tokenurl")
err = c.Insert(&TokenUrl{id, tokenstring, thistime}, )
if err != nil {
return "", err
}
// create index //add session ttl // cant create if exists
if err := session.DB("tokenresults").C("tokenurl").EnsureIndex(sessionTTL); err != nil {
return "", err
}
```

Answer 1

The Solution
The approach is documented: Use a date field, set the value to the date the document expires, create a TTL-Index with ExpireAfterSeconds set to 0 and the MongoDB background TTL purging process will delete the expired documents.
Notes
However, there is some fuzziness in using TTL indices. Since it would be too costly to have a process for each document which is to be expired, waiting for the expiration time and then deleting the document, MongoDB chose a different solution. There is a background process which checks for expired documents once a minute. So there is no guarantee that your documents will expire immediately at their expiration time and a document might exist up to slightly under 2 minutes longer than the set date of expiration (missing the first run because of overload or whatever and only being deleted in the next run). Note however that this only occurs under very special circumstances. Usually, your documents get deleted within the minute of their expiration.
Explanation
What we basically do here is to add a field ExpirationDate and create a TTL index which is set to check for this expiration date. To which value this ExpirationDate is set is totally up to you. Use a Factory pattern to generate Sessions or whatever.
Note that there are some caveats explained in the code below.
package main
import (
"flag"
"fmt"
"log"
"time"
mgo "gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
const (
// SESSION_TIMEOUT is a fixed and relatively short
// timeout for demo purposes
SESSION_TIMEOUT = 1 * time.Minute
)
// Session is just a sample session struct
// with various session related data and the
// date on which a session should expire.
type Session struct {
ID bson.ObjectId `bson:"_id"`
User string
Foo string
Bar string
ExpirationDate time.Time `bson:"expirationDate"`
}
// NewSession is just a simple helper method to
// return a session with a properly set expiration time
func NewSession(user, foo, bar string) Session {
// We use a static timeout here.
// However, you can easily adapt this to use an arbitrary timeout.
return Session{
ID: bson.NewObjectId(),
User: user,
Foo: foo,
Bar: bar,
ExpirationDate: time.Now().Add(SESSION_TIMEOUT),
}
}
var (
mgohost string
mgoport int
db string
col string
)
func init() {
flag.StringVar(&mgohost, "host", "localhost", "MongoDB host")
flag.IntVar(&mgoport, "port", 27017, "MongoDB port")
flag.StringVar(&db, "db", "test", "MongoDB database")
flag.StringVar(&col, "collection", "ttltest", "MongoDB collection")
}
func main() {
flag.Parse()
c, err := mgo.Dial(fmt.Sprintf("mongodb://%s:%d/%s", mgohost, mgoport, db))
if err != nil {
log.Fatalf("Error connecting to '%s:%d/%s': %s", mgohost, mgoport, db, err)
}
// We use a goroutine here in order to make sure
// that even when EnsureIndex blocks, our program continues
go func() {
log.Println("Ensuring sessionTTL index in background")
// Request a conncetion from the pool
m := c.DB(db).Session.Copy()
defer m.Close()
// We need to set this to 1 as 0 would fail to create the TTL index.
// See https://github.com/go-mgo/mgo/issues/103 for details
// This will expire the session within the minute after ExpirationDate.
//
// The TTL purging is done once a minute only.
// See https://docs.mongodb.com/manual/core/index-ttl/#timing-of-the-delete-operation
// for details
m.DB(db).C(col).EnsureIndex(mgo.Index{ExpireAfter: 1 * time.Second, Key: []string{"expirationDate"}})
log.Println("sessionTTL index is ready")
}()
s := NewSession("mwmahlberg", "foo", "bar")
if err := c.DB(db).C(col).Insert(&s); err != nil {
log.Fatalf("Error inserting %#v into %s.%s: %s", s, db, col, err)
}
l := Session{}
if err := c.DB(db).C(col).Find(nil).One(&l); err != nil {
log.Fatalf("Could not load session from %s.%s: %s", db, col, err)
}
log.Printf("Session with ID %s loaded for user '%s' which will expire in %s", l.ID, l.User, time.Until(l.ExpirationDate))
time.Sleep(2 * time.Minute)
// Let's check if the session is still there
if n, err := c.DB(db).C(col).Count(); err != nil {
log.Fatalf("Error counting documents in %s.%s: %s", db, col, err)
} else if n > 1 {
log.Fatalf("Uups! Someting went wrong!")
}
log.Println("All sessions were expired.")
}