Mongoose update / schema design - mongodb

I am trying to update a mongoose model. Having some difficulites. This is how my schema looks like right now.
I want a user to have a library which consists of many songs. Those songs can be in multiple users libraries. And can also be in multiple playlists of which a user can have several.
var LibrarySchema = new Schema({
user: Schema.ObjectId
});
var PlaylistSchema = new Schema({
title: String,
description: String,
//songs: [SongsSchema],
user: Schema.ObjectId
});
var SongsSchema = new Schema({
name: String,
artist: String,
album: String,
time: Number,
url: String,
gid: String,
location: String,
playlist: [{playlist_id:Schema.ObjectId, position: Number}],
library: [Schema.ObjectId]
});
Now I want to do something like check if the url already exists in the song schema (url is an index). If that song does not exist I want to add it with all the values. However, if the song exists I want to append the playlist and position its in and the library.
For example here is the query I am trying now:
Song.update({url: s.url, "playlist.playlist_id": playlistId}, {$set: {name: s.name, artist: s.artist, album: s.album, time: s.time, url: s.url}, $addToSet: {playlist: {playlist_id: playlistId, position: s.position}, library: libId}},{upsert: true}, function(err, data) {});
And this is a sample of the database stored:
{ "name" : "Kanye West - All Of The Lights ft. Rihanna, Kid Cudi", "artist" : "unknown", "album" : "unknown", "time" : 328, "url" : "HAfFfqiYLp0", "_id" : ObjectId("4f127923ce0de70000000009"), "library" : [ ObjectId("4f1203af23c98cfab1000006") ], "playlist" : [
{
"playlist_id" : ObjectId("4f127923ce0de70000000007"),
"position" : "1"
}
] }
{ "name" : "Kanye West - Heartless", "artist" : "unknown", "album" : "unknown", "time" : 221, "url" : "Co0tTeuUVhU", "_id" : ObjectId("4f127923ce0de7000000000a"), "library" : [ ObjectId("4f1203af23c98cfab1000006") ], "playlist" : [
{
"playlist_id" : ObjectId("4f127923ce0de70000000007"),
"position" : "2"
}
] }
{ "name" : "Kanye West - Stronger", "artist" : "unknown", "album" : "unknown", "time" : 267, "url" : "PsO6ZnUZI0g", "_id" : ObjectId("4f127923ce0de7000000000b"), "library" : [ ObjectId("4f1203af23c98cfab1000006") ], "playlist" : [
{
"playlist_id" : ObjectId("4f127923ce0de70000000007"),
"position" : "3"
}
] }
Now what I am trying to do is if the url for the song is not in the database it should add it with all the information. If it is not then it will add to the library (which each user has one of) the library id UNLESS it already contains that value.
That works pretty good. The problem I am having right now is when it finds the url I want it to add to playlist the playlistId of the current playlist UNLESS it is already in the array and if it is and was passed a new value for position (user can change the order of a playlist) it should change the value of the position.
Perhaps I am not modelling it in a proper way. Any tips would be awesome!

MongooseJS contains something called DBRef. I think you want to use that, instead of handling all the references yourself.
For inserting if something is not there, either do a lookup first and insert if needed, or use an upsert. Not sure if Mongoose supports Upserts though...

Related

how to update one table with the _id from another table in Mongodb

I exported data from a MySQL database into JSON and imported it into MongoDB. The problem:
When I imported clients, MongoDB created its own _id field (I know this is built in functionality, but MySQL used a clientID, autoincrementing integer).
SO, when I imported my appointments collection, the clientID was renamed oldClientID. I'd like the clientID field to be the ObjectID of the corresponding client.
My schemas:
const apptSchema = new mongoose.Schema({
ID: Number,
clientID: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Client'
},
oldClientID: Number,
...other field data
});
const clientSchema = new mongoose.Schema({
ID: Number,
familyID: Number,
first: String,
last: String,
});
Sample Patient Data:
{
"_id" : ObjectId("5d82240f7c8ddd03b62aee6a"),
"ID" : 18,
"familyID" : 6,
"first" : "Jane",
"last" : "Doe"
}
Sample Appointment Data
{
"_id" : ObjectId("5d82c8b95627367d122087f9"),
"ID" : 1885,
"oldPatientID" : 18,
"scheduled" : ISODate("2015-08-05T11:20:00Z"),
"note" : ""
},
{
"_id" : ObjectId("5d82c8b95627367d122088a8"),
"ID" : 2066,
"oldPatientID" : 18,
"scheduled" : ISODate("2015-09-17T16:00:00Z"),
"note" : ""
}
What appointments should look like:
{
"_id" : ObjectId("5d82c8b95627367d122087f9"),
"ID" : 1885,
"clientID": ObjectId("5d82240f7c8ddd03b62aee6a"),
"oldPatientID" : 18,
"scheduled" : ISODate("2015-08-05T11:20:00Z"),
"note" : ""
},
{
"_id" : ObjectId("5d82c8b95627367d122088a8"),
"ID" : 2066,
"clientID" : ObjectId("5d82240f7c8ddd03b62aee6a"),
"oldPatientID" : 18,
"scheduled" : ISODate("2015-09-17T16:00:00Z"),
"note" : ""
}
I am open to learning how to achieve this in the mongo shell or using mongoose in express (or if there is another cool way, like in Robo3T).
MongoDB will always use _id as the primary key, this behavior cannot be overwritten, though you can use the _id with values from your custom id. Though this might be confusing, it is better to use indexing on your custom id, and you don't need to use ObjectId for the custom index field, but can use your own custom id schema, like UUID or an incrementing integer value etc. though it has to be generated/incremented by you or some framework, like JPA
Check Indexes
For Mongoose, you can do;
new mongoose.Schema({
customId: { type: Number, index: true }
...other field data
});
with index: true
Ok, this worked out for me, although I'm sure there has to be an easier way:
db.getCollection("appts").aggregate(
[
{
"$lookup" : {
"from" : "clients",
"localField" : "clientID",
"foreignField" : "ID",
"as" : "CLIENT"
}
},
{
"$lookup" : {
"from" : "appttypes",
"localField" : "type",
"foreignField" : "ID",
"as" : "TYPE"
}
},
{
"$lookup" : {
"from" : "apptstatuses",
"localField" : "status",
"foreignField" : "ID",
"as" : "STATUS"
}
},
{
"$project" : {
"_id" : "$_id",
"clientID" : "$CLIENT._id",
"scheduled" : "$scheduled",
"note" : "$note",
}
},
{
"$out" : "apptslinked"
}
]
);
Then I exported that as JSON, dropped the appts table, and did a mongoimport using that file.

Referencing for mongoose

i'm currently trying to reference a collection called items with the structure below
packageSchema = schema({
recipient: String,
contents: [{item :{type: mongoose.Schema.Types.ObjectId,
ref: 'items', required : true}, amount: String}]
Below is my code for getting one package via its id
getOnePackage : function(id,callback)
{
packageModel.findById(id,callback)
.populate('contents')
}
So when i call the above function i'm expecting to get this result
{
recipient : Dave
contents : [
{item : {
_id:5d2b0c444a3cc6438a7b98ae,
itemname : "Statue",
description : "A statue of Avery"
} ,amount : "2"},
{item : {
_id:5d25ad29e601ef2764100b94,
itemname : "Sugar Pack",
description : "Premium Grade Sugar From China"
} ,amount : "5"},
]
}
But what i got from testing in Postman is this :
{
recipient : Dave,
contents : []
}
May i know where did it went wrong? And also how do i prevent mongoose from automatically insert an objectId for every single element in the contents array....
Because element in contents array is object with item field so your populate should be:
.populate('contents.item')

Mongoose object schema: How do I declare schema in mongoosejs to be able to iterate an options field?

Currently the data in mogo via mongoose is as below. I want to be able to iterate options once I pull it out of mongo.
currently if I save this record to a variable “record". I would need to do record.options[0].option1.check, I want to avoid [0].
I want to be able to just iterate over an object and be able to do the following.
record.option1.choice1
record.option2.choice2
my schema is
var questionSchema = new mongoose.Schema({
question: String,
question_slug: String,
options: []
});
{
"_id" : ObjectId("5832609483bf491db3d8faa1"),
"question" : “What is the capital of Illinois?",
"question_slug" : “what-is-the-capital-of-illinois",
"options" : [
{
"option1" : {
"check" : "on",
"choice" : “Springfield"
},
"option2" : {
"check" : "",
"choice" : “Chicago"
}
}
],
"__v" : 0
}
You can't define the schema for record.option1.choice1 and record.option2.choice2 since option1 and option2 seems to be dynamic.
But you can avoid [0] for record.options[0].option1.check by defining options as Object in the schema instead of array as below.
Hence, you can iterate over the object record.options with record.options.option1 and record.options.option2 etc.
var questionSchema = new mongoose.Schema({
question: String,
question_slug: String,
options: {
type: Object
}
});
{
"_id" : ObjectId("5832609483bf491db3d8faa1"),
"question" : “What is the capital of Illinois?",
"question_slug" : “what-is-the-capital-of-illinois",
"options" : {
"option1" : {
"check" : "on",
"choice" : “Springfield"
},
"option2" : {
"check" : "",
"choice" : “Chicago"
}
},
"__v" : 0
}

Error in mongodb query to get movie based on id

> db.movmodels.findOne()
{
"_id" : ObjectId("55320b0e0e9e0d9d0540593c"),
"username" : "punk",
"favMovies" : [
{
"alternate_ids" : {
"imdb" : "0137523"
},
"abridged_cast" : [
{
"characters" : [
"Tyler"
],
"id" : "162652627",
"name" : "Brad Pitt"
},
{
"characters" : [
"Narrator"
],
"id" : "162660884",
"name" : "Edward Norton"
},
{
"characters" : [
"Robert"
],
"id" : "162676383",
"name" : "Meat Loaf"
},
{
"characters" : [
"Angel Face"
],
"id" : "162653925",
"name" : "Jared Leto"
},
{
"characters" : [
"Boss"
],
"id" : "770706064",
"name" : "Zach Grenier"
}
],
"synopsis" : "",
"ratings" : {
"audience_score" : 96,
"audience_rating" : "Upright",
"critics_score" : 80,
"critics_rating" : "Certified Fresh"
},
"release_dates" : {
"dvd" : "2000-06-06",
"theater" : "1999-10-15"
},
"critics_consensus" : "",
"runtime" : 139,
"mpaa_rating" : "R",
"year" : 1999,
"title" : "Fight Club",
**"id" : "13153"**
}
],
"__v" : 0
}
This is my data in mongodb.
As I am new to mongodb I wanted to know query to get movie with a particular id.
The query that I tried is. I need to get the movie based on id so that I can remove it from my database
db.movmodels.findOne({username:"punk"},{favMovies:{id:13153}})
but this gives me error.
2015-04-18T05:41:26.221-0400 E QUERY Error: error: {
"$err" : "Can't canonicalize query: BadValue ported projection option: favMovies: { id: 13153.0 }",
"code" : 17287
}
at Error (<anonymous>)
at DBQuery.next (src/mongo/shell/query.js:259:15)
at DBCollection.findOne (src/mongo/shell/collection.js:188:22)
at (shell):1:14 at src/mongo/shell/query.js:259
There are several problems with your query:
The second parameter to find() is a projection, not part of the query. What you want is to supply one document for the query that has two properties: {"username" : "punk", favMovies : { ... } }
However, you also don't want to compare the entire sub-document favMovies, but you only want to match on one of its properties, the id, which requires to 'reach into the object' using the dot operator: {username:"punk", "favMovies.id" : 13153}.
However, that will probably not work yet, because 13153 is not the same as "13153", the latter being a string while the former is a number in JSON.
db.movmodels.findOne({username:"punk", "favMovies.id" : "13153"})
Keep in mind, however, that this will find the entire document for the user named "punk". I'm not sure what exactly your data structure should look like, but it appears you'll have to $pull the movie from the user. In general, I'd say you're embedding too much data into the user, but that's hard to tell without knowing the exact use case.
Here you go:
If you just wanted to get first user who has this fav movie:
db.movmodels.findOne({"favMovies.id": 13153});
And, if you want to know if that user has that movie as favorite.
db.movmodels.findOne({"favMovies.id": 13153, username:"punk"});
Second argument in the findOne is used to only return particular field.
You can use also $elemMatch projection operator (not to be confused with the $elemMatch query operator)
db.movmodels.find({username:"punk"},{favMovies:{$elemMatch:{id:"13153"}}});
`
If you want to find a movie that has another movie (with id 13153) in 'favMovies' array, then write the query as below:
db.movmodels.findOne({username:"punk",'favMovies.id':13153})
And if you want to find a movie with _id 55320b0e0e9e0d9d0540593cwrite the following query:
db.movmodels.findOne({username:"punk",'_id':ObjectId("55320b0e0e9e0d9d0540593c")})

MongoDB: Removing duplicate document based on ObjectId?

This is really an open question. I am sorry if this goes little vague but I am trying to collect thoughts from other people since I am very new to Mongo
Situation
I realized that my collection has multiple duplicate documents (based on name key)
These documents may be same or might got changed during the subsequent dumps from file(we want to keep later changes)
Since there is no insert date, it will be hard to tell looking at document which one is latest (bad schema design)
Wanted
To remove the documents which were inserted earlier
I read that each document in collection is assigned an ObjectId(here) that makes document unique
Question
Is it possible to know which document is inserted earlier based on ObjectId and remove it using Map Reduce?
Any other thoughts and advices?
I'm bored this evening, so here we go.
Step 1. Let's prepare our test data.
> db.users.insert({name: 'John', other_field: Math.random()})
> db.users.insert({name: 'Bob', other_field: Math.random()})
> db.users.insert({name: 'Mary', other_field: Math.random()})
> db.users.insert({name: 'John', other_field: Math.random()})
> db.users.insert({name: 'Jeff', other_field: Math.random()})
> db.users.insert({name: 'Ivan', other_field: Math.random()})
> db.users.insert({name: 'Mary', other_field: Math.random()})
> db.users.find()
{
"_id" : ObjectId("501976e9bee9b253265bba8b"),
"name" : "John",
"other_field" : 0.9884713875252772
}
{
"_id" : ObjectId("501976e9bee9b253265bba8c"),
"name" : "Bob",
"other_field" : 0.048004131996396415
}
{
"_id" : ObjectId("501976e9bee9b253265bba8d"),
"name" : "Mary",
"other_field" : 0.20415803582615222
}
{
"_id" : ObjectId("501976e9bee9b253265bba8e"),
"name" : "John",
"other_field" : 0.5514446987265585
}
{
"_id" : ObjectId("501976e9bee9b253265bba8f"),
"name" : "Jeff",
"other_field" : 0.8685077449753242
}
{
"_id" : ObjectId("501976e9bee9b253265bba90"),
"name" : "Ivan",
"other_field" : 0.2842514340422925
}
{
"_id" : ObjectId("501976eabee9b253265bba91"),
"name" : "Mary",
"other_field" : 0.984048520281136
}
Step 2. The map-reduce
var map = function() {
emit(this.name, this);
};
var reduce = function(name, vals) {
var last_obj = null;
vals.forEach(function(v) {
if(!last_obj || v._id > last_obj._id) {
last_obj = v;
}
});
return last_obj;
};
db.users.mapReduce(map, reduce, {out: 'temp_coll'})
It basically groups all documents by name and then selects the one with the largest _id.
Step 3. Do something with unique data.
> db.temp_coll.find()
{
"_id" : "Bob",
"value" : {
"_id" : ObjectId("501976e9bee9b253265bba8c"),
"name" : "Bob",
"other_field" : 0.048004131996396415
}
}
{
"_id" : "Ivan",
"value" : {
"_id" : ObjectId("501976e9bee9b253265bba90"),
"name" : "Ivan",
"other_field" : 0.2842514340422925
}
}
{
"_id" : "Jeff",
"value" : {
"_id" : ObjectId("501976e9bee9b253265bba8f"),
"name" : "Jeff",
"other_field" : 0.8685077449753242
}
}
{
"_id" : "John",
"value" : {
"_id" : ObjectId("501976e9bee9b253265bba8e"),
"name" : "John",
"other_field" : 0.5514446987265585
}
}
{
"_id" : "Mary",
"value" : {
"_id" : ObjectId("501976eabee9b253265bba91"),
"name" : "Mary",
"other_field" : 0.984048520281136
}
}
For example, drop the original collection, iterate this one and insert values into new collection. Don't forget to drop the temp collection when you're done.
Important
I didn't bother with extraction of a timestamp from objectid, because I assumed that you run your import jobs not twice a second (not even every second, maybe).
Ok since object id uses timestamp as it's leading four bytes you can do this with a bit of math.
Thankfully the mongo shell has a way to get the timestamp from an object id by you will need to do some more javascript to first query your documents with the same name then store them in a temp variable (if using the command line) or in a temp table (if using drivers) and parse each individual id's using the timestamp getter that's shown in the link below.
http://www.mongodb.org/display/DOCS/Optimizing+Object+IDs#OptimizingObjectIDs-Extractinsertiontimesfromidratherthanhavingaseparatetimestampfield.
Remember that object id's are only accurate to the second so this still doesn't help in rapid insertion mode.
But either way what you are asking for is doable either in a map reduce function or in the way shown above which does it through the command line.
Give that a shot and if you get stuck let me know. If i know your collection structure i can probably whip up something real quick but only after you bang your head on it a couple of times :)