So from time to time I get new exports of a cities database of POIs and info about them and I want to have all that data in my MongoDB with a Loopback-API on it. Therefore I reduce the data to my desired structure and try to import it.
For the first time I receive such an export, I can simply insert the data with insertMany().
When I get a new export, it means that it includes updated POIs which I actually want my existing POIs to be replaced with that new data. So I thought I'd use updateMany() but I could'nt figure out how I'd do that in my case.
Here's what I have so far:
const fs = require('fs');
const mongoose = require('mongoose');
const data = JSON.parse(fs.readFileSync('data.json', 'utf8'));
// Connect to database
mongoose.connect('mongodb://localhost/test', {
useMongoClient: true
}, (err) => {
if (err) console.log('Error', err);
});
// Define schema
let poiSchema = new mongoose.Schema({
_id: Number,
name: String,
geo: String,
street: String,
housenumber: String,
phone: String,
website: String,
email: String,
category: String
});
// Set model
let poi = mongoose.model('poi', poiSchema);
// Generate specified data from given export
let reducedData = data['ogr:FeatureCollection']['gml:featureMember'].reduce((endData, iteratedItem) => {
endData = endData.length > 0 ? endData : [];
endData.push({
_id: iteratedItem['service']['fieldX'],
name: iteratedItem['service']['fieldX'],
geo: iteratedItem['service']['fieldX']['fieldY']['fieldZ'],
street: iteratedItem['service']['fieldX'],
housenumber: iteratedItem['service']['fieldX'],
phone: iteratedItem['service']['fieldX'],
website: iteratedItem['service']['fieldX'],
email: iteratedItem['service']['fieldX'],
category: iteratedItem['service']['fieldX']
});
return endData;
}, []);
//
// HERE: ?!?!? Insert/update reduced data in MongoDB collection ?!?!?
//
mongoose.disconnect();
So I just want to update everything that has changed.
Of course if I leave it to insertMany() it fails due to dup key.
For the second time, use mongo's update command with upsert set to true.
db.collection.update(query, update, options)
In the query pass the _id ,in update pass the object and in option set upsert to true. This will update the document if it exists creates a new document if that doesn't exist.
Related
Problem
I'm trying to send data from my client-side form into my database and am running into the error below:
TypeError: artist.fans.push is not a function
The data is an email address that should be saved into my artist model as a subdoc (in an object).
What I've tried
I'm using the syntax from the mongo docs that says parent.children.push() is the proper way to add subdocs to arrays and parent.children.create() to create new subdocuments. Both yield the same error.
Here's my function:
module.exports.addFanEmail = async (req, res) => {
const fan = req.body;
const artist = await Artist.findById(req.params.id);
artist.fans.create(fan);
await artist.save();
res.redirect(`/artists/${artist._id}`);
}
Right now req.body is only the "fan's" email - here's an example of the object's format: { email: 'tom#test.com' }
DB Model
const artistSchema = new Schema({
image: [ImageSchema],
genre: [ String ],
fans: {
email: String,
subscribed: String,
gender: String,
age: Number
},
});
The object is coming through from the client to the function without any problem, I just can't get it to save to the db?
I have a model Franchise that has another Schema Employee as its subdocuments for a field. The structure is as follows.
Franchise.js
const Franchise = new mongoose.Schema(
{
franchiseName: String,
address: String,
managers: [Employee]
});
export default mongoose.model(
"Franchise",
Franchise
);
Employee.js
const Employee = new mongoose.Schema(
{
name: String,
email: String,
phoneNo: Number,
password: String,
});
export default Employee;
The issue I am facing is with every query to Franchise, it returns a new _id for the objects in managers field. Is there any way to make it constant ?
For instance, I am running a simple findById on Franchise and store it in franchise variable.
and then I console.log(franchise.managers).
It prints different IDs each time the query is run.
[
{
_id: new ObjectId("61925d2697852574eb0ba9ab"),
name: 'Franchise Manager 1',
email: 'franchise1#sfc.com',
phoneNo: 1234567890,
}
]
Second time the query is run:
[
{
_id: new ObjectId("61925ba8130aca93a7dd3dbc"),
name: 'Franchise Manager 1',
email: 'franchise1#sfc.com',
phoneNo: 1234567890
}
]
As you can see, the employee is the same, however it has different Id for each call. Kindly help.
Thanks.
Alright, I figured it out. The issue is that there was no _id stored in the database for existing data of managers. The Employee schema was added later on, so as the _id was not present in the database only, a new one was being generated each time.
Ok, so I am new to MongoDB and the world of document based databases. I have stored in MongoDB a collection of profiles which has two subdocuments; 'Interest' and 'Country'. Below is schema information:
var Country = new mongoose.Schema({
name: String,
countryCode: String
});
var Interest = new mongoose.Schema({
label: {
type: String
}
});
var Profile = connection.model('profile', {
uid: {
type: String,
unique: true,
sparse: true
},
username: String
country: Country,
interests: [Interest],
member_since: Date
});
Let's say I want to be able to run a fast and efficient query such that I can select all of the users interested in 'music' and whose countryCode is 'AU' but done in an efficient way that doesn't scan all documents (I'm guessing I need an index?), how can I do this? Below is a sample profile as it appears in Compass:
_id:59d17efa3ed3a453e2b865f9
username:"Rudolph"
country:Object
name: "Australia"
countryCode:"AU"
_id:59d17efa3ed3a453e2b865fa
__v:0
interests:Object
label:Array
0:"music"
1:"film"
2:"dance"
member_since:2017-10-01 19:49:14.565
I'm getting document via MongoDB Shell:
db.page_about_love.find()
But I can't get document via mongoose. What is wrong?
mongoose.connect(db_uri);
var loveSchema = new mongoose.Schema({
title: String,
content: String,
tag: String
});
mongoose.model('page_about_love', loveSchema);
var about = mongoose.model('page_about_love');
about.find(function (err, love) {
if (err) return console.error(err);
console.log(love);
});
Test output:
[]
To prevent Mongoose generating a collection name to use, you should be explicit and pass which collection name it should use:
var loveSchema = new mongoose.Schema({
title: String,
content: String,
tag: String
}, { collection : 'page_about_love' });
Otherwise, Mongoose will apply the utils.toCollectionName() function to the model name to determine the collection name, which in your case would yield page_about_loves (notice the pluralization).
More information here.
so I've been at it for like 4 hours, read the documentation several times, and still couldn't figure out my problem. I'm trying to do a simple populate() to my model.
I have a User model and Store model. The User has a favoriteStores array which contains the _id of stores. What I'm looking for is that this array will be populated with the Store details.
user.model
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
var UserSchema = new Schema({
username: String,
name: {first: String, last: String},
favoriteStores: [{type: Schema.Types.ObjectId, ref: 'Store'}],
modifiedOn: {type: Date, default: Date.now},
createdOn: Date,
lastLogin: Date
});
UserSchema.statics.getFavoriteStores = function (userId, callback) {
this
.findById(userId)
.populate('favoriteStores')
.exec(function (err, stores) {
callback(err, stores);
});
}
And another file:
store.model
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
var StoreSchema = new Schema({
name: String,
route: String,
tagline: String,
logo: String
});
module.exports = mongoose.model('Store', StoreSchema);
After running this what I get is:
{
"_id": "556dc40b44f14c0c252c5604",
"username": "adiv.rulez",
"__v": 0,
"modifiedOn": "2015-06-02T14:56:11.074Z",
"favoriteStores": [],
"name": {
"first": "Adiv",
"last": "Ohayon"
}
}
The favoriteStores is empty, even though when I just do a get of the stores without the populate it does display the _id of the store.
Any help is greatly appreciated! Thanks ;)
UPDATE
After using the deepPopulate plugin it magically fixed it. I guess the problem was with the nesting of the userSchema. Still not sure what the problem was exactly, but at least it's fixed.
I think this issue happens when schemas are defined across multiple files. To solve this, try call populate this way:
.populate({path: 'favoriteStores', model: 'Store'})