I have just started using MongoDB so I apologize if this is an obvious or simple question.
I am trying to store my information using the following data structure:
database: {
Customers: {
Date_added_1: {
{customer 1 info},
{customer 2 info},
{customer 3 info}
}
Date_added_2: {
{customer 1 info},
{customer 2 info}
}
}
Employees: {
Date_hired_1: {
{employee 1 info},
{employee 2 info}
}
Date_hired_2: {
{employee 1 info}
}
}
}
The code I have written to input the information into the database looks like this:
from pymongo import MongoClient
def addLeadsToDatabase(personCategory, personInformation, date):
client = MongoClient('localhost', port#)
db = client.database[personCategory][date]
db.insert({personInformation})
person_to_add = {'Name':'John Smith', 'Phone':'888-888-8888', 'Email':'example#email.com'}
addLeadsToDatabase('Customers', person_to_add, '06/28/2017')
However when navigating through the database it looks like each [personCategory][date] is getting saved as a separate collection. Rather than storing the data first within the personCategory collection, and then within the date sub-collection.
Therefore when I run 'show collections' in the MongoDB shell it outputs:
Customers.6/25/2017
Employees.6/25/2017
Customers.6/26/2017
Customers.6/27/2017
Employees.6/27/2017
Rather than just:
Customers
Employees
With the date category stored within each.
Is there a way to store the data the way I have described so it is not making a new collection each time I run the code and just storing data within the appropriate collection(s)?
Please consider:
from pymongo import MongoClient
def addLeadsToDatabase(personCategory, personInformation):
client = MongoClient('localhost', port#)
db = client.database[personCategory]
db.insert({personInformation})
person_to_add = {'Name':'John Smith', 'Phone':'888-888-8888', 'Email':'example#email.com', 'Date': '06/28/2017' } //I would suggest you to use new Date()
addLeadsToDatabase('Customers', person_to_add)
Summary:
Keep it simple. Later on you can filter by date or even aggregate your query by this date.
Related
I'm looking for a solution, using MongoDB, to regroup/aggregate/whateverthenameis a specific field present in each collection inside a new collection or view.
It is my first time using MongoDB so I'm not familiar with it. What the project I've joined has, is a MongoDB database with multiple collections that save the same kind of information but from different provider.
Each collection has the field called "legalInformation" that has a name and an identifier. What we actually have in our project is an other collection, called name-id that duplicates informations from the provider's collection legalInformation. The purpose of the name-id collection is to centralize every name-id in the app, regardless of the provider. But I think that we could create a collection/view instead of programmatically duplicates those data.
I don't know what MongoDB can offer to me to achieve this. I would like to have a way to fetch and aggregate all the legalInformation from all the providers inside on collection/view.
As anyone an idea about how I could do this ?
To illustrate, this is a representation of the DB schema:
providerA({
legalInformations: { name: ..., id: ... },
specificDataFromProviderA: { ... }
})
providerB({
legalInformations: { name: ..., id: ... },
specificDataFromProviderB: { ... }
})
providerC({
legalInformations: { name: ..., id: ... },
specificDataFromProviderC: { ... }
})
and I want a simple collection/view called legalInformation that aggregates all legalInformations
legalInformation({
name: ...,
id: ...
})
Thanks !
I have a complete list of catalog inventory data in Mongo
The basic schema is:
productSku (string)
inventory (number)
This collection consists of approximately 14 million records.
I have another list of actively sold products with a similar schema.
Right now I have it as a json file.
It consists of approximately 23,000 records.
Every 5 hours the 14 million records updates with the latest inventory data.
Once that happens I need to create a CSV of the 23,000 product's latest inventory.
I'm doing it like this:
const inventoryModel = require('../data/inventoryModel');
const activeProducts = require('./activeProducts.json');
const inventoryUpdate = [];
for (const product of activeProducts) {
let latest = await inventoryModel.findOne({ productSku: product.sku }).exec()
latest = latest ? latest._doc : null;
// If there's no current inventory record for the product
if (!lastest) {
// If there was previously an inventory greater than 0
if (product.inventory) {
// Set the latest inventory to you
inventoryUpdate.push({ sku: product.sku, inventory: 0 });
}
} else {
// If there's a change in inventory
if (latest.inventory != product.inventory) {
inventoryUpdate.push({ sku: product.sku, inventory: latest.inventory });
}
}
}
This gives me an array inventoryUpdate that I can use to create a CSV for a mass update. This works fine but it's very slow. It takes about an hour to complete!
I was thinking about maybe adding activeProducts to Mongo as well and if I can somehow keep the execution of the logic within Mongo this would be a lot faster. If possible this is beyond my current understanding and ability.
Anyone have any suggestions?
I'm currently using mongoose v. 5.25, against mongoDB v.3.6.
My application is supposed to query data from many different views, for instance, a view I currently have at my DB: db.joboffers_view.find()
will return many records that have been aggregated from different collections.
For a normal collection model, I query it like so:
const model = db.model(attribute);
/*where attribute, can be any registered schema */
model.find().
then((result) => {
resolve(result);
}).
catch((err) => {
reject(err);
});
Then way I register my models is something like this (simplified code):
//...
//abstracting boring connection methods
const db = mongoose.connection
//...
//simple model schema
const users_schema = {
_id: ObjectId,
another_field: String
};
//here I'm registering a schema for a VIEW, instead of normal collection
const view_schema = {
_id: ObjectId,
another_field: String
};
//...
//then
db.model('users', users_schema);
db.model('view', view_schema);
When I run a query from any of my registered models, I get the results just fine. However, when I run it against a model that represents a view on my mongo database, it returns an empty array.
No errors, no nothing, just an empty array.
I have looked through mongoose documentation, and I didn't find any specific method or pattern for querying a view, instead of a collection data.
It seems to be the same way I would do for any other collection I have in my system.
Am I missing something?
I also faced the same issue and figured out the problem is that mongoose, by default, reads collection names by pluralizing the model/view name.
So when you create any view and want to use it in mongoose, either make sure your view name is plural (add s to end of view name) or pass a collection name when initializing a schema.
Example
const users_schema = {
_id: ObjectId,
another_field: String
};
mongoose.model('vw_user_info', users_schema, 'vw_user_info');
I have same problem, but i solved it, please check the name of the view in mongodb, it must be same with db.model('view_name', view_schema);
You can open Mongoose debug by config like this mongoose.set('debug', true);
Add 3rd argument
db.model('view', view_schema, 'view_name_in_db')
I have a chat app, that is using Ionic 2 and Meteor with MongoDB. It works perfectly.
However, everything is stored in the MongoDB on the server, so each time a user wants to view their messages, they need to be connected to the Meteor/Mongo Server running in the cloud. Also, if one user deletes their chat, it will delete the chat on the MongoDB, and the corresponding other user will also have their chat deleted.
I would like similar functionality as WhatsApp where the messages are held locally on the device (I am using SQLite), and only new messages are held in the cloud until both users download them.
Currently my app iterates over a Mongo.Cursor<Chat> object. It also observes this object (this.chats.observe({changed: (newChat, oldChat) => this.disposeChat(oldChat), removed: (chat) => this.disposeChat(chat)});).
I get chat data from SQLlite that I have stored locally (Array<Chat>).
Question
Is it possible to add the SQLite data (Array<Chat>) to the Mongo.Cursor<Chat>? When I do so, I want to just add to minimongo and not MongoDB on the server.
Thanks
UPDATE
Asp per advise below, I do the following:
let promise: Promise<Mongo.Cursor<Chat>> = new Promise<Mongo.Cursor<Chat>>(resolve => {
this.subscribe('chats', this.senderId, registeredIds, () => {
let chats: Mongo.Cursor<Chat> = Chats.find(
{ memberIds: { $in: registeredIds } },
{
sort: { lastMessageCreatedAt: -1 },
transform: this.transformChat.bind(this),
fields: { memberIds: 1, lastMessageCreatedAt: 1 }
}
);
this.localChatCollection = new Mongo.Collection<Chat>(null);
console.log(this.localChatCollection);
chats.forEach(function (chat: Chat) {
console.log('findChats(): add chat to collection: ' + chat);
this.localChatCollection.insert(chat);
});
Will update if it works.
UPDATE
When I do the following, it inserts the chat object:
let promise: Promise<Mongo.Collection<Chat>> = this.findChats();
promise.then((data: Mongo.Collection<Chat>) => {
let localChatCollection: Mongo.Collection<Chat> = new Mongo.Collection<Chat>(null);
data.find().forEach(function (chat: Chat) {
console.log('==> ' + chat);
localChatCollection.insert(chat);
});
However, if I define the localChatCollection globally, it does not insert the chat object. There are no errors but the process just stops on the insert line.
private localChatCollection: Mongo.Collection<Chat> = new Mongo.Collection<Chat>(null);
....
this.localChatCollection.insert(chat);
Any ideas how I can get this to insert into a globally defined collection?
Is it possible to add the SQLite data (Array) to the Mongo.Cursor? When I do so, I want to just add to minimongo and not MongoDB on the server.
Meteor itself knows nothing about SQLite, but it sounds like you have that part of it working.
To just add to minimongo and not the mongodb server, you're looking for a client-side collection. Just pass null in as the first parameter to the call to create your collection i.e.
var localChatCollection = new Mongo.Collection(null)
You can then insert to localChatCollection the same way you would with a synchronized collection.
Source: Meteor docs
I am very new to meteor.js and try to build an application with it. This time I wanted to try it over MEAN stack but at this point I am struggled to understand how to join two collection on server side...
I want very identical behaviour like mongodb populate to fetch some properties of inner document.
Let me tell you about my collection it is something like this
{
name: 'Name',
lastName: 'LastName',
anotherObject: '_id of another object'
}
and another object has some fields
{
neededField1: 'asd',
neededField2: 'zxc',
notNeededField: 'qwe'
}
So whenever I made a REST call to retrieve the first object I want it contains only neededFields of inner object so I need join them at backend but I cannot find a proper way to do it.
So far while searching it I saw some packages here is the list
Meteor Collections Helper
Publish with Relations
Reactive joins in Meteor (article)
Joins in Meteor.js (article)
Meteor Publish Composite
You will find the reywood:publish-composite useful for "joining" related collections even though SQL-like joins are not really practical in Mongo and Meteor. What you'll end up with is the appropriate documents and fields from each collection.
Using myCollection and otherCollection as pseudonyms for your two collections:
Meteor.publishComposite('pseudoJoin', {
find: function() {
return myCollection.find();
},
children: [
{
find: function(doc) {
return otherCollection.find(
{ _id: post.anotherObject },
{ fields: { neededField1: 1, neededField2: 1 } });
}
}
]
});
Note that the _id field of the otherCollection will be included automatically even though it isn't in the list of fields.
Update based on comments
Since you're only looking to return data to a REST call you don't have to worry about cursors or reactivity.
var myArray = myCollection.find().fetch();
var myOtherObject = {};
var joinedArray = myArray.map(function(el){
myOtherObject = otherCollection.findOne({ _id: el.anotherObject });
return {
_id: el._id,
name: el.name,
lastName: el.lastName,
neededField1: myOtherObject.neededField1,
neededField2: myOtherObject.neededField2
}
});
console.log(joinedArray); // These should be the droids you're looking for
This is based on a 1:1 relation. If there are many related objects then you have to repeat the parent object to the number of children.