I'm looking for a solution, using MongoDB, to regroup/aggregate/whateverthenameis a specific field present in each collection inside a new collection or view.
It is my first time using MongoDB so I'm not familiar with it. What the project I've joined has, is a MongoDB database with multiple collections that save the same kind of information but from different provider.
Each collection has the field called "legalInformation" that has a name and an identifier. What we actually have in our project is an other collection, called name-id that duplicates informations from the provider's collection legalInformation. The purpose of the name-id collection is to centralize every name-id in the app, regardless of the provider. But I think that we could create a collection/view instead of programmatically duplicates those data.
I don't know what MongoDB can offer to me to achieve this. I would like to have a way to fetch and aggregate all the legalInformation from all the providers inside on collection/view.
As anyone an idea about how I could do this ?
To illustrate, this is a representation of the DB schema:
providerA({
legalInformations: { name: ..., id: ... },
specificDataFromProviderA: { ... }
})
providerB({
legalInformations: { name: ..., id: ... },
specificDataFromProviderB: { ... }
})
providerC({
legalInformations: { name: ..., id: ... },
specificDataFromProviderC: { ... }
})
and I want a simple collection/view called legalInformation that aggregates all legalInformations
legalInformation({
name: ...,
id: ...
})
Thanks !
Related
I need store and build fast query for next structure:
class Model {
id: number:
alias: string;
schema: Record<string, any>;
}
where schema it's can be:
{
someField: '$model_alias',
otherField: {
nestedField: '$other_model_alias'
}
}
Example data:
{ id: 1, alias: "model_one", schema: { field1: "test", field2: "demo" } }
{ id: 2, alias: "model_second", schema: { someField: "$model_one", otherField: { nestedField: 5 } }
{ id: 3, alias: "model_third", schema: { field5: "$model_second", field6: "$model_one", field7: "$model_fourth" } }
{ id: 4, alias: "model_fourth", schema: { field8: "$model_second" } }
As you can see, json field schema contains fields which may refer to another models with schemas. Thus, there can be a lot of nesting, and relationships can be many-to-many.
Is it possible to achieve such a structure with Postgres or should some alternative be used? I need possible to easy manage structure and very fast queries (get tree children or get tree parents).
Thanks.
Choosing the right db type is tricky at the best of times. Can you provide more information about what sorts of queries you'd be doing? And how big is your dataset?
If your requirement is to exclusively get the parents and children of a model, a relational db such as postgres would do it. If the relations are many-to-many, you'll have a bridging table (https://dzone.com/articles/how-to-handle-a-many-to-many-relationship-in-datab), and will be able to do efficient queries on that.
If you're doing significant, multi-hop traversals between the relationships, you might indeed want to look at a graph database to avoid expensive joins. Postgres even has a plugin that allows this: https://www.postgresql.org/about/news/announcing-age-a-multi-model-graph-database-extension-for-postgresql-2050/
I wouldn't recommend a document store for data that's heavily relational like this, just because managing relationships between documents has to be handled manually by the user, and that's normally more trouble that it's worth.
I have a model that is scattered all around the application. I have a redux state tree:
{
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
I need to keep a reference to mongoDb __v in my state too. Where is the best place to place it?
I was thinking about a separate branch model_metadata that would keep the metadata about docs (_id, __v, ...).
{
model_metadata: { <------------------------ HERE
model: {
_id: id,
__v: 2
}
}
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
Is it a valid approach or would you recommend a different one?
Every reducer only can access its own part of state, so when you do
combineReducers({
one,
another
});
and access state in one, it is equivalent to doing store.getState().one, and the same for another. So, you need to split the data in page property of state into two parts: actual data and metadata. Just like the object you retrieve from Mongo.
The point in having metadata and actual data being processed by the same reducer is that every time a reducer function is performed, you have everything you need about your object in state argument of that function. Splitting the data into two different reducers would make things way more complicated.
So, the new data representation in page would look like
{
model_metadata: { <------------------------ HERE
model: {
_id: id,
__v: 2
}
}
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
while connecting to page would look like
connect(state => ({
page: state.page
})(...)
I am very new to meteor.js and try to build an application with it. This time I wanted to try it over MEAN stack but at this point I am struggled to understand how to join two collection on server side...
I want very identical behaviour like mongodb populate to fetch some properties of inner document.
Let me tell you about my collection it is something like this
{
name: 'Name',
lastName: 'LastName',
anotherObject: '_id of another object'
}
and another object has some fields
{
neededField1: 'asd',
neededField2: 'zxc',
notNeededField: 'qwe'
}
So whenever I made a REST call to retrieve the first object I want it contains only neededFields of inner object so I need join them at backend but I cannot find a proper way to do it.
So far while searching it I saw some packages here is the list
Meteor Collections Helper
Publish with Relations
Reactive joins in Meteor (article)
Joins in Meteor.js (article)
Meteor Publish Composite
You will find the reywood:publish-composite useful for "joining" related collections even though SQL-like joins are not really practical in Mongo and Meteor. What you'll end up with is the appropriate documents and fields from each collection.
Using myCollection and otherCollection as pseudonyms for your two collections:
Meteor.publishComposite('pseudoJoin', {
find: function() {
return myCollection.find();
},
children: [
{
find: function(doc) {
return otherCollection.find(
{ _id: post.anotherObject },
{ fields: { neededField1: 1, neededField2: 1 } });
}
}
]
});
Note that the _id field of the otherCollection will be included automatically even though it isn't in the list of fields.
Update based on comments
Since you're only looking to return data to a REST call you don't have to worry about cursors or reactivity.
var myArray = myCollection.find().fetch();
var myOtherObject = {};
var joinedArray = myArray.map(function(el){
myOtherObject = otherCollection.findOne({ _id: el.anotherObject });
return {
_id: el._id,
name: el.name,
lastName: el.lastName,
neededField1: myOtherObject.neededField1,
neededField2: myOtherObject.neededField2
}
});
console.log(joinedArray); // These should be the droids you're looking for
This is based on a 1:1 relation. If there are many related objects then you have to repeat the parent object to the number of children.
I'm working on a project that use MongoDB; and I would like to hear your opinion about a feature I'd like to implement.
In paticular there are "Users" that reside in "Cities" where they offer "Services".
I have created three Collections representing the three above mentioned entities:
the User collection has a one-to-one reference with City and a one-to-many one with Service.
I would like making a search function that search in the user collection and in referenced collections for a given string available.
Therefor given the following two users, two cities and three services ...
User
{
_id:"u1",
name:"Jhon",
City: ObjectId("c1"),
Services: [
ObjectId("s1"),
ObjectId("s2")
]
}
{
_id:"u2",
name:"Jack",
City: ObjectId("c2"),
Services: [
ObjectId("s2"),
ObjectId("s3")
]
}
City
{
_id:"c1",
name: "Rome"
}
{
_id:"c2",
name: "London"
}
Services
{
_id:"s1",
name: "Repair"
}
{
_id:"s2",
name: "Sell"
}
{
_id:"s3",
name: "Buy"
}
...and searching for the word "R", the result should be the u1 user (due to the R in "Rome" and "Repair").
Given that I cannot do joins, I was thinking making a mongo shell script that adds an additional field to the User collection with all the searcheable referenced strings.
As in the following example
{
_id:"u1",
name:"Jhon",
City: ObjectId("c1"),
Services: [
ObjectId("s1"),
ObjectId("s2")
],
"idx":{
city: "Rome",
services:["Repair","Sell"]
}
}
Finally the question(s)...
Do you think is it a good choice? And Can you propose an alternative solution (or share a link about that, i didn't find nothing usefull)?
And how would you mantain that field constantly updated; for instance, What about if the referenced city name or the services offered by a user change?
I am creating a blog system in Node.js with mongodb as the db.
I have contents like this: (blog articles):
// COMMENTS SCHEMA:
// ---------------------------------------
var Comments = new Schema({
author: {
type: String
},
content: {
type: String
},
date_entered: {
type: Date,
default: Date.now
}
});
exports.Comments = mongoose.model('Comments',Comments);
var Tags = new Schema({
name: {
type: String
}
});
exports.Tags = mongoose.model('Tags',Tags);
// CONTENT SCHEMA:
// ---------------------------------------
exports.Contents = mongoose.model('Contents', new Schema({
title: {
type: String
},
author: {
type: String
},
permalink: {
type: String,
unique: true,
sparse: true
},
catagory: {
type: String,
default: ''
},
content: {
type: String
},
date_entered: {
type: Date,
default: Date.now
},
status: {
type: Number
},
comments: [Comments],
tags: [Tags]
}));
I am a little new to this type of database, im used to MySQL on a LAMP stack.
Basically my question is as follows:
whats the best way to associate the Contents author to a User in the
DB?
Also, whats the best way to do the tags and categories?
In MYSQL we would have a tags table and a categories table and relate by keys, I am not sure the best and most optimal way of doing it in Mongo.
THANK YOU FOR YOUR TIME!!
Couple of ideas for Mongo:
The best way to associate a user is e-mail address - as an attribute of the content/comment document - e-mail is usually a reliable unique key. MongoDB doesn't have foreign keys or associated constraints. But that is fine.
If you have a registration policy, add user name, e-mail address and other details to the users collection. Then de-normalize the content document with the user name and e-mail. If, for any reason, the user changes the name, you will have to update all the associated contents/comments. But so long as the e-mail address is there in the documents, this should be easy.
Tags and categories are best modelled as two lists in the content document, IMHO.
You can also create two indices on these attributes, if required. Depends on the access patterns and the UI features you want to provide
You can also add a document which keeps a tag list and a categories list in the contents collection and use $addToSet to add new tags and categories to this document. Then, you can show a combo box with the current tags as a starting point.
As a final point, think through the ways you plan to access the data and then design documents, collections & indices accordingly
[Update 12/9/11] Was at MongoSv and Eliot (CTO 10gen) presented a pattern relevant to this question: Instead of one comment document per user (which could grow large) have a comment document per day for a use with _id = -YYYYMMDD or even one per month depending on the frequency of comments. This optimizes index creation/document growth vs document proliferation (in case of the design where there is one comment per user).
The best way to associate the Content Authors to a User in the MongoDB, is to take an array in Author Collection which keeps an reference to User. Basically Array because One Content/Book may have multiple Authors i.e. you need to associate one Content to many Users.
The best way for category is to create a different collection in your DB and similarly as above keep a array in Contents.
I hope it helps at-least a little.