I have a problem, deciding how to design my data model for later easy querying and extracting of field values...
The thing is, I use the MEAN stack and I have two collections in my MongoDB database: FA and FP.
var FASchema = new Schema({
Timestamp: Date,
ProgBW: Number,
posFlexPot: Number,
negFlexPot: Number,
Leistungsuntergrenze: Number,
Leistungsobergrenze: Number,
posGesEnergie: Number,
negGesEnergie: Number,
Preissignal: Number,
Dummy1: Schema.Types.Mixed,
Dummy2: Schema.Types.Mixed,
Dummy3: Schema.Types.Mixed
//same: Dummy: {}
});
var FPSchema = mongoose.schema( {
_id: { type: String },//mongoose.Schema.Types.ObjectId,
Demonstrator: Number,
erstellt: {type: Date, 'default': Date.now},
von: Date,
bis: Date,
FAs: [{type: mongoose.Schema.Types.ObjectId, ref: "FA"}]
})
First question here, is it possible to automatically create _id fields as Strings, because I heard it should be easier to query for id's as strings later..
Second question: my FP schema contains instances (or rather subdocuments as plain text) inside the field "FAs" (How exactly this is done, is a later topic). My question is, should I draft that field as an array of documents
[{FAinstance1.field1value, FAinstance1.field2.value},{FAinstance2.fiel1.value,...}]
with lots of FA documents, or should I do something like this (document of documents of arrays:
{FA.field1: [valueFA1, valueFA2,..], FA.field2: [value2FA1, value2FA2,..],...}
Because for each "FA" document, I later want to extract some values from fields like Timestamp, negGesEnergie, etc. and do this for every FA instance in the list. I extract them either from a mongoDB or directly from a API (e.g POST request).
Because I want to plot the values later in some chart (written in javascript), where every FA1_instance.x1 value and FA2_instance.x1 value (coming sequentially in the list) represents the numbers on the x-axis, and other fields are the sequence of numbers for the y-values (which are the values of the sequences of FA instances in the list) accordingly.
What would be the easiest data model to extract the values for each FA instance field later on?
Related
In a nodejs backend I have a function where I insert some values on a MongoDB collection.
I'm using this:
const result = await SomeModel.insertMany(someArrayOfData);
This takes a lot of time, like 2 minutes.
I'm inserting like 11.000 documents, each document has this schema:
const someModelSchema = mongoose.Schema({
IDEvento: Number,
descMet: String,
metrica: Number,
local: String,
porcent: Number,
tipoBono: String,
year: String
}, { timestamps: true });
Once is inserted I can see the values like:
Why could be some reasons for taking so much? the array is not a big array of data (the length is 2800), only 2800 documents, and each document are pretty simple, not objects nor long strings, they have very short values.
Also, there's a:
await SomeModel.deleteMany({ year });
And takes a lot of time too, like 1 minute
As part of my MongoDB, I have three different collections - A, B and AtoB.
A and B are different types of entities, where AtoB connects between them as follows-
A:
{
_id: ObjectId,
timestamp: Date,
keyA: string
}
B:
{
_id: ObjectId,
timestamp: Date,
keyB: number
}
AtoB:
{
_id: ObjectId,
aId: ObjectId, // Points to a document from A
bId: ObjectId // Points to a document from B
}
I created a TTL index on A documents - that will be deleted when the timestamp key is older than an hour.
Is it possible somehow to remove all the related AtoB documents, based on the removed _id property of the removed As?
In other words, is it possible to not only remove the A documents using the TTL, but also remove the related documents of the ones the were removed?
Thanks
In a word - no, not in this set up.
The options you have:
set up a changestream worker to delete links
set up a cron job to clean up links collection every minute
embed AtoB links into A documents
I would recommend the later, but it really depends on how feasible the change is for the rest of your application. Having a dedicated lookup collection is really a RDBS practice. It has very niche usecases in Mongo universe.
Your A documents will looks like this:
{
_id: ObjectId,
timestamp: Date,
keyA: string,
bIds: [
bId: ObjectId,
bId: ObjectId,
....
]
}
When the document's ttl expires the document is removed with all links at once.
I am designing a schema where a User document stores the Skill ids. A Skill document look like this
Skill = {
id: ObjectId
title: String
description: String
}
User = {
id: ObjectId
skills: [ObjectId]
}
And on the frontend side, a user can add Skills by searching it's title. Therefore, I indexed skills collection by {id: 1, title: "text"}. I want to know whether ordering matters when we combine text indexes with the numeric ones.
Yes, it makes a big difference, the documentation about composite text indexes says
If the compound text index includes keys preceding the text index key, to perform a $text search, the query predicate must include equality match conditions on the preceding keys.
That means if you do {id: 1, title: 'text'} you can only use the text index on title if you also constrain the search to within a single id.
If you do {title: 'text', id: 1} you will be able to text-search title by itself, or further constrained by id, or retrieve the matching id for the text search results.
Are you sure you need that id column in the index? Why not just the text index?
Yes, it does matter. Straight from the docs:
The order of the fields listed in a compound index is important. The index will contain references to documents sorted first by the values of the item field and, within each value of the item field, sorted by values of the stock field. See Sort Order for more information.
In your current structure if you only query the title field without an restrictions (no query) on the id field you will not be able to utilize the index.
I try to understand how to best structure a MongoDB Schema and therefore looking for guidance especially on using substructures (embedded documents) vs. a flat data structure.
Let's imagine we want to store a User account within MongoDB. The user has only one address, therefore we could choose one of the two following structures:
{
_id: String,
username: String,
firstname: String,
surname: String,
email: String,
street: String,
city String,
zip: Number,
}
or
{
_id: String,
name: {
first: String,
last: String,
}
email: String,
address: {
street: String,
city String,
zip: Number,
}
}
What are the advantages / disadvantages of each of the structures. Is there a rule when to use substructures or when to use a flat structure? What is the reasoning for one against the other?
Thank you in advance!
There are various data modeling patterns and schema design provided in MongoDB.I will share my experience what problems I have faced and what are the benefits of different DB schema. We will discuss it one by one below:
Embedded VS Flat data structure: In this case there is not much difference between both of the pattern but in case of data model in embedded form we are grouping similar kind of data so that makes your query little bit easy or small in size while you will $project data from any collection.
For example: if you want to fetch complete address then in case of embedded doc you don't need to $project address fields individually and if you want to skip address field while fetching document then you do not need to skip address fields individually.
Embedded (one to one) VS Embedded (one to many): As we discuss benefits of the embedded document on flat data structure but in case, if our users are having more then one addresses then we need to go for embedded documents with one to many relationship.
The schema for defining one to one and one to many relationship is as below:
One To One Relation schema:
{
_id: String,
name: {
first: String,
last: String,
}
email: String,
address: {
street: String,
city String,
zip: Number,
}
}
One To Many Relationship schema:
{
_id: String,
name: {
first: String,
last: String,
}
email: String,
address: [{ // Embedded address doc with one to many relationship
street: String,
city String,
zip: Number,
}]
}
In case of one to one relationship it will not that much affect your query part but if you will go with one to many relationship there will be many conceptual changes in your query.
For example: As mainly we are facing different scenarios while updating both kinds of data structures so I will share the difference between update queries.
To update data embedded with one to one relationship you can simply use dot notation.
db.collection.update(
{ _id: 'anyId' },
{ $set: { "address.street": "abc" } }
)
To update data embedded with one to many relationship you need to use $ operator. In this one there are two different cases. First, if you want to update specific element of subdocument and second if you want to update all subdocuments:
Case 1 query will be (with the use of $ operator):
db.collection.update(
{ 'address.streent': 'abc' },
{ $set: { "address.$.street": "xyz" } }
)
Case 2 query will be (with use of $[]):
db.collection.update(
{ 'address.streent': 'abc' },
{ $set: { "address.$[]": "xyz" } }
)
We're using collection2 (obviously with simple schema) and trying to save an array of objects to a single property on the Meteor.users collection. For example our data might be:
[
{name: "paul"},
{name: "darryn"},
{name: "tom"}
]
in reality our object is more complex
when trying to do this with $set in an update on the users collection we've either gotten 500's or managed to delete the user object entirely when turning off validation.
we've also gotten the following error a number of times:
Validation object must have at least one operator / meteor mongo
This StackOverflow Question mentions it but doesn't offer a solution that makes sense in our context.
My question is two fold. How should the schema be defined for this as we've been trying with type: [Object] which I'm not sure is right, and secondly how should the update statement be created in the method.
Any thoughts, or help would be amazing.
First define the schema for your complex object. Here I've just added age as a key:
Schema.Foo = new SimpleSchema({
name: { type: String },
age: { type: Number, min: 0 }
});
Then augment the user schema with a key whose type is an array of the type you just defined
Schema.User = new SimpleSchema({
foo: { type: [Schema.foo] },
etc...
});