I have a "log" type of collection, where depending on the source, there might be some id-fields. I want to search those fields with queries, but not sort by them. should I index them to improve search performance? To visualize the problem:
[{
_id: ObjectID("...") // unique
userId: ObjectID("...") // not unique
createdAt: ...
type: 'USER_CREATED'
},
{
_id: ObjectID("...") // unique
basketId: ObjectID("...") // not unique
createdAt: ...
type: 'BASKET_CREATED'
},
...]
I want to filter by (nullable) userId or basketId as well as the type-enum. I am not sure if those fields need an index. createdAT certainly does since it it is sortable. But sparse fields containing enums or null (and simply non-unique) values: how should those be treated as a rule of thumb?
Related
I am trying to build an API for search jobs
Frontend input: single filed keyword with a string
API response: Return list of jobs that match any of the following fields
skills
location
company
Schemas
1.Job schema
title: {
type: String,
required: true,
},
location: {
type: mongoose.Schema.Types.ObjectId,
ref: 'location',
},
skills:[{
type: mongoose.Schema.Types.ObjectId,
ref: 'Skill'
}],
company: {
type: mongoose.Schema.Types.ObjectId,
ref: 'company',
},
As you see skills, location and company are mapped in another collection and frontend gives no separation on the keyword I am not sure which way I can write an effective search query
Right now approach is
Find skill_id based on skill name and fetch all jobs that have desired skill
Follow the same for location and company
But I am not sure this is the right approach, can somebody' advise a proper way of doing this
many strategies can be applied in your case (yours, AdamExchange's one, aggregation with $lookup stages...) depending of size of collections, indexes, etc...
But i think think you really have to look to index and index intersection strategies to really optimize your query
I would :
first create 3 single indexes on skill.name / location.name / company indexes ===>So you can find the ids in your different collections, using index.
Create single indexes on job collection : location, skill, company
Then you can simply run your queries like this (assuming MyKeyword is the value of your frontend field) [pseudo code, i don't know the language you use]:
skillId = db.skill.find({name:MyKeyword });
locationId = db.location.find({name:MyKeyword });
companyId = db.company.find({name:MyKeyword });
db.job.find({
$or: [
{
skill: {
$eq: skillId
}
},
{
location: {
$eq: locationId
}
},
{
company: {
$eq: companyId
}
}
]
})
So you can take benefit of indexes to query 'secondary collections' and of indexes intersection for each case of your $or condition for main collection.
Imaging we have documents like this:
{
_id: ObjectId(""),
accountId: ObjectId(""),
userId: ObjectId(""),
someOtherFieldA: ["some", "array", "values"],
someOtherFieldB: ["other", "array", "values"],
...
}
Furthermore there are multiple compound indices, ex.:
{ userId: 1, someOtherFieldA: 1, ... }
{ userId: 1, someOtherFieldB: 1, ... }
We want to shard by accountId.
Would it be enough to add a single field index for accountId, so that the existing indices still work? Or would all indices need the accountId as prefix (first part)?
When you run the sh.shardCollection() command then MongoDB automatically creates an index on the shard key field (unless such an index exist already), so you don't need to care about this question.
I need an index that will provide me uniqueness of the field among all fields. For example, I have the document:
{
_id: ObjectId("123"),
fieldA: "a",
fieldB: "b"
}
and I want to forbid insert the document
{
_id: ObjectId("456"),
fieldA: "new value for field a",
fieldB: "a"
}
because already exists the document that has the value "a" set on field "fieldA". Is it possible?
It seems you need a multikey index with a unique constraint.
Take into account that you can only have one multikey index in each collection for this reason you have to include all the fields you like to uniqueness inside an array
{
_id: ObjectId("123"),
multikey: [
{fieldA: "a"},
{fieldB: "b"}
]
}
Give a try to this code
db.collection.createIndex( { "multikey": 1}, { unique: true } )
To query you have to code
db.collection.findOne({"multikey.fieldA": "a"}, // Query
{"multikey.fieldA": 1, "multikey.fieldB": 1}) // Projection
For more info you can take a look at embedded multikey documents.
Hope this helps.
another option is to create a document with each unique key, indexed by this unique key and perform a loop over the field of each candidate document cancelling the write if any key is found.
IMO this solution is more resource consuming, in change it gets you a list of all keys consumed in written documents.
db.collection.createIndex( { "unikey": 1}, { unique: true } )
db.collection.insertMany( {[{"unikey": "$FieldA"},{"unikey": "$FieldB"}]}
db.collection.find({"unikey": 1})
I have the following schema:
var lessonSchema = mongoose.Schema({
_id: mongoose.Schema.Types.ObjectId,
name: String,
students: [{
_id: mongoose.Schema.Types.ObjectId,
attendance: {
type: Boolean,
default: false,
},
}],
});
The students array is an array of students who attended the particular lesson. I want to find a lesson using whether a particular user is present in the students array and then sent only that element of the students array which corresponds to the user making the request, along with all other fields as it is. For example, the query should return:
{
_id: 'objectid',
name: 'lesson-name'
students: [details of just the one student corresponding to req.user._id]
}
I tried using:
Lesson.find({'students._id': String(req.user._id)}, {"students.$": 1})
The query returns the document with just the id and the relevant element from the students array:
{
_id: 'objectid'
students: [details of the one student corresponding to req.user._id]
}
I tried using:
Lesson.find({'students._id': mongoose.Types.ObjectId(req.user._id)})
This returns the document with the details of all the students:
{
_id: 'objectid',
name: 'lesson-name'
students: [array containing details of all the students who attended the lesson]
}
How can I modify the query to return it the way I want?
You can return the name field by adding it to the projection object like this:
Lesson.find({ "students._id": String(req.user._id) }, { "name": 1, "students.$": 1 })
When you add a projection object (2nd parameter to find), the _id field is returned by default, plus whichever fields you set to 1.
Therefore, you were returning just the _id and the desired student but not the name field.
If you want to return all other fields and just limit the array to the matched item then you can make use of $slice in your projection:
Lesson.find({ "students._id": String(req.user._id) }, { "students.$": { $slice: 1 } })
I'm a new user of mongodb and I have a model like below. For update list data, I have to specify the element in an array. So I think I need to store a unique value for each element. Because list.name and list.price are variable data.
So are there any good ways to create an unique id in mongodb? Or should I create unique ids by myself?
{
name: 'AAA',
list: [
{name: 'HOGE', price: 10, id: 'XXXXXXXXXX'}, // way to add id
{name: 'FUGA', price: 12, id: 'YYYYYYYYYY'} // way to add id
]
}
Mongodb creates unique id only for documents. There is no better way for list or array elements. So, you should create Unique ids yourself.
Add keep in mind that, While updating your list use $addToSet.
For more information of $addToSet follow this documentation
use ObjectId() on your id field, so like..
db.test.update({name: "AAA"}, { $push: { list: {_id : ObjectId(), name: "dingles", price: 21} }});
reference: https://docs.mongodb.org/v3.0/reference/object-id/
whoever is seeing this in 2022, mongodb creates unique ids automatically we just have to provide schema for that particular array.
like,
_id : {
type: String
},
list: {
type: [{
Name : {
type: String
},
price : {
type: String
}
}]
}
this schema will generate auto id for all elements added into array
but below example will not create it.
_id : {
type: String
},
list: {
type: Array
}