I have an array of objects containing dates of when a hotel is available to book within Mongo. It looks something like this, using ISO Date formats as said here.
Here's what document looks like, trying to keep it short for the example.
available: [
{
"start":"2014-04-07T00:00:00.000000",
"end":"2014-04-08T00:00:00.000000"
},
{
"start":"2014-04-12T00:00:00.000000",
"end":"2014-04-15T00:00:00.000000"
},
{
"start":"2014-04-17T00:00:00.000000",
"end":"2014-04-22T00:00:00.000000"
},
]
Now, I need query two dates, check in date and check out date. If the dates are available, Mongo should return the document, otherwise it won't. Here are a few test cases:
2014-04-06 TO 2014-04-08 should NOT return.
2014-04-13 TO 2014-04-16 should NOT return.
2014-04-17 TO 2014-04-21 should return.
How would I go about forming this in to a Mongo query? Using $elemMatch looked like it would be a good start, but I don't know where to take it after that so all three examples I posted above work with the same query. Any help is appreciated.
db.collection.find({
"available": {
"$elemMatch": {
"start": { "$lte": new Date("2014-04-17") },
"end": { "$gte": new Date("2014-04-21") }
}
}
})
How about this command?
Well I actually hope your documents have real ISODates rather than what appears to be strings. When they do then the following query form matches as expected:
db.collection.find({
"available": {
"$elemMatch": {
"start": { "$gte": new Date("2014-04-17") },
"end": { "$gte": new Date("2014-04-21") }
}
}
})
Related
Here in provided aggregation pipeline, I need to compare a field which consists of previous date and time column called "HardStopDaysDate" with current date and time in mongo. I am using $$NOW itself here, but I am not able to see output, Can anyone please identify and help me what mistake I did.
db.customs.aggregate(
{
"$match": {
"Date": {
"$lt": "$$NOW"
},
"Status": {
"$ne": "Completed"
},
"$or": [
{
"Verification": null
},
{
"Verification": {
"$ne": 1
}
}
]
}
})
If I remove "$$NOW" then I am able to see result. But this comparison have to be done must and need to show desired result.
Here, I have to compare "Date" with current date and time so I am using "$$NOW". The query is working fine but not able to see any records. The filtered records have to be displayed which records are less than "Date".
Giving Sample records how "Date" is there in db.
[{"Date":2022-02-18 21:27:00}]
Can anyone please help me on this to get records while comparing with "$$NOW"
The core collection (other collections in the DB refer back to this one) in my DB contains 3 fields with date information which at this point is formatted as strings like MM/DD/YYYY. Further, there are a range of documents for which this field contains missing data, i.e. "". I populated this collection by running the mongoimport command on a JSON file.
My goal is to convert these date-fields into actual ISODate data types, so as to allow filtering the collection by dates. Further, I want MongoDB to know that empty strings indicate missing values. I have read quite widely on this, leading me to try a bunch of things:
Trying a forEach statement - This worked, but only for the very first document.
db.collection.find().forEach(function(element){
element.startDate = ISODate(element.startDate);
db.collection.save(element);
})
Using kind of a for-loop: this worked well, but stopped once it encountered a missing value (so it transformed about 11 values):
db.collection.update(
{
"startDate":{
"$type":"string"
}
},
[
{
"$set":{
"startDate":{
"$dateFromString":{
"dateString":"$startDate",
"format":"%m/%d/%Y"
}
}
}
}
]
)
So, both of these approaches kind of worked - but I don't know how to apply them to the entire collection. Further, I'd be interested in performing this task in the most efficient way possible. However, I only want to do this once - data that will be added in the future should hopefully be correctly formatted at the import stage.
db.collection.updateMany(
{
"$and": [
{ "startDate": { "$type": "string" } },
{ "startDate": { "$ne": "" } }
]
},
[
{
"$set": {
"startDate": {
"$dateFromString": {
"dateString": "$startDate",
"format": "%m/%d/%Y"
}
}
}
}
]
)
Filtering out empty string than doing the transformation will ignore documents that have empty string in date field.
In my mongodb table, I have 2 (relevant for this Q) columns: service, timestamp.
I want to query only rows with service=liveness and that those with timestamp of 12th Novermber 2020.
How can I do it, if timestamp field is of type Number (UNIX epoch number)..?
This is my query currently:
{ service: "liveness" }.
This is how the timestamp column looks like:
To query by two fields you only need this syntax:
db.collection.find({
"field1": yourField1Value,
"field2": yourField2Value
So, if your date is a Number instead of a Date you can try this query:
db.collection.find({
"service": "liveness",
"timestamp": 1600768437934
})
And should works. Example here.
Now, if the problem is parse 12th November 2020 to UNIX timestamp, then the easiest way is convert first the date in your app language.
Edit:
Also, I don't know if I've missunderstood your question but, here is another query.
db.collection.aggregate([
{
"$match": {
"service": "liveness",
}
},
{
"$project": {
"timestamp": {
"$toDate": "$timestamp"
}
}
},
{
"$match": {
"timestamp": {
"$gt": ISODate("1990-01-01"),
"$lt": ISODate("2060-01-01")
}
}
}
])
This query first match all documents with service as liveness, so the next stage is faster. Into $project the timestamp is parsed to Date so you can match again with your date.
Using $gt and $lt you can search by a whole day.
And also, if you can get the days into UNIX timestamp you can do this:
db.collection.find({
"service": "liveness",
"timestamp": {
"$gte": yourDay,
"$lt": nextrDay
}
})
Using $gte and $lt you ensure the query will find all values in the day.
I'm not quite understanding how fuzzy works in the $searchBeta stage of aggregation. I'm not getting the desired result that I want when I'm trying to implement full-text search on my backend. Full text search for MongoDB was released last year (2019), so there really aren't many tutorials and/or references to go by besides the documentation. I've read the documentation, but I'm still confused, so I would like some clarification.
Let's say I have these 5 documents in my db:
{
"name": "Lightning Bolt",
"set_name": "Masters 25"
},
{
"name": "Snapcaster Mage",
"set_name": "Modern Masters 2017"
},
{
"name": "Verdant Catacombs",
"set_name": "Modern Masters 2017"
},
{
"name": "Chain Lightning",
"set_name": "Battlebond"
},
{
"name": "Battle of Wits",
"set_name": "Magic 2013"
}
And this is my aggregation in MongoDB Compass:
db.cards.aggregate([
{
$searchBeta: {
search: { //search has been deprecated, but it works in MongoDB Compass; replace with 'text'
query: 'lightn',
path: ["name", "set_name"],
fuzzy: {
maxEdits: 1,
prefixLength: 2,
maxExpansion: 100
}
}
}
}
]);
What I'm expecting my result to be:
[
{
"name": "Lightning Bolt", //lightn is in 'Lightning'
"set_name": "Masters 25"
},
{
"name": "Chain Lightning", //lightn is in 'Lightning'
"set_name": "Battlebond"
}
]
What I actually get:
[] //empty array
I don't really understand why my result is empty, so it would be much appreciated if someone explained what I'm doing wrong.
What I think is happening:
db.cards.aggregate... is looking for documents in the "name" and "set_name" fields for words that have a max edit of one character variation from the "lightn" query. The documents that are in the cards collection contain edits that are greater than 2, and therefor your expected result is an empty array. "Fuzzy is used to find strings which are similar to the search term or terms"; used with maxEdits and prefixLength.
Have you tried the term operator with the wildcard option? I think the below aggregation would get you the results you were actually expecting.
e.g.
db.cards.aggregate([
{$searchBeta:
{"term":
{"path":
["name","set_name"],
"query": "l*h*",
"wildcard":true}
}}]).pretty()
You need to provide an index to use with your search query.
The index is basically the analyzer that your query will use to process your results regarding if you want to a full match of the text, or you want a partial match etc.
You can read more about Analyzers from here
In your case, an index based on STANDARD analyzer will help.
After you create your index your code, modified below, will work:
db.cards.aggregate([
{
$search:{
text: { //search has been deprecated, but it works in MongoDB Compass; replace with 'text'
index: 'index_name_for_analyzer (STANDARD in your case)'
query: 'lightn',
path: ["name"] //since you only want to search in one field
fuzzy: {
maxEdits: 1,
prefixLength: 2,
maxExpansion: 100
}
}
}
}
]);
While creating a query, I realized that the app I have inherited have a collection with a timestamp field containing either a string or an ISODate value.
So, this aggregation stage :
{
"$addFields": {
"timestamp": {
"$dateFromString": {
"dateString": "$activity.timestamp"
}
},
"minTimestamp": {
"$dateFromString": {
"dateString": "2016-01-01"
}
},
"maxTimestamp": {
"$dateFromString": {
"dateString": "2017-01-01"
}
}
}
}
produces the error : $dateFromString requires that 'dateString' be a string, found: date on certain documents.
Obviously, the logical answer would be to convert all field values into ISODate, however this behemoth of a system seem to save inconsistent values being set for that field, and it is not possible for me to guarantee the type in advance.
Is there a way to conditionally convert the field to ISODate?
The app is running on MongoDB 3.6.4.
If some of the fields are actually BSON Dates, then you probably want to leave them alone and output them as is. For this you can use $type along with a $cond expression:
{ "$addFields": {
"timestamp": {
"$cond": {
"if": { "$eq": [{ "$type": "$activity.timestamp" }, "string" ] },
"then": {
"$dateFromString": {
"dateString": "$activity.timestamp"
}
},
"else": "$activity.timestamp"
}
}
}}
That's okay from MongoDB 3.4 and upwards which is when $type was added.
Noting for users of MongoDB 4.0 and above, the $convert operator actually has built in error handling branching:
{ "$addFields": {
"timestamp": {
"$convert": {
"input": "$activity.timestamp",
"to": "date",
"onError": "Neither date or string"
}
}
}}
The onError can be any expression and is returned in the instances where the conversion was invalid. Finding a BSON Date is not actually an error and errors would only occur for an invalid numeric or string value or different type which did not support conversion.
If you know for certain that the data is always either a BSON Date or a valid string for conversion then there is the $toDate helper which is basically a wrapper for $convert without the onError handling:
{ "$addFields": {
"timestamp": { "$toDate": "$activity.timestamp" }
}}
So some data scrubbing and/or query conditions can often be combined with that for a more streamlined coding experience.
A note on $dateToString usage
Within the question the $dateToString is also being used to convert "static values" from strings into BSON Date. This is not a good idea.
Running functional code within a server expression which is more cleanly expressed in natural language code is not and never really was a good practice. As part of the general MongoDB philosophy, the parts that really should and can be expressed in that language should be done so that way.
For JavaScript simple Date objects are serialized as BSON Date on submission to the server anyway:
"minTimestamp": new Date("2016-01-01")
Since the value of the "string" is external then it does not need server side expression to manipulate it. Just like issuing a query you cast such types before you submit to the server and not afterwards.
The same concept is true of all language implementations, as all languages have a "Date" type which the implemented driver understands and correctly serializes as a BSON Date anyway.
Conversion
With all of that said, the general "best practice" here is of course to actually convert the data. "Behemoth" or not, it's really only making matters worse to rely on conversion of the data at run time. This is even more so if your actual intention in such run-time conversion is to use the BSON Dates in further output processing, rather than just a pretty output.
Mileage varies on which approach works the best, but the basics are to either iterate the collection and update values in place, and the $type "query" operator can help with selection here:
// Presuming date strings in"yyyy-mm-dd" format
var batch = [];
db.collection.find({ "activity.timestamp": { "$type": "string" } }).forEach(d => {
batch.push({
"updateOne": {
"filter": { "_id": d._id },
"update": { "$set": { "activity.timestamp": new Date(d.activity.timestamp) } }
}
});
if (batch.length >= 1000) {
db.collection.bulkWrite(batch);
batch = [];
}
})
if (batch.length > 0) {
db.collection.bulkWrite(batch);
batch = [];
}
Or run the aggregation with $out to a new collection if constraints allow this:
{ "$addFields": {
"activity": {
"timestamp": {
"$cond": {
"if": { "$eq": [{ "$type": "$activity.timestamp" }, "string" ] },
"then": {
"$dateFromString": {
"dateString": "$activity.timestamp"
}
},
"else": "$activity.timestamp"
}
}
}
}},
{ "$out": "newcollection" }
Any of the above aggregation methods shown can be used here where they are supported, but just showing this in example. Note also the $addFields does allow the nested object syntax since output is "merged" into the existing document structure.
Even on a production system you could always output to a new collection and then drop and rename with minimal downtime. The main constraint here would actually be index rebuilding, which would take significantly longer than a collection rename.