I am new to MongoDB and I am trying to run an aggregate function, to sum up all the current bid price of all the category Toyota. My code seems to look ok but I am getting errors
{
"_id" : ObjectId("6226613eeb82220b6ccbe598"),
"user" : 12,
"name" : "Camry",
"category" : "Toyota",
"description" : "Simple but smooth",
"closing date" : "12/06/22",
"starting bid price" : "180.00",
"Current bid price" : "960.00",
"auction status" : "open"
}
{
"_id" : ObjectId("62266282eb82220b6ccbe59b"),
"user" : 10,
"name" : "Prado",
"category" : "Toyota",
"description" : "Rugged and High",
"closing date" : "12/04/22",
"starting bid price" : "250.00",
"Current bid price" : "450.00",
"auction status" : "open",
}
my code first fetches the Toyota category from the json then it groups based on the category and it sums up the Current bid price
db.auction.aggregate([ { $match: { "category", "Toyota"}},
{$group: { _id:"$category",total:{$sum: "$Current bid price"}}} ])
My error
SyntaxError: missing : after property id :
#(shell):1:45
I have seen the error I needed to add colon in-between category and toyota instead of a comma
db.auction.aggregate([ { $match: { "category": "Toyota"}},
{$group: { _id:"$category",total:{$sum: "$Current bid price"}}} ])
Related
I am having records stored in mongo-db collection for customer and there transactions with below format:
{
"_id" : ObjectId("59b6992a0b54c9c4a5434088"),
"Results" : {
"id" : "2139623696",
"member_joined_date" : ISODate("2010-07-07T00:00:00.000+0000"),
"account_activation_date" : ISODate("2010-07-07T00:00:00.000+0000"),
"family_name" : "XYZ",
"given_name" : "KOKI HOI",
"gender" : "Female",
"dob" : ISODate("1967-07-20T00:00:00.000+0000"),
"preflanguage" : "en-GB",
"title" : "MR",
"contact_no" : "60193551626",
"email" : "abc123#xmail.com",
"street1" : "address line 1",
"street2" : "address line 2",
"street3" : "address line 3",
"zipcd" : "123456",
"city" : "xyz",
"countrycd" : "Malaysia",
"Transaction" : [
{
"txncd" : "411",
"txndate" : ISODate("2017-08-02 00:00:00.000000"),
"prcs_date" : ISODate("2017-08-02 00:00:00.000000"),
"txn_descp" : "Some MALL : SHOP & FLY FREE",
"merchant_id" : "6587867dsfd",
"orig_pts" : "0.00000",
"text" : "Some text"
}
]
}
I want to create index on fields "txn_descp", "txndate", "member_joined_date", "gender", "dob" for faster access. Can some one help me in creating index for this document? Will appreciate any kind of help and suggestions.
While creating the index there are a few things to keep in mind.
Always create the index for the queries you use.
Go for compound indexes whenever possible.
First field in the index should be the one with the minimum possible values.Ie, if there is an index with gender and DOB as keys, It is better to have {gender:1,dob:1}
In the following model a product is owned by a customer. and cannot be ordered by other customers. So I know that in an order by customer 1 there can only be products owned by customer one.
To give you an idea here is a simple version of the data model:
Orders:
{
'customer' : 1
'products' : [
{'productId' : 'a'},
{'productId' : 'b'}
]
}
Products:
{
'id' : 'a'
'name' : 'somename'
'customer' : 1
}
I need to find orders that contain certain products. I know the product id and customer id. I'm free to add/change indexes on my database.
Now my question is. Is it faster to just add a single field index on the product id's and query only using that ID. Or should I go for a compound index with customer and product id?
I'm not sure if this matters, but in my real model the list of products is actually a list of objects which have an amount and a dbref to the product. And the customer is also a dbref.
Here is a full order object:
{
"_id" : 0,
"_class" : "nl.pfa.myprintforce.models.Order",
"orderNumber" : "e35f1fa8-b4c4-4d53-89c9-66abe94a3553",
"status" : "ERROR",
"created" : ISODate("2017-03-30T11:50:50.292Z"),
"finished" : false,
"orderTime" : ISODate("2017-01-12T12:50:50.292Z"),
"expectedDelivery" : ISODate("2017-03-30T11:50:50.292Z"),
"totalItems" : 19,
"orderItems" : [
{
"amount" : 4,
"product" : {
"$ref" : "product",
"$id" : NumberLong(16)
}
},
{
"amount" : 7,
"product" : {
"$ref" : "product",
"$id" : NumberLong(26)
}
},
{
"amount" : 8,
"product" : {
"$ref" : "product",
"$id" : NumberLong(7)
}
}
],
"stateList" : [
{
"timestamp" : ISODate("2017-03-28T11:50:50.074Z"),
"status" : "NEW",
"message" : ""
},
{
"timestamp" : ISODate("2017-03-29T11:50:50.075Z"),
"status" : "IN_PRODUCTION",
"message" : ""
},
{
"timestamp" : ISODate("2017-03-30T11:50:50.075Z"),
"status" : "ERROR",
"message" : "Something went wrong"
}
],
"customer" : {
"$ref" : "customer",
"$id" : ObjectId("58dcf11a71571a24c475c044")
}
}
When I have the following indexes:
1: {"customer" : 1, "orderItems.product" : 1}
2: {"orderItems.product" : 1}
both count queries (I use count to forcefully find all documents without the network transfer):
a: db.getCollection('order').find({
'orderItems.product' : DBRef('product',113)
}).count()
b: db.getCollection('order').find({
'customer' : DBRef('customer',ObjectId("58de009671571a07540a51d5")),
'orderItems.product' : DBRef('product',113)
}).count()
Run with the same time of ~0.007 seconds on a set of 200k.
When I add 1000k record for a different customer (and different products) it does not effect the time at all.
an extended explain shows that:
query 1 just uses index 2.
query 2 uses index 2 but also considered index 1. Perhaps index intersection is used here?
Because if I drop index 1 the results are:
Query a: 0.007 seconds
Query b: 0.035 seconds (5x as long!)
So my conclusion is that with the right indexing both methods work about as fast. However, if you do not need the compound index for anything else it's just a waste of space & write speed.
So: single field index is better in my case.
I am trying to rename my ID field in the project phase but I have an error message. The $match and $sort phases work fine. Here are the details:
db.complaints.aggregate([
{$match:{$text:{$search:"\"loan\""}}},
{$group:{"_id":{Year:{$substr: ["$received", 0, 4]}}, "loan":{$sum:1}}},
{$sort:{_id:-1}},
{$project:{_id:0, "Year":"_id.Year", "loan":1}}
])
Here is my schema:
> db.complaints.findOne()
{
"_id" : ObjectId("55e5990d991312e2c9b266e3"),
"complaintID" : 1388734,
"product" : "mortgage",
"subProduct" : "conventional adjustable mortgage (arm)",
"issue" : "loan servicing, payments, escrow account",
"subIssue" : "",
"state" : "va",
"ZIP" : 22204,
"submitted" : "web",
"received" : "2015-05-22",
"sent" : "2015-05-22",
"company" : "green tree servicing, llc",
"response" : "closed with explanation",
"timely" : "yes",
"disputed" : ""
}
My collection has the following simplified (booking) schema:
{
name: "room 1",
from: ISODate("2014-06-10T12:00:00Z"),
to: ISODate("2014-06-14T12:00:00Z")
},
{
name: "room 1",
from: ISODate("2014-06-25T12:00:00Z"),
to: ISODate("2014-06-27T12:00:00Z")
},
{
name: "room 2",
from: ISODate("2014-06-12T12:00:00Z"),
to: ISODate("2014-06-26T12:00:00Z")
}
I'd like to query, if a room is available in a given range. For example I'd like to know if
room 1 is available FROM 2014-06-11 TO 2014-06-13
room 1 is available FROM 2014-06-13 TO 2014-06-26
room 1 is available FROM 2014-06-15 TO 2014-06-18
Ok let's break this down, there are 4 ranges of booking conflicts:
start of another booking is before this start but the end is also before the end of this booking
start of another booking is after this start and the end is after this end
start of another booking is after this start and the end is before this end
start of other booking is before this start and after the end of this end
So you are looking for a query that can find all those ranges if they exist.
Now I set up this data:
> db.rooms.find()
{ "_id" : ObjectId("53ad206e1d8f2d8351182830"), "id" : 1, "from" : ISODate("2014-06-26T00:00:00Z"), "to" : ISODate("2014-06-28T00:00:00Z") }
{ "_id" : ObjectId("53ad276f1d8f2d8351182831"), "id" : 1, "from" : ISODate("2014-06-24T00:00:00Z"), "to" : ISODate("2014-07-01T00:00:00Z") }
{ "_id" : ObjectId("53ad28ad1d8f2d8351182832"), "id" : 1, "from" : ISODate("2014-06-20T00:00:00Z"), "to" : ISODate("2014-06-28T00:00:00Z") }
{ "_id" : ObjectId("53ad28c61d8f2d8351182833"), "id" : 1, "from" : ISODate("2014-06-20T00:00:00Z"), "to" : ISODate("2014-07-03T00:00:00Z") }
{ "_id" : ObjectId("53ad29971d8f2d8351182834"), "id" : 1, "from" : ISODate("2014-06-20T00:00:00Z"), "to" : ISODate("2014-06-21T00:00:00Z") }
(the 5th range being a tester to make sure the query doesn't return random results)
And then I ran:
> db.rooms.find({from:{$lte: ISODate('2014-06-30T00:00:00.000Z')}, to:{$gte: ISODate('2014-06-23T00:00:00.000Z')}})
{ "_id" : ObjectId("53ad206e1d8f2d8351182830"), "id" : 1, "from" : ISODate("2014-06-26T00:00:00Z"), "to" : ISODate("2014-06-28T00:00:00Z") }
{ "_id" : ObjectId("53ad276f1d8f2d8351182831"), "id" : 1, "from" : ISODate("2014-06-24T00:00:00Z"), "to" : ISODate("2014-07-01T00:00:00Z") }
{ "_id" : ObjectId("53ad28ad1d8f2d8351182832"), "id" : 1, "from" : ISODate("2014-06-20T00:00:00Z"), "to" : ISODate("2014-06-28T00:00:00Z") }
{ "_id" : ObjectId("53ad28c61d8f2d8351182833"), "id" : 1, "from" : ISODate("2014-06-20T00:00:00Z"), "to" : ISODate("2014-07-03T00:00:00Z") }
If that query returns there are bookings and as such you cannot insert.
That should cover all bases I think, I am a little sleep deprived so I might be wrong.
Considering your data is showing the dates on which the room is booked and you want to query for certain date ranges then you have to make sure that both these dates(for that you want to check availability for) lies outside the interval of "from" and "to" of the booked rooms, for that you have to run a query where both your input dates are either less than the "from" date or both are greater than the "to" date,this is how you do it
db.room.find({
$or:[
{$and : [{from:{$gt:new Date("2014-06-11T00:00:00.000Z")}},{to:{$gt:new Date("2014-06-13T00:00:00.000Z")}}]},
{$and : [{from:{$lt:new Date("2014-06-11T00:00:00.000Z")}},{to:{$lt:new Date("2014-06-13T00:00:00.000Z")}}]}
]
})
if you want to count the room just add .count() to the above query and if you want to query for a specific room then just add "name":"roomname" in the above query.
Hope it helps.
You can create range queries to check if from and to are within the searching dates. In addition the third use case is when the search dates cover one more documents' from/to interval.
My collection on which I performed tests looks like this:
> db.booking.find()
{ "_id" : ObjectId("53aa8d8cdfae7e8ccdd4b49d"), "name" : "room 1", "from" : ISODate("2014-06-10T12:00:00Z"), "to" : ISODate("2014-06-14T12:00:00Z") }
{ "_id" : ObjectId("53aa8da1dfae7e8ccdd4b49e"), "name" : "room 1", "from" : ISODate("2014-06-25T12:00:00Z"), "to" : ISODate("2014-06-27T12:00:00Z") }
{ "_id" : ObjectId("53aab77fe0cd5e6e1c56c08c"), "name" : "room 2", "from" : ISODate("2014-06-12T12:00:00Z"), "to" : ISODate("2014-06-26T12:00:00Z") }
{ "_id" : ObjectId("53ad1107d20cc7b2fe2df520"), "name" : "room 1", "from" : ISODate("2014-07-05T12:00:00Z"), "to" : ISODate("2014-07-07T12:00:00Z") }
{ "_id" : ObjectId("23ad1107d20cc7b2fe2df520"), "name" : "room 1", "from" : ISODate("2014-06-12T12:00:00Z"), "to" : ISODate("2014-06-26T12:00:00Z") }
>
Here are queries you can execute to check if a room is available (the return value of 0 means the room is available):
room 1 is available FROM 2014-06-11 TO 2014-06-13
db.booking.count({
name:"room 1",
$or:[
{
from:{$lte:new Date("2014-06-11T00:00:00.000Z")},
to:{$gte:new Date("2014-06-11T00:00:00.000Z")}
},
{
from:{$lte:new Date("2014-06-13T00:00:00.000Z")},
to:{$gte:new Date("2014-06-13T00:00:00.000Z")}
},
{
from:{$gte:new Date("2014-06-11T00:00:00.000Z")},
to:{$lte:new Date("2014-06-13T00:00:00.000Z")}
}
]})
room 1 is available FROM 2014-06-13 TO 2014-06-26
db.booking.count({
name:"room 1",
$or:[
{
from:{$lte:new Date("2014-06-13T00:00:00.000Z")},
to:{$gte:new Date("2014-06-13T00:00:00.000Z")}
},
{
from:{$lte:new Date("2014-06-26T00:00:00.000Z")},
to:{$gte:new Date("2014-06-26T00:00:00.000Z")}
},
{
from:{$gte:new Date("2014-06-13T00:00:00.000Z")},
to:{$lte:new Date("2014-06-26T00:00:00.000Z")}
}
]})
room 1 is available FROM 2014-06-15 TO 2014-06-18
db.booking.count({
name:"room 1",
$or:[
{
from:{$lte:new Date("2014-06-15T00:00:00.000Z")},
to:{$gte:new Date("2014-06-15T00:00:00.000Z")}
},
{
from:{$lte:new Date("2014-06-18T00:00:00.000Z")},
to:{$gte:new Date("2014-06-18T00:00:00.000Z")}
},
{
from:{$gte:new Date("2014-06-15T00:00:00.000Z")},
to:{$lte:new Date("2014-06-18T00:00:00.000Z")}
}
]})
room 1 is available FROM 2014-07-02 TO 2014-07-09
db.booking.count({
name:"room 1",
$or:[
{
from:{$lte:new Date("2014-07-02T00:00:00.000Z")},
to:{$gte:new Date("2014-07-02T00:00:00.000Z")}
},
{
from:{$lte:new Date("2014-07-09T00:00:00.000Z")},
to:{$gte:new Date("2014-07-09T00:00:00.000Z")}
},
{
from:{$gte:new Date("2014-07-02T00:00:00.000Z")},
to:{$lte:new Date("2014-07-09T00:00:00.000Z")}
}
]})
I need some ideas/tips for this. Here is a sample document I am storing:
{
"_id" : new BinData(0, "C3hBhRCZ5ZFizqbO1hxwrA=="),
"gId" : 237,
"name" : "WEATHER STATION",
"mId" : 341457,
"MAC" : "00:00:00:00:00:01",
"dt" : new Date("Fri, 24 Feb 2012 13:59:02 GMT -05:00"),
"hw" : [{
"tag" : "Weather Sensors",
"snrs" : [{
"_id" : NumberLong(7),
"sdn" : "Wind Speed"
}, {
"_id" : NumberLong(24),
"sdn" : "Wind Gust"
}, {
"_id" : NumberLong(28),
"sdn" : "Wind Direction"
}, {
"_id" : NumberLong(31),
"sdn" : "Rainfall Amount"
}, {
"_id" : NumberLong(33),
"sdn" : "Rainfall Peak Amount"
}, {
"_id" : NumberLong(38),
"sdn" : "Barometric Pressure"
}],
"_id" : 1
}]
}
What I am currently doing is using the C# driver and performing a .Save() to my collection to get upsert, however, what I want is kinda a hybrid approach I guess. Here are the distinct operations I need to be able to perform:
Upsert entire document if it does not exist
Update the dt field with a new timestamp if the document does exist
For the hw field, I need several things here. If hw._id exists, update its tag field as well as handling the snrs field by either updating existing entries so the sdn value is updated or adding entirely new entires when _id does not exist
Nothing should ever be removed from the hw array and nothing should ever be removed from the snrs array.
A standard upsert does not appear to get me what I am after, so I am looking for the best way to do what I need with as few roundtrips to the server as possible. I am thinking some of the $ Operators may be what I am needing here, but just need some thoughts on how best to approach this.
The gist of what I am doing here is keeping an accumulating, historical document of snrs entries with the immediate current value as well as retaining any historical entries in the array even though they are no longer "alive", being reported, etc. This allows future reporting on things that no longer exist in current time, but were at some point in the past. _id values are application-generated, globally unique across all documents, and never change after initial creation. For example, last week "Wind Speed" was being reported, but this week it is not. It's _id value, however, will not change if "Wind Speed" starts reporting again. Follow?
Clarifications or more detail can be provided if needed.
Thanks.
By changing the structure of your document from embedded arrays to subdocuments key'ed by the _ids you can do this.
e.g.
{
"MAC" : "00:00:00:00:00:01",
"_id" : 1,
"dt" : ISODate("2012-02-24T18:59:02Z"),
"gId" : 237,
"hw" : {
"1" : {
"snrs" : {
"1" : "Wind Speed",
"2" : "Wind Gust"
},
"tag" : "Weather Sensors"
}
},
"mId" : 341457,
"name" : "WEATHER STATION 1"
}
I created the above document by the following upsert
db.foo.update(
{_id:1},
{
$set: {
"gId" : 237,
"name" : "WEATHER STATION 1",
"mId" : 341457,
"MAC" : "00:00:00:00:00:01",
"dt" : new Date("Fri, 24 Feb 2012 13:59:02 GMT -05:00"),
"hw.1.tag" : "Weather Sensors",
"hw.1.snrs.1" : "Wind Speed",
"hw.1.snrs.2" : "Wind Gust"
}
},
true
)
Now when I run
db.foo.update(
{_id:1},
{
$set: {
"dt" : new Date(),
"hw.2.snrs.1" : "Rainfall Amount"
}
},
true
)
I get
{
"MAC" : "00:00:00:00:00:01",
"_id" : 1,
"dt" : ISODate("2012-03-07T05:14:31.881Z"),
"gId" : 237,
"hw" : {
"1" : {
"snrs" : {
"1" : "Wind Speed",
"2" : "Wind Gust"
},
"tag" : "Weather Sensors"
},
"2" : {
"snrs" : {
"1" : "Rainfall Amount"
}
}
},
"mId" : 341457,
"name" : "WEATHER STATION 1"
}