I'm actually creating an autocomplete website bar using express js & mongodb 4.2.7, and using lat & lon to avoid using geocoding api.
Here is the format of my db:
{
"_id" : ObjectId("5eea03e9a7891b6d701df571"),
"id_ban_position" : "ban-position-4c882de48d894fc49ed9be76f7631d6d",
"id_ban_adresse" : "ban-housenumber-78a2651ce382476ca9c8c8fcbcbaa539",
"cle_interop" : "01001_A028_5307",
"id_ban_group" : "ban-group-37c7c3f2b61440b48bca7d8fad56055e",
"id_fantoir" : "01001A028",
"numero" : 5307,
"suffixe" : "",
"nom_voie" : "Lotissement les Lilas",
"code_postal" : 1400,
"nom_commune" : "L'Abergement-Clémenciat",
"code_insee" : 1001,
"nom_complementaire" : "",
"x" : 848436.205131189,
"y" : 6562595.33916435,
"lon" : 4.923279,
"lat" : 46.146903,
"typ_loc" : "parcel",
"source" : "dgfip",
"date_der_maj" : "2019-02-12"
}
as you can see, i got lot of fields that are not necessary for the purpose of my website, and most of all, i don't need to get one row for each number in a street, one unique name of street is enough. So i decided to supress duplicate street name ('nom_voie') and town name ('nom_commune') with aggregate as follow:
db.Addresses.aggregate(
[ {
$group: {
_id: {voie: "$nom_voie", commune: "$nom_commune"},
doc: {$first: "$$ROOT"}
}
},
{$replaceRoot: {newRoot: "$doc"}},
{$out: 'UniqueIds'}
],
{allowDiskUse: true }
);
The problem is that this utilisation mooved a lot of values from one field to an other and made my db absolutely unusable.
{
"_id" : ObjectId("5eea03e9a7891b6d701df571"),
"nom_voie" : "-",
"code_postal" : 1400,
"nom_commune" : "Lotissement les Lilas",
"nom_complementaire" : "",
"lon" : 6562595.33916435,
"lat" : 848436.205131189,
"field19" : "dgfip",
...
}
As you can see, the value of "nom_voie" is now in "nom_commune", "x" value is in "lat", and "y" value is in "lon" and finally the "nom_voie" value is now "-", and i got new fields replacing others ("field19" for "source")...
Am i using aggregate with a wrong option?
I got 47 Millions entry, is this creating some issues?
Thanks to all of you for your time, even if you just reading this!
EDIT :
After few tries and some search i found out that it is the aggregate function that create some problems, but i still don't understand why, if somebody got some hints, i just edited the doc so it's more understandable and readable!
(I thought before that it was a $unset problem in a function)
Related
Problem
I have a collection hotelreviews_collection containing 1 million rows (documents) of reviews with various metadata. I would like to group by the Hotel_Name field, count the number of times this hotel has showed up, but also get the fields "lat", "lng" and "Average_Score" with my query. The three extra rows are the same for each Hotel_Name.
I am doing the queries in R using the mongolite library connected to a local MongoDB.
My Attempt
I have gotten to retrieving the Hotel_Names and counting their appearances using the code below, but cannot for the life of me get the other fields to work.
Current Code
overviewData <- M_CONNECTION$aggregate('[{"$group":{"_id":"$Hotel_Name", "count": {"$sum":1}, "average":{"$avg":"$distance"}}}]',
options = '{"allowDiskUse":true}')
I am completely lost on this, any and all help would be greatly appreciated.
I have solved my issue using the following code.
db.getCollection("hotelreviews_collection").aggregate(
[
{
"$group" : {
"_id" : {
"Hotel_Name" : "$Hotel_Name",
"lat" : "$lat",
"lng" : "$lng",
"Average_Score" : "$Average_Score"
},
"COUNT(Hotel_Name)" : {
"$sum" : NumberInt(1)
}
}
},
{
"$project" : {
"Hotel_Name" : "$_id.Hotel_Name",
"lat" : "$_id.lat",
"lng" : "$_id.lng",
"Average_Score" : "$_id.Average_Score",
"COUNT(Hotel_Name)" : "$COUNT(Hotel_Name)",
"_id" : NumberInt(0)
}
}
]
)
> db.movmodels.findOne()
{
"_id" : ObjectId("55320b0e0e9e0d9d0540593c"),
"username" : "punk",
"favMovies" : [
{
"alternate_ids" : {
"imdb" : "0137523"
},
"abridged_cast" : [
{
"characters" : [
"Tyler"
],
"id" : "162652627",
"name" : "Brad Pitt"
},
{
"characters" : [
"Narrator"
],
"id" : "162660884",
"name" : "Edward Norton"
},
{
"characters" : [
"Robert"
],
"id" : "162676383",
"name" : "Meat Loaf"
},
{
"characters" : [
"Angel Face"
],
"id" : "162653925",
"name" : "Jared Leto"
},
{
"characters" : [
"Boss"
],
"id" : "770706064",
"name" : "Zach Grenier"
}
],
"synopsis" : "",
"ratings" : {
"audience_score" : 96,
"audience_rating" : "Upright",
"critics_score" : 80,
"critics_rating" : "Certified Fresh"
},
"release_dates" : {
"dvd" : "2000-06-06",
"theater" : "1999-10-15"
},
"critics_consensus" : "",
"runtime" : 139,
"mpaa_rating" : "R",
"year" : 1999,
"title" : "Fight Club",
**"id" : "13153"**
}
],
"__v" : 0
}
This is my data in mongodb.
As I am new to mongodb I wanted to know query to get movie with a particular id.
The query that I tried is. I need to get the movie based on id so that I can remove it from my database
db.movmodels.findOne({username:"punk"},{favMovies:{id:13153}})
but this gives me error.
2015-04-18T05:41:26.221-0400 E QUERY Error: error: {
"$err" : "Can't canonicalize query: BadValue ported projection option: favMovies: { id: 13153.0 }",
"code" : 17287
}
at Error (<anonymous>)
at DBQuery.next (src/mongo/shell/query.js:259:15)
at DBCollection.findOne (src/mongo/shell/collection.js:188:22)
at (shell):1:14 at src/mongo/shell/query.js:259
There are several problems with your query:
The second parameter to find() is a projection, not part of the query. What you want is to supply one document for the query that has two properties: {"username" : "punk", favMovies : { ... } }
However, you also don't want to compare the entire sub-document favMovies, but you only want to match on one of its properties, the id, which requires to 'reach into the object' using the dot operator: {username:"punk", "favMovies.id" : 13153}.
However, that will probably not work yet, because 13153 is not the same as "13153", the latter being a string while the former is a number in JSON.
db.movmodels.findOne({username:"punk", "favMovies.id" : "13153"})
Keep in mind, however, that this will find the entire document for the user named "punk". I'm not sure what exactly your data structure should look like, but it appears you'll have to $pull the movie from the user. In general, I'd say you're embedding too much data into the user, but that's hard to tell without knowing the exact use case.
Here you go:
If you just wanted to get first user who has this fav movie:
db.movmodels.findOne({"favMovies.id": 13153});
And, if you want to know if that user has that movie as favorite.
db.movmodels.findOne({"favMovies.id": 13153, username:"punk"});
Second argument in the findOne is used to only return particular field.
You can use also $elemMatch projection operator (not to be confused with the $elemMatch query operator)
db.movmodels.find({username:"punk"},{favMovies:{$elemMatch:{id:"13153"}}});
`
If you want to find a movie that has another movie (with id 13153) in 'favMovies' array, then write the query as below:
db.movmodels.findOne({username:"punk",'favMovies.id':13153})
And if you want to find a movie with _id 55320b0e0e9e0d9d0540593cwrite the following query:
db.movmodels.findOne({username:"punk",'_id':ObjectId("55320b0e0e9e0d9d0540593c")})
Let me start by saying I'm sorry if this has been answered, but I can't get other questions on this site to fit my needs and, more importantly, work.
I have the below example document, with a subdocument of 'address':
{
"_id" : ObjectId("....")
,"addresses" :
[{
"start" : ISODate("1973-07-10T00:11:51.111Z")
,"value" : "123 long road"
}]
}
What I need to do is to close the existing address record with an end attribute, and add a new line for the new address with a new start and value attribute. Eventually, I'll need to do this again so the code needs to update the subdocument record where end does not exist.
The below code does not work, but it's about as far as I can get:
db.sites.update(
{"_id" : ObjectId("....")
, "addresses.end" : {"$exists" : false}}
,{"$set": {"addresses.$.end" : "fdsa"}});
This gives the error:
Cannot apply the positional operator without a corresponding query field containing an array.
Can anyone point me in the right direction?
Juste replace in your query "addresses.end" : {"$exists" : false} with:
addresses: {$elemMatch: {end: {$exists: false}}}
Your address field is poorly defined. you need make it a subdocument or an array of subdocuments. ie {
"_id" : ObjectId("....")
,"addresses" :
[
{
"start" : ISODate("1973-07-10T00:11:51.111Z")
,"value" : "123 long road"
}
]
}
your query should then work!
I think that the Query should be more specific
**Updated **
db.sites.update ( {"_id" : ObjectId("...."), addresses: { "$elemMatch" : { end:{$exists : false}} } }, {"$set": {"addresses.$.end" : "fdsa"}});
db.sites.find()
results:
{
"_id" : ObjectId("53df93da560b7815e1237934"),
"addresses" : [
{
"start" : ISODate("1973-07-10T00:11:51.111Z"),
"value" : "123 long road",
"end" : "fdsa"
}
]
}
but you can update only one
Take a look http://docs.mongodb.org/manual/reference/operator/projection/positional/#proj.S
You can t update more element in Array https://jira.mongodb.org/browse/SERVER-1243
I have two collection as bellow products has reference of user. i search product by name & in return i want combine output of product and user using map reduce method
user collection
{
"_id" : ObjectId("52ac5dd1fb670c2007000000"),
"company" : {
"about" : "This is textile machinery dealer",
"contactAddress" : [{
"address" : "abcd",
"city" : "52ac4bc6fb670c1007000000",
"zipcode" : "39as46as80"
},{
"address" : "abcd",
"city" : "52ac4bc6fb670c1007000000",
"zipcode" : "39as46as80"
}],
"fax" : "58784868",
"mainProducts" : "ads,asd,asd",
"mobileNumber" : "9537236588",
"name" : "krishna steels",
}
"user" : ObjectId("52ac4eb7fb670c0c07000000")
}
product colletion
{
"_id" : ObjectId("52ac5722fb670cf806000002"),
"category" : "52a2a9cc48a508b80e00001d",
"deliveryTime" : "10 days after received the ",
"price" : {
"minPrice" : "2000",
"maxPrice" : "3000",
"perUnit" : "5288ac6f7c104203e0976851",
"currency" : "INR"
},
"productName" : "New Mobile Solar Charger with Carabiner",
"rejectReason" : "",
"status" : 1,
"user" : ObjectId("52ac4eb7fb670c0c07000000")
}
This cannot be done. Mongo support Map Reduce only on one collection. You could try to fetch and merge in a java collection. Couple of days back I solved a similar problem using java collection.
Click to see similar response about joins and multi collection not supported in mongo.
This can be done using two map reduces.
You run your first MR and then you reduce out the second MR onto the results of the first.
You shouldn't do this though. JOINs are not designed to be done through MR, in fact it sounds like you are trying to do this MR with inline output which in itself is a very bad idea.
MRs are not designed to run inline to the application.
You would be better off doing the JOIN else where.
My first adventure into Mongo. Please save me some time by answering the following. This is the schema.
"_id" : 1,
"FullName" : "Full Name",
"Email" : "email#email.com",
"FacebookId" : NumberLong(0),
"LastModified" : ISODate("2012-04-11T09:26:10.955Z"),
"Connections" : [{
"_id" : 7,
"FullName" : "Fuller name",
"Email" : "connections#email.com",
"FacebookId" : NumberLong(0),
"LastModified" : ISODate("0001-01-01T00:00:00Z")
},
....
Given an id of a single top user, i'd like to return all of the Emails in the Connections array, and preferably, just the emails. What's the querystring? Much obliged!
You can't get only values from the sub-objects in MongoDB.
If you do a query like this:
db.test.find({"_id": 1}, {"Connections.Email":1});
you will get this kind of response:
{
"_id": 1,
"Connections" : [ {"Email":"connections#email.com"},
{"Email":"foo#example.com"} ]
}
This is the closest you can get with a simple query and field selection from MongoDB.
You can then filter out the e-mails values in your code with a simple foreach.