I have a collection in the form as below:
UserName | Device | Device Rights
I have multiple documents for each user with different devices.
I am trying to aggregate and store the data grouped by user in the collection. To get output as below:
UserName | Device | Device Rights| Device 2 | Device 2 Rights...
Using the below code in Pymongo:
cur = data1_dbh.access_data
cur.aggregate=([
{'$group': {
'_id': "$username",
'accessdetails':{'$push': {'device':"$device_id", 'accesstype':"$access_type"}},
}}
])
x=cur.aggregate
for i in x:
print(i)
But I get a single line of code as output. What am i missing..
{'$group': {'_id': '$username', 'accessdetails': {'$push': {'device': '$device_id', 'accesstype': '$access_type'}}}}
Related
New to mongo and I'm struggling to write update mongo query with multiple where clause
Lets say I have a employee list
name |emp_id | old phone number | new phone number
Steve |123 | 123-456-7896 | 801-123-4567
John |456 | 123-654-9878 | 702-123-4567
Steve |789 | 789-123-7890 | 504-123-4567
I would like to write a mongo query essentially saying
Update to new phone number where both name and emp_id matches. The document has about 200 entries.
Since you're having multiple entires to be updated, you can have something like this :
When you say you've a list - Considering its in a file or somewhere to be processed through code then try to read data into an array :
let dataArray = [{name: Steve, emp_id: 123, new_phone_number :801-123-4567},{name: John, emp_id: 456, new_phone_number :702-123-4567},....]
db.getCollection('YourCollection').bulkWrite(
dataArray.map((each) =>
({
updateOne: {
filter: { 'name': each.name, 'emp_id': each.emp_id },
update: { $set: { phone_number: each.new_phone_number } }
}
})
)
)
So here we're considering phone_number as your field to be updated, We didn't consider maintaining two fields old_phone_number & new_phone_number in the documents.
Or in case if you wanted to have two phones numbers in the document then:
db.getCollection('YourCollection').bulkWrite(
dataArray.map((each) =>
({
updateOne: {
filter: { 'name': each.name, 'emp_id': each.emp_id },
update: { $set: { new_phone_number: each.new_phone_number } }
}
})
)
)
As the above - we're injecting a new field new_phone_number to all matched documents, final outcome will have old phone number as a field phone_number and also new phone number in a field new_phone_number .
Also as you haven't mentioned anything particular, if you really need to update each record individually making multiple DB calls, then you can normally use .update() or .updateOne() or .updateMany() .findAndModify() depends on your requirement/interest.
findAndModify , update , updateOne , updateMany
By Default, MongoDB $update method updates single record. But if you want to update multiple documents matching criteria, then you can use Multi parameter (set this to true).
db.people.update(
{ name: "Steve", emp_id:123 }, //this is first parameter of update method. Define your query here
{
new_phone_number: 111-111-111 //This is second parameter. Update operation here
},
{ multi: true } // This is third one. Options here
)
Also look at $updateOne and $updateMany
I have collection col that contains
{
'_id': ObjectId(...)
'type': "a"
'f1': data1
}
on same collection i have
{
'_id': ObjectId(...)
'f2': 222.234
'type': "b"
}
Spark MongoDB connector Is not working fine. It's reorder the data in wrong fields
for example:
{
'_id': ObjectId(...)
'type': "a"
'f1': data1
}
{
'_id': ObjectId(...)
'f1': data2
'type': "a"
}
Rdd will be:
------------------------
| id | f1 | type |
------------------------
| .... | a | data1 |
| .... | data2 | a |
------------------------
Is there any suggestions working with polymorphic schema
Is there any suggestions working with polymorphic schema
(Opinion alert) The best suggestion is not to have one in the first place. It is impossible to maintain in the long term, extremely error prone and requires complex compensation on the client side.
What to do if you have one:
You can try using Aggregation Framework with $project to sanitize data before it is fetched to Spark. See Aggregation section of the docs for example.
Don't try to couple it with structured format. Use RDDs, fetch data as plain Python dict and deal with the problem manually.
I am trying to import data from MongoDB to a relational DB (SQL Server).
I don't have access to the MongoDB components so I am querying my collection with the mongo java driver, in a tJava component.
I get a:
List< DBObject >
which I send to a tExtractJSONFields
An object of my collection looks like this:
[
{
"_id":{
"$oid":"1564t8re13e4ter86"
},
"object":{
"shop":"shop1",
"domain":"Divers",
"sell":[
{
"location":{
"zipCode":"58000",
"city":"NEVERS"
},
"properties":{
"description":"ddddd!!!!",
"id":"f1re67897116fre87"
},
"employee":[
{
"name":"employee1",
"id":"245975"
},
{
"name":"employee2",
"id":"458624"
}
],
"customer":{
"name":"Customer1",
"custid":"test_réf"
}
}
]
}
}
]
For a sell, I can have several employees. I have an array of employee and I want to store the affected employee in another table. So I would have 2 tables:
Sell
oid | shop | domain | zipCode | ...
1564t8re13e4ter86 | shop1 | Divers | 58000 | ...
Affected employee
employee_id | employee_name | oid
245975 | employee1 | 1564t8re13e4ter86
458624 | employee2 | 1564t8re13e4ter86
So I want to loop on the employee array, with a Jsonpath query:
"$[*].object.sell[0].employee"
The problem is that doing like this, I can't have the object_id. It seems that I can't get an attribute on a parent node if I define my Jsonpath query like this.
I also saw that I can do like in the following link:
http://techpoet.blogspot.ro/2014/06/dealing-with-nested-documents-in.html?utm_content=buffer02d59&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
But I don't understand when does he get the object_id at the lower levels.
How can I do?
my tests with JSONPath failed as well but I think this must be a bug in the component because when I query:
$..['$oid'] I'm getting back -> [].
This seems to be the case whenever you try to get a node that's on higher levels than the loop query.
I want the latest document in the query. Below I'm getting those documents whose name is coming from the variable 'personalFullName, then sorting them by a field called 'RecordID' (this field has higher numbers as later date entries), then grab the last one. I want the latest (the one with the largest RecordID number) entry in this query:
Programs.find({ FullName: personalFullName }, { sort: { RecordID: 1 }, limit: 1}).fetch().pop();
I'm getting an error that it's exceeding the call stack size.
If you are comfortable using the meteorhacks:aggregate package then you could always publish the item(s) you want using the mongo aggregate pipeline, perhaps something like this (code is coffeescript):
Meteor.publish 'latestPrograms', (personalFullName)->
return unless personalFullName?
check personalFullName, String
pipeline = [
{$match:{'Fullname': personalFullName}}
{$sort: {'RecordID': 1}}
{$group:{'_id':{Fullname: '$Fullname'}, RecordID:{$last:'$RecordID'}}}
{$limit:1}
]
#added 'latestPrograms', Random.id(), item for item in programs.aggregate pipeline
#ready()
You can then grab the data by subscribing to the latestPrograms pseudo collection. Here is an example using a iron router route:
Router.route '/home',
name: 'home'
template:'homepage'
waitOn:->
Meteor.subscribe 'latestPrograms', personalFullName
data:->
{latestPrograms: latestPrograms.find()}
I'm currently working with mongodb (mongoose). One of my example documents is:
myuser{
_id: 7777...,
money: 1000,
ships:[{
_id: 7777...
name: "myshipname",
products:[{
product_id: 7777....,
quantity: 24
}]
}
}
What I aim to do is to get a certain product given user id, ship id and product id, being the result something like: { product_id: 777..,quantity:24}
So far I got to find a certain user ship with:
findOne(userId,ships:{ $elemMatch: {_id:shipId}})
Which returns the information of the ship with shipId inside the array ships from user with userId. However, I cannot find the way to get only a certain product from that ship
What you want can probably best be done using the aggregation framework. Something like:
db.users.aggregate([
{$match: { _id : <user>}},
{$unwind: "$ships"},
{$unwind: "$ships.products"},
{$match: { "ships._id": <ship>}},
{$match: { "ships.products.product_id": <product>}}
]);
Note I'm not on a computer with mongo right now, so my syntax might be a bit off.