I'm trying to change database mysql to mongodb,
Wanting to change whole data schema.
And I succeed to migrate data.
But I want to change this mongodb documents schema from
{
"_id" : ObjectId("581c45b1a5245ca984000009"),
"memb_id" : "ME20160520041223736026",
"story_category_id" : "S001000",
"story_pet_category" : "D",
"temp_text" : "THIS IS DATA TO CHANGE",
"story_status" : "D",
"regtime" : 1463988822,
"modifytime" : 1468573528,
"story_contents" : [
{
"story_image_url" : "/uploads/2016/05/23/compression/728870e82c47901a70509cfdc2bc3b56.jpg"
},
{
"story_image_url" : "/uploads/2016/05/23/compression/fd1552e1ea5380fc0cdae793336a8d67.jpg"
},
{
"story_image_url" : "/uploads/2016/05/23/compression/2aaff6a75db4c8bfa1a0133234d3e6fe.jpg"
}
]
}
to
{
"_id" : ObjectId("581c45b1a5245ca984000009"),
"memb_id" : "ME20160520041223736026",
"story_category_id" : "S001000",
"story_pet_category" : "D",
"story_status" : "D",
"regtime" : 1463988822,
"modifytime" : 1468573528,
"story_contents" : [
{
"story_text":"THIS IS DATA TO CHANGE",
"story_image_url":""
}
{
"story_text":"",
"story_image_url" : "/uploads/2016/05/23/compression/728870e82c47901a70509cfdc2bc3b56.jpg"
},
{
"story_text":"",
"story_image_url" : "/uploads/2016/05/23/compression/fd1552e1ea5380fc0cdae793336a8d67.jpg"
},
{
"story_text":"",
"story_image_url" : "/uploads/2016/05/23/compression/2aaff6a75db4c8bfa1a0133234d3e6fe.jpg"
}
]
}
Exactly what I want is Pushing that "temp_text" in "story_contents" Array, Changing "story_contents"'s schema like this. (Whole 6423 documents)
How can I write mongodb query..?
You can go with an aggregation that will add story_text field/value to all items from story_contents array :
db.collection.aggregate([{
$unwind: "$story_contents"
}, {
$group: {
_id: "$_id",
memb_id: { $first: "$memb_id" },
story_category_id: { $first: "$story_category_id" },
story_pet_category: { $first: "$story_pet_category" },
story_status: { $first: "$story_status" },
regtime: { $first: "$regtime" },
modifytime: { $first: "$modifytime" },
story_contents: {
$push: { temp_text: "$temp_text", story_image_url: "$story_contents.story_image_url" }
}
}
}, {
$out: "collection2"
}])
1 $unwind to convert story_contents array to JSON object
1 $group to regroup all the items by _id with construction of the new array story_contents with additional temp_text field
1 $out to write the result into a new collection named collection2
Related
I am trying to delete the duplicate object inside the array in multiple documents in Mongodb.
I try many ways but not able to fix
Document Structure:-
{
"_id" : ObjectId("5a544fe234602415114601d3"),
"GstDetails" : [
{
"_id" : ObjectId("5e4837374d62f4c95163908e"),
"StateId" : "1",
"GstIn" : "33ABFFM1655H1ZF",
"StateDesc" : "TAMIL NADU",
"CityDesc" : "CHENNAI"
},
{
"_id" : ObjectId("5e4837484d62f4c9516395e8"),
"StateId" : "1",
"GstIn" : "33ABFFM1655H1ZF",
"StateDesc" : "TAMIL NADU",
"CityDesc" : "CHENNAI"
}
]
}
Like that many more documents
I tried:-
db.Supplier.find({ "GstDetails": { $size: 2 } }).limit(1).forEach(function (doc) {
var stateId;
doc.GstDetails.forEach(function (data) {
if (data.StateId == stateId) {
pull doc.GstDetails[0];
} else {
stateId = data.StateId
}
print(JSON.stringify(doc));
});
db.Supplier.save(doc)
});
Check if aggregation below meets your requirements:
db.Supplier.aggregate([
{
$unwind: "$GstDetails"
},
{
$group: {
_id: {
_id: "$_id",
StateId: "$GstDetails.StateId"
},
GstDetails: {
$push: "$GstDetails"
}
}
},
{
$addFields: {
GstDetails: {
$slice: [
"$GstDetails",
1
]
}
}
},
{
$unwind: "$GstDetails"
},
{
$group: {
_id: "$_id._id",
GstDetails: {
$push: "$GstDetails"
}
}
}
])
MongoPlayground
Note: This read-only query. If it is OK, you need to add as last stage below operator (once you execute it, it will update your documents, no rollback available):
{$out: "Supplier"}
I have a document like this:
{
"_id": ObjectId("5d779541bd4e75c58d598212")
"client": ObjectId("5d779558bd4e75c58d598213")
}
When I do $lookup like this:
{
from: 'client',
localField: 'client',
foreignField: 'id',
as: 'client',
}
I get:
{
"_id": ObjectId("5d779541bd4e75c58d598212")
"client":[
{
... client info wrapped in array
}
]
}
This forces me to add $unwind after the lookup stage.
This would work fine in this example because I know that it is a regular field (not array). But on other collections I have arrays of ObjectId's and I don't want to unwind them.
How should I tell mongo to unwind only if it's not an array?
Add $project stage with $arrayElemAt
{ $lookup ..... },
{ $project: { client: { $arrayElemAt: [ "$client" , 0 ]}} // Add other filed
The lookup always returns an array as it doesn't know if its a one-to-one or one-to-many mapping. But we can ensure that the lookup returns a single document and that document would hold all documents which were supposed to come as an array in the general lookup.
Following is the way:
db.collection.aggregate([
{
$lookup:{
"from":"client",
"let":{
"client":"$client"
},
"pipeline":[
{
$match:{
$expr:{
$eq:["$id","$$client"]
}
}
},
{
$group:{
"_id":null,
"data":{
$push:"$$ROOT"
}
}
},
{
$project:{
"_id":0
}
}
],
"as":"clientLookup"
}
},
{
$unwind:"$clientLookup"
}
]).pretty()
Query analysis: We are looking up into client collection and executing a pipeline inside that. The output of that pipeline would hold every matched document inside data field.
Data set:
Collection: collection
{
"client":1
}
{
"client":2
}
Collection: client
{
"id":1,
"name":"Tony"
}
{
"id":1,
"name":"Thor"
}
{
"id":1,
"name":"Natasha"
}
{
"id":2,
"name":"Banner"
}
Output:
{
"_id" : ObjectId("5d7792c6bd4e75c58d59820c"),
"client" : 1,
"clientLookup" : {
"data" : [
{
"_id" : ObjectId("5d779322bd4e75c58d59820e"),
"id" : 1,
"name" : "Tony"
},
{
"_id" : ObjectId("5d779322bd4e75c58d59820f"),
"id" : 1,
"name" : "Thor"
},
{
"_id" : ObjectId("5d779322bd4e75c58d598210"),
"id" : 1,
"name" : "Natasha"
}
]
}
}
{
"_id" : ObjectId("5d7792c6bd4e75c58d59820d"),
"client" : 2,
"clientLookup" : {
"data" : [
{
"_id" : ObjectId("5d779322bd4e75c58d598211"),
"id" : 2,
"name" : "Banner"
}
]
}
}
I have an aggregation framework query that is summarizing certain document data into a lookup set. Unfortunately, I can't provide the data since it's company-related. Here is the query and data fragment from the last stages of the pipeline:
...
{ $group: { _id: "$SectionId", "Questions": { $addToSet: "$Questions" } } },
{ $unwind: "$Questions" },
which returns data like this: Note that _id is not unique.
{
"_id" : "Tonometry",
"Questions" : {
"MappingId" : "Exophoria",
"PositiveLabel" : "Positive",
"NegativeLabel" : "Negative"
}
},
{
"_id" : "Tonometry",
"Questions" : {
"MappingId" : "Heterophoria",
"PositiveLabel" : "Positive",
"NegativeLabel" : "Negative"
}
},
The next stage in the pipeline is this:
{
$project: {
"_id": 1,
"id": ObjectId(),
"SectionId": "$_id",
"MappingId": "$Questions.MappingId",
"PositiveLabel": "$Questions.PositiveLabel",
"NegativeLabel": "$Questions.NegativeLabel",
}
},
which produces:
{
"_id" : "Tonometry",
"id" : ObjectId("5d1cf66cf526f23524f865c6"),
"SectionId" : "Tonometry",
"MappingId" : "Exophoria",
"PositiveLabel" : "Positive",
"NegativeLabel" : "Negative"
},
{
"_id" : "Tonometry",
"id" : ObjectId("5d1cf66cf526f23524f865c6"),
"SectionId" : "Tonometry",
"MappingId" : "Heterophoria",
"PositiveLabel" : "Positive",
"NegativeLabel" : "Negative"
},
I tried creating a new field Id that has a unique ObjectId but unfortunately just re-uses the same ObjectId in all the nodes. This is important because when I attempt to use $out, it requires a unique _id.
How do I add a unique ObjectId to each node?
Using $out does not need a unique _id, you can use $replaceRoot together with $mergeObjects prior to $out pipeline, this will merge the Question document into the desired document without an _id field and $out will create the _id field for you in the new collection:
[
....
{ "$group": { "_id": "$SectionId", "Questions": { "$addToSet": "$Questions" } } },
{ "$unwind": "$Questions" },
{ "$replaceRoot": {
"newRoot": {
"$mergeObjects": [
{ "Section": "$_id" },
"$Questions"
]
}
} },
{ "$out": "new-collection" }
]
db.school.find({ "merchant" : "cc8c0421-e7fc-464d-9e1d-37e168b216c3" })
this is an example document from school collection of that query:
{
"_id" : ObjectId("57fafasf2323232323232f57682cd42"),
"status" : "wait",
"merchant" : "cc8c0421-e7fc-464d-9e1d-37e168b216c3",
"isValid" : false,
"fields" : { "schoolid" : {
"value" : "2323232",
"detail" : {
"revisedBy" : "teacher",
"revisionDate" : ISODate("2015-06-24T09:22:44.288+0000")
},
"history" : [
]
}}
}
I want to see which has duplcate schoolid. SO i do this:
db.school.aggregate([
{$match:{ "merchant" : "cc8c0421-e7fc-464d-9e1d-37e168b216c3"
{ $group: {
_id: { fields.schoolid.value: "$fields.schoolid.value" },
count: { $sum: 1 }
} },
{ $match: {
count: { $gte: 2 }
} },
{ $sort : { count : -1} },
{ $limit : 10 }
]);
but it gives error.
a lot of errors for a lot of lines
i tried to do like this
_id: { "fields.schoolid.value": "$fields.schoolid.value" },
or
_id: { 'fields.schoolid.value': "$'fields.schoolid.value'" },
but did not work. ow can i use it?
According to the document you provided, there is no fields field, so the group stage can't work. Your query should be :
db.school.aggregate([
{ $match: { "merchant" : "cc8c0421-e7fc-464d-9e1d-37e168b216c3"}},
{ $group: {
_id: { value: "$fields.schoolid.value" },
count: { $sum: 1 }
} },
{ $match: {
count: { $gte: 2 }
} },
{ $sort : { count : -1} },
{ $limit : 10 }
]);
Also note that fields.schoolid.value is not a valid fieldname, you need to enclode it in "" or to remove the "."
I have a bunch of documents in mongo with the following structure:
{
"_id" : "",
"number" : 2,
"colour" : {
"_id" : "",
"name" : "Green",
"hex" : "00ff00"
},
"position" : {
"_id" : "",
"name" : "Defence",
"type" : "position"
},
"ageGroup" : {
"_id" : "",
"name" : "Minor Peewee",
"type" : "age"
},
"companyId" : ""
}
I'm currently using Mongo's aggregate to group the documents by ageGroup.name which returns:
//Query
Jerseys.aggregate([
{$match: { companyId: { $in: companyId } } },
{$group: {_id: "$ageGroup.name", jerseys: { $push: "$$ROOT" }} }
]);
//returns
{
_id: "Minor Peewee",
jerseys: array[]
}
but I'd like it to also group by position.name within the age groups. ie:
{
_id: "Minor Peewee",
positions: array[]
}
//in positions array...
{
_id: "Defence",
jerseys: array[]
}
// or ageGroups->positions->jerseys if that makes more sense.
I've tried multiple groups but I don't think I'm setting them up correctly I always seem to get an array of _id's. I'm using Meteor as the server and I'm doing it within a meteor method.
You can use a composite aggregate _id in the first grouping stage.
Then, you can use one of those keys as the "main" _id of the final aggregate and $push the other into another array.
Jerseys.aggregate([
{
$match: { companyId: { $in: companyId } }
},
{
$group: { // each position and age group have an array of jerseys
_id: { position: "$position", ageGroup: "$ageGroup" },
jerseys: { $push: "$$ROOT" }
}
},
{
$group: { // for each age group, create an array of positions
_id: { ageGroup: "$_id.ageGroup" },
positions: { $push: { position: "$_id.position", jerseys:"$jerseys" } }
}
}
]);