I am trying to write a table with various numbers of rows and columns to a database. For this, I have a Criterion collection that saves the table headers and an Option collection that saves the table data.
The structure is like this:
for Criterion:
{
{ "_id" : "hId1", "name" : "Option Name", "tableId" : "tId1" },
{ "_id" : "hId2", "name" : "Rent", "score" : 9, "tableId" : "tId1" },
{ "_id" : "hId3", "name" : "Surface", "score" : 5, "tableId" : "tId1" },
{ "_id" : "hId4", "name" : "Price", "score" : 5, "tableId" : "tId1" },
{ "_id" : "hId5", "name" : "CPU", "score" : 5, "tableId" : "tId4" }
etc.
}
for Option:
{
{ "_id" : "id1", "score" : 5,
"hId1" : { "value" : "Apt 1" },
"hId2" : { "value" : "800 sqft", "score" : 1 },
"hId3" : { "value" : "$800", "score" : 3 },
etc.
"tableId" : "tId1"
}
{ "_id" : "id2", "score" : 5,
"hId1" : { "value" : "Apt 2" },
"hId2" : { "value" : "780 sqft", "score" : 10 },
"hId3" : { "value" : "$700", "score" : 3 },
etc.
"tableId" : "tId1"
}
etc.
}
The first row for Criterion will always have "Option Name". For the data above, the table with "tableId" = "tId1" would end up looking like this (tableId and headerId are the keys):
| Option Name | Surface | Price |
| =========== | ======== | ===== |
| Apt 1 | 800 sqft | $800 |
| Apt 2 | 780 sqft | $700 |
My code looks like this (imports/api/comparison.js):
/**
* Options are for the rows
*/
export var Option = new Mongo.Collection('option');
/**
* Criteria are the columns
*/
export var Criterion = new Mongo.Collection('criterion');
Meteor.methods({
'comparison.insertRow' (query, headerId, tableId, isFirst) {
check(query, Object);
check(headerId, String);
check(tableId, String);
check(isFirst, Boolean);
if(isFirst){
var data = {};
data._id = headerId;
data.tableId = tableId;
data.name = "Option Name";
Criterion.insert(data);
}
query._id = tableId;
Option.insert(query);
},
});
Where isFirst is a boolean expressing whether this is the first row in a table or not.
My query is constructed like this (imports/ui/Menu/DataInsert.jsx):
var query = {};
query.score = // value
// get the header separately
query[headerId] = {
value: //valueH from form
};
// Find the text field via the React ref
for (var i = 1, len = cols.length; i < len; i++) {
query[cols[i]._id] = {
value: //valueV from form,
score: //valueS from form
};
}
My files are available on the server because I am doing this in server/main.js: import '../imports/api/comparison.js';
The query gets inserted no problem into Option no problem.
Why isn't data getting inserted into Criterion (when isFirst = true)?
I did a console.log(data) and a console.log(query) and it looks like this:
whereas the data in the db looks like this:
#jordanwillis was correct above, this was happening because I was setting the _id manually on the query. I did need to set the id, but I needed to set the tableId.
So, for the record, this is what I did:
'comparison.insertRow' (query, headerId, tableId, isFirst) {
check(query, Object);
check(headerId, String);
check(tableId, String);
check(isFirst, Boolean);
if(isFirst){
var data = {};
data._id = headerId;
data.tableId = tableId;
data.name = "Option Name";
Criterion.insert(data);
}
query.tableId = tableId;
Option.insert(query);
},
And my foreign keys are tableId and headerId (which is part of query).
Related
I want to find one document and clone/copy that document and create 100 new documents with new value for few fields using shell script in mongodb.
Below is my document
{
"_id" : ObjectId("5ef59bde562c9824176e9f20"),
"productDefinition" : {
"product" : {
"companies" : {
"company" : {
"productionformation" : {
"productNumber" : "E128",
"venderNumber" : "0470",
"venderName" : "ALPHA SERVICES LLC"
}
}
}
}
},
"executionId" : "123456"
}
After executing the shell script, i want to have new 100 collection with new values for the below fields
"executionId" : "NewExecutionId" // This value will be Fixed for all new 100 documents
"productNumber" : "1" //This value will be increasing.. for first document 1, for second document 2, etc..
"venderNumber" : "1" //This value will be increasing.. for first document 1, for second document 2, etc..
My new collection will be looking like this.
First new document
{
"_id" : ObjectId("5ef59bde562c9824176e9f20"),
"productDefinition" : {
"product" : {
"companies" : {
"company" : {
"productionformation" : {
"productNumber" : "1",
"venderNumber" : "1",
"venderName" : "ALPHA SERVICES LLC"
}
}
}
}
},
"executionId" : "newExecutionId"
}
Second new document
{
"_id" : ObjectId("5ef59bde562c9824176e9f20"),
"productDefinition" : {
"product" : {
"companies" : {
"company" : {
"productionformation" : {
"productNumber" : "2",
"venderNumber" : "2",
"venderName" : "ALPHA SERVICES LLC"
}
}
}
}
},
"executionId" : "newExecutionId"
}
Third new document
{
"_id" : ObjectId("5ef59bde562c9824176e9f20"),
"productDefinition" : {
"product" : {
"companies" : {
"company" : {
"productionformation" : {
"productNumber" : "3",
"venderNumber" : "3",
"venderName" : "ALPHA SERVICES LLC"
}
}
}
}
},
"executionId" : "newExecutionId"
}
Like this fourth document , fifth document, etc... till 100th document...
I tried with this script. but its not working.
copy = db.myCollection.find({"executionId" : "123456",
"productDefinition.product.companies.company.productionformation.productNumber" : "E128" ,
"productDefinition.product.companies.company.productionformation.venderNumber" :"0470" })
for (var i = 1; i< 101; i++){
copy.executionId = "newExecutionId";
copy.productDefinition.product.companies.company.productionformation.productNumber = i;
copy.productDefinition.product.companies.company.productionformation.venderNumber" = i;
db.myCollection.insert(copy);
}
You will be needing to fix following things:
Use findOne instead of find as it will return single matching document.
Use let (instead of var) while running the loop because there are asynchronous DB operations in loop body.
Similarly, create a Deep copy of matchedDoc result / (copy variable) inside for loop body, to avoid updating same object's reference value.
Hope it helps !
Need help on MongoDB nested query. Below is my mongo collection.
Preference collection
{
"_id" : "user123",
"preferences" : {
"product-1" : {
"frequency" : "Weekly",
"details" : {
"email" : {
"value" : "On"
}
}
},
"product-2" : {
"preferencesFor" : "mpc-other",
"preferencesForType" : "Product",
"details" : {
"email" : {
"value" : "Off"
}
}
},
"product-3" : {
"preferencesFor" : "mpc-other",
"preferencesForType" : "Product",
"details" : {
"email" : {
"value" : "On"
}
}
}
}
}
Product Collection
{
"_id" : "product-1",
"name" : "Geo-Magazine"
}
{
"_id" : "product-2",
"name" : "History-Magazine"
}
{
"_id" : "product-3",
"name" : "Science-Magazine"
}
product-1, product-2... are keys from a Map.
The keys are stored in another collection Product Collection.
Can I create a nested query to cross-reference the product keys from another table?
I need the output in the below table format. Please suggest.
user123 product-1 email On
user123 product-2 email Off
user123 product-3 email On
I tried the below but can't get result. Please suggest.
var cursor = db.productSummary.find();
while(cursor.hasNext()){
var sku = cursor.next()._id;
var skuCol = "preferences."+sku+".details.email";
var skuVal = "preferences."+sku+".details.email.value";
db.marketingPreferences.find( {}, {_id:1, skuCol:1, skuVal:1});
}
> var myCursor = db.productSummary.find();
> while(myCursor.hasNext()){
var sku = myCursor.next()._id;
var skuCol = "preferences."+sku+".details.email";
var skuVal = "$preferences."+sku+".details.email.value";
var result = db.marketingPreferences.aggregate([{"$project":{"_id":1,value:skuVal,preferences:{$literal: sku}}}],{allowDiskUse: true});
while(result.hasNext()){
printjson(result.next());
}
}
Result
{ "_id" : "user123", "preferences" : "product-1", "value" : "On" }
{ "_id" : "user123", "preferences" : "product-2", "value" : "Off" }
{ "_id" : "user123", "preferences" : "product-3", "value" : "On" }
There's a difference between MongoDB and normal SQL DB. Firstly, when you query a MongoDB collection, it doesn't return a row as it will in a SQL db. What you get here is a document similar to JSON.
Also when you use preferences.product-1.details.email : 1 it wont return you the word 'email', rather it will return you the value ie. {"value" : "On" }.
Using this: db.preference.find({},{"_id":1,"preferences.product1.details.email.value":1})
you will be able to get two details which are user123 and On and you can get product-1 from your previous query. You can store these values in a variable and keep printing them to obtain the table necessary. Also you would need another cursor to store the result of the second second query that you would do.
Here's what your query will produce if it was single standalone query:
> db.preference.find({},{"_id":1,"preferences.product1.details.email.value":1})
.pretty()
{
"_id": "user123",
"preferences": {
"product-1": {
"details": {
"email": {
"value": "On"
}
}
}
}
}
public static void test(){
MongoCollection<Document> collection = getDatadase().getCollection("product");
MongoCollection<Document> pref = getDatadase().getCollection("pref");
List<Document> allDocList = collection.find().into(new ArrayList<Document>());
for(Document doc:allDocList){
System.out.println(doc.get("_id"));
String preferences = doc.get("_id")+"";
String sku = "$preferences."+preferences+".details.email.value";
Document aggregation = new Document().append("$project", new Document().append("_id", 1).append("value", sku));
List<Document> pipeline = new ArrayList<Document>();
pipeline.add(aggregation);
List<Document> aggList = pref.aggregate(pipeline).into(new ArrayList<Document>());
for(Document doc1:aggList){
System.out.println(doc1.append("preferences", preferences));
}
}
}
This Will return
product-1
Document{{_id=user123, value=On, preferences=product-1}}
product-2
Document{{_id=user123, value=Off, preferences=product-2}}
product-3
Document{{_id=user123, value=On, preferences=product-3}}
We have a basic enquiry management tool that we're using to track some website enquiries in our administration suite, and we're using the ObjectId of each document in our enquiries collection to sort the enquiries by the date they were added.
{
"_id" : ObjectId("53a007db144ff47be1000003"),
"comments" : "This is a test enquiry. Please ignore. We'll delete it shortly.",
"customer" : {
"name" : "Test Enquiry",
"email" : "test#test.com",
"telephone" : "07890123456",
"mobile" : "07890123456",
"quote" : false,
"valuation" : false
},
"site" : [],
"test" : true,
"updates" : [
{
"_id" : ObjectId("53a007db144ff47be1000001"),
"status" : "New",
"status_id" : ObjectId("537de7c3a5e6e668ffc2335c"),
"status_index" : 100,
"substatus" : "New Web Enquiry",
"substatus_id" : ObjectId("5396bb9fa5e6e668ffc23388"),
"notes" : "New enquiry received from website.",
},
{
"_id" : ObjectId("53a80c977d299cfe91bacf81"),
"status" : "New",
"status_id" : ObjectId("537de7c3a5e6e668ffc2335c"),
"status_index" : 100,
"substatus" : "Attempted Contact",
"substatus_id" : ObjectId("53a80e06a5e6e668ffc2339e"),
"notes" : "In this test, we pretend that we've not managed to get hold of the customer on the first attempt.",
},
{
"_id" : ObjectId("53a80e539b966b8da5c40c36"),
"status" : "Approved",
"status_id" : ObjectId("52e77a49d85e95f00ebf6c72"),
"status_index" : 200,
"substatus" : "Enquiry Confirmed",
"substatus_id" : ObjectId("53901f1ba5e6e668ffc23372"),
"notes" : "In this test, we pretend that we've got hold of the customer after failing to contact them on the first attempt.",
}
]
}
Within each enquiry is an updates array of objects which also have an ObjectId as their main identity field. We're using an $unwind and $group aggregation to pull the first and latest updates, as well as the count of updates, making sure we only take enquiries where there have been more than one update (as one is automatically inserted when the enquiry is made):
db.enquiries.aggregate([
{
$match: {
"test": true
}
},
{
$unwind: "$updates"
},
{
$group: {
"_id": "$_id",
"latest_update_id": {
$last: "$updates._id"
},
"first_update_id": {
$first: "$updates._id"
},
"update_count": {
$sum: 1
}
}
},
{
$match: {
"update_count": {
$gt: 1
}
}
}
])
This results in the following output:
{
"result" : [
{
"_id" : ObjectId("53a295ad122ea80200000005"),
"latest_update_id" : ObjectId("53a80bdc7d299cfe91bacf7e"),
"first_update_id" : ObjectId("53a295ad122ea80200000003"),
"update_count" : 2
},
{
"_id" : ObjectId("53a007db144ff47be1000003"),
"latest_update_id" : ObjectId("53a80e539b966b8da5c40c36"),
"first_update_id" : ObjectId("53a007db144ff47be1000001"),
"update_count" : 3
}
],
"ok" : 1
}
This is then passed through to our code (node.js, in this case) where we perform a few operations on it and then present some information on our dashboard.
Ideally, I'd like to add another $group pipeline aggregation to the query which would subtract the timestamp of first_update_id from the timestamp of latest_update_id to give us a timespan, which we could then use $avg on.
Can anyone tell me if this is possible? (Thank you!)
As Neil already pointed out, you can't get to the timestamp from the ObjectId in the aggregation framework.
You said that speed is not important, so using MapReduce you can get what you want:
var map = function() {
if (this.updates.length > 1) {
var first = this.updates[0];
var last = this.updates[this.updates.length - 1];
var diff = last._id.getTimestamp() - first._id.getTimestamp();
var val = {
latest_update_id : last._id,
first_update_id : first._id,
update_count : this.updates.length,
diff: diff
}
emit(this._id, val);
}
};
var reduce = function() { };
db.runCommand(
{
mapReduce: "enquiries",
map: map,
reduce: reduce,
out: "mrresults",
query: { test : true}
}
);
This are the results:
{
"_id" : ObjectId("53a007db144ff47be1000003"),
"value" : {
"latest_update_id" : ObjectId("53a80e539b966b8da5c40c36"),
"first_update_id" : ObjectId("53a007db144ff47be1000001"),
"update_count" : 3,
"diff" : 525944000
}
}
Edit:
If you want to get the average diff for all documents you can do it like this:
var map = function() {
if (this.updates.length > 1) {
var first = this.updates[0];
var last = this.updates[this.updates.length - 1];
var diff = last._id.getTimestamp() - first._id.getTimestamp();
emit("1", {diff : diff});
}
};
var reduce = function(key, values) {
var reducedVal = { count: 0, sum: 0 };
for (var idx = 0; idx < values.length; idx++) {
reducedVal.count += 1;
reducedVal.sum += values[idx].diff;
}
return reducedVal;
};
var finalize = function (key, reducedVal) {
reducedVal.avg = reducedVal.sum/reducedVal.count;
return reducedVal;
};
db.runCommand(
{
mapReduce: "y",
map: map,
reduce: reduce,
finalize : finalize,
out: "mrtest",
query: { test : true}
}
);
And the example output:
> db.mrtest.find().pretty()
{
"_id" : "1",
"value" : {
"count" : 2,
"sum" : 1051888000,
"avg" : 525944000
}
}
I am trying to find in a collection all of the documents that have the given key equal to one of the strings in an array.
Heres an example of the collection.
{
roomId = 'room1',
name = 'first'
},
{
roomId = 'room2',
name = 'second'
},
{
roomId = 'room3',
name = 'third'
}
And heres an example of the array to look through.
[ 'room2', 'room3' ]
What i thought would work is...
collection.find({ roomId : { $in : [ 'room2', 'room3' ]}}, function( e, r )
{
// r should return the second and third room
});
How can i achieve this?
One way this could be solve would be to do a for loop...
var roomIds = [ 'room2', 'room3' ];
for ( var i=0; i < roomIds.length; i++ )
{
collection.find({ id : roomIds[ i ]})
}
But this is not ideal....
What you posted should work - no looping required. The $in operator does the job:
> db.Room.insert({ "_id" : 1, name: 'first'});
> db.Room.insert({ "_id" : 2, name: 'second'});
> db.Room.insert({ "_id" : 3, name: 'third'});
> // test w/ int
> db.Room.find({ "_id" : { $in : [1, 2] }});
{ "_id" : 1, "name" : "first" }
{ "_id" : 2, "name" : "second" }
> // test w/ strings
> db.Room.find({ "name" : { $in : ['first', 'third'] }});
{ "_id" : 1, "name" : "first" }
{ "_id" : 3, "name" : "third" }
Isn't that what you expect?
Tested w/ MongoDB 2.1.1
I am trying to run a map/reduce function in mongodb where I group by 3 different fields contained in objects in my collection. I can get the map/reduce function to run, but all the emitted fields run together in the output collection. I'm not sure this is normal or not, but outputting the data for analysis takes more work to clean up. Is there a way to separate them, then use mongoexport?
Let me show you what I mean:
The fields I am trying to group by are the day, user ID (or uid) and destination.
I run these functions:
map = function() {
day = (this.created_at.getFullYear() + "-" + (this.created_at.getMonth()+1) + "-" + this.created_at.getDate());
emit({day: day, uid: this.uid, destination: this.destination}, {count:1});
}
/* Reduce Function */
reduce = function(key, values) {
var count = 0;
values.forEach(function(v) {
count += v['count'];
}
);
return {count: count};
}
/* Output Function */
db.events.mapReduce(map, reduce, {query: {destination: {$ne:null}}, out: "TMP"});
The output looks like this:
{ "_id" : { "day" : "2012-4-9", "uid" : "1234456", "destination" : "Home" }, "value" : { "count" : 1 } }
{ "_id" : { "day" : "2012-4-9", "uid" : "2345678", "destination" : "Home" }, "value" : { "count" : 1 } }
{ "_id" : { "day" : "2012-4-9", "uid" : "3456789", "destination" : "Login" }, "value" : { "count" : 1 } }
{ "_id" : { "day" : "2012-4-9", "uid" : "4567890", "destination" : "Contact" }, "value" : { "count" : 1 } }
{ "_id" : { "day" : "2012-4-9", "uid" : "5678901", "destination" : "Help" }, "value" : { "count" : 1 } }
When I attempt to use mongoexport, I can not separate day, uid, or destination by columns because the map combines the fields together.
What I would like to have would look like this:
{ { "day" : "2012-4-9" }, { "uid" : "1234456" }, { "destination" : "Home"}, { "count" : 1 } }
Is this even possible?
As an aside - I was able to make the output work by applying sed to the file and cleaning up the CSV. More work, but it worked. It would be ideal if I could get it out of mongodb in the correct format.
MapReduce only returns documents of the form {_id:some_id, value:some_value}
see: How to change the structure of MongoDB's map-reduce results?