How to insert in a collection when other collections already exists - mongodb

I am very new to noSQL databases. I have started my journey with mongodb Database in noSQL. Upto today I used mySQL and MS SQL Server. I know how to create tables and relationships between them. But this noSQL concept is totally different than that. So, I am asking this question. I hope I will learn sooner to work with noSQL databases.
I have used the following queries in mongodb to create collections and insert data in it:
db.createCollection('Parties');
db.parties.insert({partyName:"Best Sellers", mobileNo:"9876543214"});
db.parties.insert({partyName:"National Traders", mobileNo:"9876543215"});
db.createCollection('items');
db.items.insert({itemName:"Item-A", size:"12"});
db.items.insert({itemName:"Item-B", size:"8"});
db.createCollection('orders');
Visually my order will have these fields:
Order No : AB/123
Date of Order: 17-06-2016
Party Name : National Traders // It will be a select
Item Name Quantity Rate Amount
--------------------------------------------------------
Item-A //Select 200 20 4000
Item-B //Select 100 30 3000
--------------------------------------------------------
300 7000
My question is:
How can I make a query for mongodb to insert in order collection? I have never used something like nested collections. Can anybosy give me an example??
Update:
Here is the diagram of my database in relational database :
How can I convert it to mongodb style??
Update2:
I have tried to add this code for model order in Node.js:
var mongoose = require('mongoose');
var orderSchema = new mongoose.Schema({
salesOrderId: String,
orderDate: Date,
party: {type: Schema.Types.ObjectId, ref:'Party'},
items:[{{type: Schema.Types.ObjectId, ref:'Item'}, quantity: Number, rate: Number}],
dispatches:{
invoiceId: Number,
dispatchDate: Date,
items: [{{type: Schema.Types.ObjectId, ref:'Item'}, quantity: Number, rate: Number}]
}
});
mongoose.model('Order', orderSchema);

MongoDB doesn't follow the relational database data structure.
Copying the SQL data structure to MongoDB is not the ideal way of using MongoDB. Your application should be modified to work with MongoDB style of data structure.
For storing order data, I would suggest using just one collection which has the entire order data.
{
orderNumber : 7283,
party : {partyName:"Best Sellers", mobileNo:"9876543214"},
items : [
{itemName:"Item-A", size:"12"},
{itemName:"Item-B", size:"8"},
.....
]
orderDate : <>,
total : <>,
status : <>,
.....
}

this problem need three collections:
Item - which will hold item data
Party - which will hold party data
Order which will hold all related data
When order is placed by party, in order collection we are creating document and placing in it some subdocuments:
Order data (id, date, etc)
PartyDetails - as a whole document from party collection
Items - contains an array of all items selected by party with details (Quantity, Rate,etc)
Dispatch and DispatchDetails - those depend of how items are shipped
this structure utilities efficiently document oriented storage, and every time you need to check order status, you need just to ask for one document and you will get subdocuments at one go.
To insert data in such collection will depend of application language you will use, but I think this answer will provide you with power of document database solution.
Any comments welcome!
* EDIT *
var itemToInsertA = db.items.find({"itemName" : "Item-A"}).toArray()
var itemToInsertB = db.items.find({"itemName" : "Item-B"}).toArray()
var party = db.parties.find({"partyName" : "National Traders"}).toArray()
var order = {
//_id - will be assigned by db - but we can do it mannually
orderDate : new Date(), // now
salesOrderId : "Need to provide this from sale now or later",
PartyDetails : party[0],
Items : [{
item : itemToInsertA[0],
Quantity : 200,
Rate : 20,
ValueAtTransactionDate : 4000
}, {
item : itemToInsertB[0],
Quantity : 100,
Rate : 30,
ValueAtTransactionDate : 3000
}
]
}
db.orders.insertOne(order)
so finnaly our order looks like this:
{
"_id" : ObjectId("5767d030ecf8248c30af068c"),
"orderDate" : ISODate("2016-06-20T11:14:56.372Z"),
"salesOrderId" : "Need to provide this from sale now or later",
"PartyDetails" : {
"_id" : ObjectId("5767ce26ecf8248c30af0686"),
"partyName" : "National Traders",
"mobileNo" : "9876543215"
},
"Items" : [
{
"item" : {
"_id" : ObjectId("5767ce26ecf8248c30af0687"),
"itemName" : "Item-A",
"size" : "12"
},
"Quantity" : 200.0,
"Rate" : 20.0,
"ValueAtTransactionDate" : 4000.0
},
{
"item" : {
"_id" : ObjectId("5767ce26ecf8248c30af0688"),
"itemName" : "Item-B",
"size" : "8"
},
"Quantity" : 100.0,
"Rate" : 30.0,
"ValueAtTransactionDate" : 3000.0
}
]
}
to create a diagram I used plantUML with QEditor.
plantumpl dump
#startuml
package "Party"{
[PartyId]
[PartyCode]
[PartyName]
[MobileNumber]
}
package "Item"{
[ItemId]
[ItemCode]
[ItemName]
}
package "Order"{
[OrderId]
[SalesOrderId]
[OrderDate]
node "Dispatch"{
[InvoiceId]
[SalesInvoiceId]
[DateOfDispatch]
}
node "DispatchItemTransaction"{
[DispatchItemTId]
[Quantity.]
[Rate.]
[ItemId.]
[InvoiceId.]
}
component PartyDetails
node "Items"{
component ItemDetails
[Quantity]
[Rate]
[Value]
}
}
Item -> ItemDetails
ItemDetails-->ItemId.
Party -down-> PartyDetails
#enduml

Related

GET a string inside a object nested within a array

I have a get router that gets orders with the time stamp
router.get('/api/order/:date/:time', function (req,res,next) {
Box.find({orders: {date:req.params.date}}.then (function (order){
console.log('GET ORDER / with ', req.params.date);
res.send(order);
}).catch(next);
});
the :time is just to allow my frontend to call this specific get which offers a timestamp inside the :date parameter
now, the model
const orderSchema = new Schema({
name : { type : String },
date : { type : String }, // date stamp of only YYYY/MM/DD
orders : { type : Array}
});
Inside this array of orders you can find elements such as:
"orders" : [
{
"type" : "Meat Lover Slice",
"extraType" : "na",
"extraInfo" : "na",
"date" : "2018-09-27:08:47:07",
"quantity" : "1",
"size" : "medium",
"crust" : "normal",
"split" : false,
and so on.. (15 or so elements)
You can see inside this array of orders, there are time stamped with YYYY/MM/DD:HH:MM:SS (2018-09-27:08:47:07).
inside the router, I do get
console.log('GET ORDER / with ', req.params.date) // > GET ORDER / with 2018-09-27:08:47:07
so it does receive the time at this route.
But with the params, how do I filter out the the specific orders matching that element?
If I have understood the question correctly is the short answer is that you can't, there is no way to "filter" sub-documents in a standard query. (Have a look at Return only matched sub-document elements within a nested array for a more in depth answer.)
What you could do is either use MongoDB Aggregations (https://docs.mongodb.com/manual/aggregation/) or do the filtering yourself when you have received the query result.
I would suggest doing the filtering yourself.

What's the better way to represent this multidimensional data?

We are trying to represent this data in a web application.What will be the appropriate way to represent this data? We thought of using relational structure but data are hierarchical in nature.Is it better to use MongoDB in this scenario ?
As per comment mongo dynamic schemas is a perfect solution.
let assume that we have our document structured like this:
report {
_id...
documentBycode {
_id : "G8",
dataSource,
remarks,
fields : [{
indicator : "sucide",
baseline {
data,
year,
source
},
milestone[{
year : 2017,
value : 15
}, {}
]
...
...
fields : [{
name : "nfhs1996",
value : "n/a",
order : 1 /* this is not mandatory*/
}, {
name : "ndhs2011",
value : "n/a",
order : 2
}
]
]
}
}
then you can add/modify elements as needed inside [arrays] and always get report data by retrieving only one document from datastore.
What's also could be interesting you could have mulitple diffrent reports structures stored in same collection - so you can literally store full ViewModel data AS IS
Any comment welcome!

Complex Grid Implementation in meteor(blaze)

First let me explain schema of my collections.
I have 3 collections
company,deal,price
I want to use information from all three collection and make a single reactive,responsive table. Here is the image
Now the schema for price collection is like this
{
"_id" : "kSqH7QydFnPFHQmQH",
"timestamp" : ISODate("2015-10-11T11:49:50.241Z"),
"dealId" : "X5zTJ2y675PjmaLMx",
"deal" : "Games",
"price" : [{
"type" : "worth",
"value" : "Bat"
}, {
"type" : "Persons",
"value" : 4
}, {
"type" : "Cost",
"value" : 5
}],
"company" : "Company1"
}
Schema for company collection is
{
"_id" : "da2da"
"name" : "Company1"
}
Schema for deal collection is
{
"_id" : "X5zTJ2y675PjmaLMx",
"name" : "Games"
}
For each company there will be 3 columns added in table(worth,persons,cost)
For each deal there will be a new row in table.
As the information is coming from 3 collections into a single table. First I want to ask is it wise to make a table from 3 different collections? If yes how could I do that in blaze?
If no. Then I will have to make table from price collection only . What should be schema of this collection in best way.
P.S in both cases I want to make table reactive.
Firstly, I recommend reywood:publish-composite for publishing related collections.
Secondly there is no intrinsic problem in setting up a table like this, you'll first figure out which collection to loop over with your {{#each}} in spacebars and then you'll define helpers that return the values from the related collections to your templates.
As far as your schema design, the choice as to whether to use nesting within a collection vs. using an entirely separate collection is typically driven by size. If the related object is overall "small" then nesting can work well. You automatically get the nested object when you publish and query that collection. If otoh it's going to be "large" and/or you want to avoid having to update every document when something in the related object changes then a separate collection can be better.
If you do separate your collections then you'll want to refer to objects from the other collection by _id and not by name since names can easily change. For example in your price collection you'd want to use companyId: "da2da" instead of company: "Company1"

The correct way of storing document reference in one-to-one relationship in MongoDB

I have two MongoDB collections user and customer which are in one-to-one relationship. I'm new to MongoDB and I'm trying to insert documents manually although I have Mongoose installed. I'm not sure which is the correct way of storing document reference in MongoDB.
I'm using normalized data model and here is my Mongoose schema snapshot for customer:
/** Parent user object */
user: {
type: Schema.Types.ObjectId,
ref: "User",
required: true
}
user
{
"_id" : ObjectId("547d5c1b1e42bd0423a75781"),
"name" : "john",
"email" : "test#localhost.com",
"phone" : "01022223333",
}
I want to make a reference to this user document from the customer document. Which of the following is correct - (A) or (B)?
customer (A)
{
"_id" : ObjectId("547d916a660729dd531f145d"),
"birthday" : "1983-06-28",
"zipcode" : "12345",
"address" : "1, Main Street",
"user" : ObjectId("547d5c1b1e42bd0423a75781")
}
customer (B)
{
"_id" : ObjectId("547d916a660729dd531f145d"),
"birthday" : "1983-06-28",
"zipcode" : "12345",
"address" : "1, Main Street",
"user" : {
"_id" : ObjectId("547d5c1b1e42bd0423a75781")
}
}
Remember these things
Embedding is better for...
Small subdocuments
Data that does not change regularly
When eventual consistency is acceptable
Documents that grow by a small amount
Data that you’ll often need to perform a second query to fetch Fast reads
References are better for...
Large subdocuments
Volatile data
When immediate consistency is necessary
Documents that grow a large amount
Data that you’ll often exclude from the results
Fast writes
Variant A is Better.
you can use also populate with Mongoose
Use variant A. As long as you don't want to denormalize any other data (like the user's name), there's no need to create a child object.
This also avoids unexpected complexities with the index, because indexing an object might not behave like you expect.
Even if you were to embed an object, _id would be a weird name - _id is only a reserved name for a first-class database document.
One to one relations
1 to 1 relations are relations where each item corresponds to exactly one other item. e.g.:
an employee have a resume and vice versa
a building have and floor plan and vice versa
a patient have a medical history and vice versa
//employee
{
_id : '25',
name: 'john doe',
resume: 30
}
//resume
{
_id : '30',
jobs: [....],
education: [...],
employee: 25
}
We can model the employee-resume relation by having a collection of employees and a collection of resumes and having the employee point to the resume through linking, where we have an ID that corresponds to an ID in th resume collection. Or if we prefer, we can link in another direction, where we have an employee key inside the resume collection, and it may point to the employee itself. Or if we want, we can embed. So we could take this entire resume document and we could embed it right inside the employee collection or vice versa.
This embedding depends upon how the data is being accessed by the application and how frequently the data is being accessed. We need to consider:
frequency of access
the size of the items - what is growing all the time and what is not growing. So every time we add something to the document, there is a point beyond which the document need to be moved in the collection. If the document size goes beyond 16MB, which is mostly unlikely.
atomicity of data - there're no transactions in MongoDB, there're atomic operations on individual documents. So if we knew that we couldn't withstand any inconsistency and that we wanted to be able to update the entire employee plus the resume all the time, we may decide to put them into the same document and embed them one way or the other so that we can update it all at once.
In mongodb its very recommended to embedding document as possible as you can, especially in your case that you have 1-to-1 relations.
Why? you cant use atomic-join-operations (even it is not your main concern) in your queries (not the main reason). But the best reason is each join-op (theoretically) need a hard-seek that take about 20-ms. embedding your sub-document just need 1 hard-seek.
I believe the best db-schema for you is using just an id for all of your entities
{
_id : ObjectId("547d5c1b1e42bd0423a75781"),
userInfo :
{
"name" : "john",
"email" : "test#localhost.com",
"phone" : "01022223333",
},
customerInfo :
{
"birthday" : "1983-06-28",
"zipcode" : "12345",
"address" : "1, Main Street",
},
staffInfo :
{
........
}
}
Now if you just want the userinfo you can use
db.users.findOne({_id : ObjectId("547d5c1b1e42bd0423a75781")},{userInfo : 1}).userInfo;
it will give you just the userInfo:
/* 0 */
{
"name" : "john",
"email" : "test#localhost.com",
"phone" : "01022223333"
}
And if you just want the **customerInfo ** you can use
db.users.findOne({_id : ObjectId("547d5c1b1e42bd0423a75781")},{customerInfo : 1}).customerInfo;
it will give you just the customerInfo :
/* 0 */
{
"birthday" : "1983-06-28",
"zipcode" : "12345",
"address" : "1, Main Street"
}
and so on.
This schema has the minimum hard round-trip and actually you are using mongodb document-based feature with best performance you can achive.

How to get paginated/sliced data of subdocument array in mongo collection?

I have a collection like this:
{
"_id" : ObjectId("51f4ad560364f5490ccebe26"),
"fiTpcs" : [
"uuid1",
"uuid2",
"uuid3",
"uuid4",
"uuid5"
],
"fiTpcsCnt" : 5
}
The list of fiTpcs is long and can go to hundreds later. When I retrieve my collection, I want to get a limited list of fiTpcs, say 20 at a time and fire separate requests to get subsequent data from fiTpcs. I just want to ensure that the queries don't get slow later when I have a lot more data. Is there a way to do it in mongodb? until now, I have been doing
db.userext.find({"_id" : ObjectId("51f4ad560364f5490ccebe26")}).pretty();
which always gets me the full fiTpcs array. I am using java driver with Spring and a solution using Spring/java would also be fine. Please note - if the solution requires mongo to scan through the whole fiTpcs array and then slice a part of it, it doesn't really add any performance benefits, that is not what I am looking for.
I may not understand your question in full depth, but seems like $slice is the droid your are looking for:
> db.page.find()
{ "_id" : ObjectId("51f4ad560364f5490ccebe26"), "fiTpcs" : [ "uuid1", "uuid2", "uuid3", "uuid4", "uuid5" ], "fiTpcsCnt" : 2 }
> db.page.find({}, {"fiTpcs" : {$slice : 3}})
{ "_id" : ObjectId("51f4ad560364f5490ccebe26"), "fiTpcs" : [ "uuid1", "uuid2", "uuid3" ], "fiTpcsCnt" : 2 }
> db.page.find({}, {"fiTpcs" : {$slice : [1,3]}})
{ "_id" : ObjectId("51f4ad560364f5490ccebe26"), "fiTpcs" : [ "uuid2", "uuid3", "uuid4" ], "fiTpcsCnt" : 2 }
After a couple of days of thinking/trying various options, this is what I did finally. I modified my document like this:
{
"_id" : ObjectId("51f4ad560364f5490ccebe26"),
"page" : 1, //1 is the default
"slug" : "some-unique-string-identifier"
"fiTpcs" : [
"uuid1", //these could be long text, like a long comment/essay
"uuid2",
"uuid3",
"uuid4",
"uuid5"
],
"fiTpcsCnt" : 5
}
I keep a "pageCount" and "totalFiTpcsCnt" in memcached. I have set MAX_FITPCSCNT = 500 (500 for now, experimental). When I create a new document of type userext, I set the page value to 1.
If I have to push a new object to fiTpcs array:
1) check if "totalFiTpcsCnt" is a multiple of 500. If yes, create a new document of type userext with the same slug, fiTpcsCnt as 0 and fiTpcs array as null.
2) update the last userext - query by slug and "pageCount", push to fiTpcs. Evict cache for "pageCount" and "totalFiTpcsCnt".
Whenever I need my userext document, I always take just the first page. This way I'll never need to query for more than 500 objects of type fiTpcs at a time and I will still have totalFiTpcsCnt always updated in memcached.