I am having some difficulty structuring my data so that I can benefit from reactivity in Meteor. Mainly nesting arrays of objects makes queries tricky.
The three main views I am trying to project this data onto are
waiter: shows order for one table, each persons meal (items nested, essentially what I have below)
kitchen manager: columns of orders by table (only needs table, note, and the items)
cook: columns of items, by category where started=true (only need item info)
Currently I have a meteor collection of order objects like this:
Order {
table: "1",
waiter: "neil",
note: "note from kitchen",
meals: [
{
seat: "1",
items: [ {n: "potato", category: "fryer", started: false },
{n: "water", category: "drink" }
]
},
{
seat: "2",
items: [ {n: "water", category: "drink" } ]
},
]
}
Is there any way to query inside the nested array and apply some projection, or do I need to look at an entirely different data model?
Assuming you're building this app for one restaurant, there shouldn't be many active orders at any given time—presumably you don't have thousands of tables. Even if you want to keep the orders in the database after they're served, you could add a field active to separate out the current ones.
Then your query is simple: activeOrders = Orders.find({active: true}).fetch(). The fetch returns an array, which you could loop through several times for each of your views, using nested if and for loops as necessary to dig down into the child objects. See also Underscore's _.pluck. You don't need to get everything right with some complicated Mongo query, and in fact your app will run faster if you query once and process the results however many times you need to.
Related
Basically we have four tables that are joined: Malls, Stores, Brands, and Categories.
Malls can have many Stores.
Each Store is linked to a Brand.
Each Brand has many Categories.
e.g. Mall A has 2 "McDonald's Cafes", each belonging to the Brand "McDonald's". "McDonald's" Brand has "Fast Food" and "Breakfast" as Categories.
We're trying to figure out how to display all the Categories that exist within Mall A.
e.g. Mall A has "Fast Food" and "Breakfast" categories, based on the "McDonald's" stores.
Ideally this would be a query so that the Categories are updated automatically.
We've tried querying for all Stores within a Mall, and then finding the Categories via the Store-Brand join, then reducing duplicates. But some Malls have more than 700 Stores, so this process is quite expensive in terms of querying and processing the data.
I managed to figure it out! Using Sequelize, my query was as follows:
postgres.BrandsCategories.findAndCountAll({
limit,
offset,
include: [
{
model: postgres.Brands,
as: "brands",
include: [
{
model: postgres.Stores,
as: "stores",
where: {
mallId: parent.dataValues.id
}
}
]
}
]
})
Fellow programmers.
Is it considered as a bad practice to use such MongoDb model:
{
companyId: '',
companyName: '',
companyDivisions: [
{
divisionId: '',
divisionName: '',
divisionDepartments: [
{
departmentId: '',
departmentName: ''
},
...
]
},
...
],
},
...
Because right now it's getting complicated to update certain departments.
Thanks.
I don't think this is a bad practice generally speaking. If your model resembles this data structure it is a good choice storing data this way, leveraging a document database. You can naturally handle data and most likely you have a direct map onto your data model.
Another choice would be to have three different collections:
companies;
divisions;
departements.
However, in this case you would end up storing data as you would do in a relational database. Thus, more than a general rule, it is a matter of data model and expected query profile on your database.
Edit: using MongoDb 3.6+
Using your document oriented approach, a single department can be granularly updated using the following update:
db.companies.findAndModify(
{
query: {
"companyId": "yourCompanyId"
},
update: {
$set : { "companyDivisions.$[element1].divisionDepartments.$[element2].divisioneName": "yourNewName" }
},
arrayFilters: [
{ "element1.divisionId": "yourDivisioneId" },
{ "element2.departmentId": "yourDepartementId" }
]
});
This update uses the new powerful filtered positional operator feature introduced by MongoDB v3.6. The $[<identifier>] syntax allows to select an array entry based on a specific condition expressed in the arrayFilters option of the db.collection.findAndModify() method.
Note that in case the condition matches multiple array items, the update affects all such items, thus allowing for multiple updates as well.
Furthermore, note that I would apply such an optimization only in case of need, since premature optimization is the root of all evil. (D. Knuth).
I am building a game app using Firebase and Swift, and store my data like this.
players: {
"category": {
"playerId": {
name: "Joe",
score: 3
}
}
}
There are two queries I need to make based on the score:
Get the top 5 players from a single category.
Get the overall top 100 players of the game.
I have no problem getting the data for the first query, using
ref.child("players").child(category).queryOrdered(byChild: "score").queryLimited(toLast:5)
But I am having trouble figuring out the second query. How can I do this efficiently?
You'll either need to add an additional data structure to allow the second query (which is quite common in NoSQL databases) or change your current structure to allow both queries.
The latter could be accomplished by turning the data into a single flat list, with some additional synthetic properties. E.g.
players: {
"playerId": {
category: "category1"
name: "Joe",
score: 3,
category_score: "category1_3"
}
}
With the new additional properties category and category_score, you can get the top scorers for a category with:
ref.child("players")
.queryOrdered(byChild: "category_score")
.queryStarting(atValue:"category1_")
.queryEnding(atValue("category1_~")
.queryLimited(toLast:5)
And the top 100 scores overall with:
ref.child("players")
.queryOrdered(byChild: "score")
.queryLimited(toLast:100)
For more on this, see my answer here: Firebase Database queries can only order/filter on a single property. In many cases it is possible to combine the values you want to filter on into a single (synthetic) property. For an example of this and other approaches, see my answer here: http://stackoverflow.com/questions/26700924/query-based-on-multiple-where-clauses-in-firebase.
I'm making a game; players form leagues and make competing predictions. A league looks like this:
{ leagueName: "Premier League",
players:[
{name: "Goodie", secretPrediction: "abc"},
{name: "Baddie", secretPrediction: "def"}
] }
For each player, I need to publish to the client the names of all the players in the league, but only their own secret prediction. So from above, if Goodie is logged in, the document on mini-mongo should be:
{ leagueName: "Premier League",
players:[
{name: "Goodie", secretPrediction: "abc"},
{name: "Baddie"}
] }
To do this, I have two publications - one to get the whole League document but excluding ALL secret predictions, and one to get the current player's subdocument in the players array including her secret prediction. My publications are:
// Publish whole players array excluding secretPrediction
Leagues.find({"players.name": "Goodie"}, {fields: {"players.secretPrediction": 0}})
// Publish the whole Goodie item in the players array and nothing else
Leagues.find({"players.name": "Goodie"}, {fields: {players: {$elemMatch: {name: "Goodie"}}}})
The problem is that when I subscribe to both the above publications, I don't get the document I want - the secret prediction is excluded even with the second publication. (On their own, the publications behave as expected, it's only when I subscribe to both.)
Now, I understand from this answer that the two publications should be "merged" on the client
Down to the level of top level fields, Meteor takes care to perform a set union among documents, such that subscriptions can overlap - publish functions that ship different top level fields to the client work side by side and on the client, the document in the collection will be the union of the two sets of fields.
So I have two main questions (and well done / thanks for making it this far!):
Is the union of documents not happening because I'm not dealing with top level fields? Is there a way around this?
Am I going about this completely the wrong way? Is there a better way to get the results I want?
Yes, the merging multiple subscriptions of Meteor only works with the top level fields, it is mentioned in the Meteor docs: Meteor.subscribe
I can not say that you are heading the wrong direction, this really depends on your situation, what features you want to help. Only speak of myself, I would decouple the above collection to two separate collections. Because players may join many leagues and leagues may have many players, so their relation is many-to-many (n-n). For this kind of relation, we should split them to two collections and use an associative table to reflect their relation
So in your case, I would have:
League collection:
[{
_id: 'league1',
name: 'League 1',
// ...
}]
Player collection:
[{
_id: 'player1',
name: 'Player 1',
// ...
}]
League2Player collection:
[{
_id: 'league1palyer1',
playerId: 'player1',
leagueId: 'league1',
secretPrediction: 'abc',
// ...
}]
Could you instead rearrange the data document so that you can use a single query e.g.
{ leagueName: "Premier League",
players:[
{name: "Goodie"},
{name: "Baddie"}
]
playerPredictions:[
{name: "Goodie", secretPrediction: "abc"},
{name: "Baddie", secretPrediction: "def"}
]
}
That way it would be possible in a single query to return all the players and only the playerPrediction for the given person.
I have two collections - shoppers (everyone in shop on a given day) and beach-goers (everyone on beach on a given day). There are entries for each day, and person can be on a beach, or shopping or doing both, or doing neither on any day. I want to now do query - all shoppers in last 7 days who did not go to beach.
I am new to Mongo, so it might be that my schema design is not appropriate for nosql DBs. I saw similar questions around join and in most cases it was suggested to denormalize. So one solution, I could think of is to create collection - activity, index on date, embed actions of user. So something like
{
user_id
date
actions {
[action_type, ..]
}
}
Insertion now becomes costly, as now I will have to query before insert.
A few of suggestions.
Figure out all the queries you'll be running, and all the types of data you will need to store. For example, do you expect to add activities in the future or will beach and shop be all?
Consider how many writes vs. reads you will have and which has to be faster.
Determine how your documents will grow over time to make sure your schema is scalable in the long term.
Here is one possible approach, if you will only have these two activities ever. One record per user per day.
{ user: "user1",
date: "2012-12-01",
shopped: 0,
beached: 1
}
Now your query becomes even simpler, whether you have two or ten activities.
When new activity comes in you always have to update the correct record based on it.
If you were thinking you could just append a record to your collection indicating user, date, activity then your inserts are much faster but your queries now have to do a LOT of work querying for both users, dates and activities.
With proposed schema, here is the insert/update statement:
db.coll.update({"user":"username", "date": "somedate"}, {"shopped":{$inc:1}}, true)
What that's saying is: "for username on somedate increment their shopped attribute by 1 and create it if it doesn't exist aka "upsert" (that's the last 'true' argument).
Here is the query for all users on a particular day who did activity1 more than once but didn't do any of activity2.
db.coll.find({"date":"somedate","shopped":0,"danced":{$gt:1}})
Be wary of picking a schema where a single document can have continuous and unbounded growth.
For example, storing everything in a users collection where the array of dates and activities keeps growing will run into this problem. See the highlighted section here for explanation of this - and keep in mind that large documents will keep getting into your working data set and if they are huge and have a lot of useless (old) data in them, that will hurt the performance of your application, as will fragmentation of data on disk.
Remember, you don't have to put all the data into a single collection. It may be best to have a users collection with a fixed set of attributes of that user where you track how many friends they have or other semi-stable information about them and also have a user_activity collection where you add records for each day per user what activities they did. The amount or normalizing or denormalizing of your data is very tightly coupled to the types of queries you will be running on it, which is why figure out what those are is the first suggestion I made.
Insertion now becomes costly, as now I will have to query before insert.
Keep in mind that even with RDBMS, insertion can be (relatively) costly when there are indices in place on the table (ie, usually). I don't think using embedded documents in Mongo is much different in this respect.
For the query, as Asya Kamsky suggest you can use the $nin operator to find everyone who didn't go to the beach. Eg:
db.people.find({
actions: { $nin: ["beach"] }
});
Using embedded documents probably isn't the best approach in this case though. I think the best would be to have a "flat" activities collection with documents like this:
{
user_id
date
action
}
Then you could run a query like this:
var start = new Date(2012, 6, 3);
var end = new Date(2012, 5, 27);
db.activities.find({
date: {$gte: start, $lt: end },
action: { $in: ["beach", "shopping" ] }
});
The last step would be on your client driver, to find user ids where records exist for "shopping", but not for "beach" activities.
One possible structure is to use an embedded array of documents (a users collection):
{
user_id: 1234,
actions: [
{ action_type: "beach", date: "6/1/2012" },
{ action_type: "shopping", date: "6/2/2012" }
]
},
{ another user }
Then you can do a query like this, using $elemMatch to find users matching certain criteria (in this case, people who went shopping in the last three days:
var start = new Date(2012, 6, 1);
db.people.find( {
actions : {
$elemMatch : {
action_type : { $in: ["shopping"] },
date : { $gt : start }
}
}
});
Expanding on this, you can use the $and operator to find all people went shopping, but did not go to the beach in the past three days:
var start = new Date(2012, 6, 1);
db.people.find( {
$and: [
actions : {
$elemMatch : {
action_type : { $in: ["shopping"] },
date : { $gt : start }
}
},
actions : {
$not: {
$elemMatch : {
action_type : { $in: ["beach"] },
date : { $gt : start }
}
}
}
]
});