Hello! I'm trying to make two Firebase queries:
All children under basicInfo, for multiple users at once (query limited by 20 results). For example, in this query with the above database I will get all basicInfo sections for -Kwz1eoUMPym... and also for -Kwzhj4pzBJxbb...
All children under basic info where name == (some name).
I can't understand how to approach a name query under the AutoID key and also under basicInfo key, for both queries.
What is the most logical approach to this?
The Firebase Realtime Database is a NoSQL database, and it's only possible to filter by direct descendents of children in a list. Therefore, you will need to use a flatter data structure, and avoid splitting your fields into basicInfo, fullInfo:
users
|- userId
|- name: "John"
|- age: 17
|- gender: "m"
|- birthday: "10.10.2000"
...
With this approach, you can attach a listener to the users node and then filter by child values using the queryOrderedByChild and queryEqualToValue methods. For example, something like this in Swift:
Database.database().reference().child("users").queryOrderedByChild("name").queryEqualToValue("(some name)").observeSingleEventOfType(.Value, with: { snap in
// Do your thing with snap
})
With either of your structures, you can limit your results with the queryLimitedToFirst and queryLimitedToLast methods. For example, from the filtering data section of the documentation:
The following example demonstrates how example blogging app retrieves a list of the 100 most recent posts by all users:
// Last 100 posts, these are automatically the 100 most recent
// due to sorting by push() keys
let recentPostsQuery = (ref?.child("posts").queryLimited(toFirst: 100))!
For more details on filtering and limiting Firebase data in Swift/iOS, see the working with lists of data on iOS documentation.
Related
I want to create a simple SwiftUI app that uses FireStore and has a collection "recipes" and a collection "users" for user specific data. In the users collection, I want to add a document for every user that holds a sub-collection with the favourite recipes of the user.
On the client side, I am using something like this for the user data:
import FirebaseFirestoreSwift
import Foundation
struct Account: Codable {
#DocumentID var id: String?
var favoriteRecipes = Set<Recipe>()
}
Now if I write the document to firebase using the Codable support, it creates a Map for the set of recipes (which is fine I guess, I just want to have it another way).
So I obviously can handle the sub-collection "manually" and use it like any other stand-alone collection.
Nevertheless, I am wondering if there is some sort of "Best-Practice" for handling sub-collections in Firebase with Codable in Swift?
Thanks a lot!
There are two parts to your question. Let me try to answer them individually.
How to handle nested data structures in Firestore / Codable?
Any attribute on a Codable struct will be mapped against the respective attribute on a Firestore document (you have some influence over thus by using the CodingKeys enum - see this article.
Nested types will be mapped to dictionaries on the document, whereas arrays and other sequences will be mapped to arrays on the document.
In order to retrieve a sub-collection of a document, you will need to perform a separate fetch request. Firestore doesn't support fetching a nested tree of documents/sub-collections on the client. It's a different story on the server, though. Check out Renaud's article to learn more about this.
How to store user-specific data?
For any user=specific data, I would recommend one of the following two ways to structure your data:
Storing as a sub-collection
In this scenario, we have one top-level collection users, which contains documents for all your users (let Firestore auto-generate the document IDs for you, and store Firebase Auth's user ID as an attribute on the respective user document.
/(root)
+ users <-- (collection)
+ 0FABQ...RiGg <-- (user document)
- uid: "6ZPt...BLiK3fnl2" <-- (Firebase Auth user ID)
- name: "Johnny Appleseed" <-- (attribute)
+ recipes (collection) <-- (sub-collection)
+ A69EF...4EFA <-- (recipe document)
- name: "Scones" <-- (attribute)
+ FCED...12D5 <-- (another user document)
You can then use the user's ID (from Firebase Auth) to query all the user's recipes.
Storing as a top-level collection
In this scenario, we have two top-level collections: one for all your users, and another one for all the recipes. In order to distinguish a user's recipes, each recipe doc has a uid attribute which contains the respective user's user ID:
/(root)
+ users <-- (collection)
+ 0FABQ...RiGg <-- (user document)
- uid: "6ZPt...BLiK3fnl2" <-- (Firebase Auth user ID)
- name: "Johnny Appleseed" <-- (attribute)
+ FCED...12D5 <-- (another user document)
+ recipes (collection) <-- (collection)
+ A69EF...4EFA <-- (recipe document)
- uid: "6ZPt...BLiK3fnl2" <-- (Firebase Auth user ID)
- name: "Scones" <-- (attribute)
To retrieve a user's recipes, you query all recipes and filter for the ones that match the user ID of the currently signed in user.
To learn more about Firestore data modelling, check out the documentation, which also contains links to a number of really useful videos. Fireship also has a really good article about this.
I want to query data that is two levels down, however, would I still be able to retrieve data from its original node?
To explain better, my Firebase Database looks like:
posts
-192u3jdj0j9sj0
-message: haha this is funny (CAN I STILL GET THIS DATA)
-genre: comedy (CAN I STILL GET THIS DATA)
-author
-user: "jasonj"
-comment
-ajiwj2319j0jsf9d0jf
-comment: "lol"
-user: "David" (QUERY HERE****)
-jfaiwjfoj1ijifjojif
-comment: "so funny"
-user: "Toddy"
I essentially want to query by all of the comments David has posted. However, with how query works, can I still grab the original (message & genre) that was from "level 1"? Or would I have to restructure my data? Possibly rewriting the level 1 data under comment.
(End goal: something like Yahoo answers, where the user can see the questions he posted, as well as the questions to where he posted comments)
Below code works, but I'm not sure how to pull up level 1 data or if its even possible
ref = Database.database().reference().child("posts").child(myPost).child("comment")
var queryRef:DatabaseQuery
queryRef = ref.queryOrdered(byChild: "user").queryEqual(toValue: "David")
queryRef.observeSingleEvent(of: .value, with: { (snapshot) in
if snapshot.childrenCount > 0 {
Your current data structure makes it easy to find the comments for a specific post. It does not however make it easy to find the comments from a specific author. The reason for that is that Firebase Database queries treat your content as a flat list of nodes. The value you want to filter on, must be at a fixed path under each node.
To allow finding the comments from a specific author, you'll want to add an additional node where you keep that information. For example:
"authorComments": {
"David": {
"-192u3jdj0j9sj0_-ajiwj2319j0jsf9d0jf": true
},
"Toddy": {
"-192u3jdj0j9sj0_-jfaiwjfoj1ijifjojif": true
}
}
This structure is often known as a reverse index, and it allows you to easily find the comment paths (I used a _ as the separator of path segments above) for a specific user.
This sort of data duplication is quite common when using NoSQL databases, as you often have to modify/expand your data structure to allow the use-cases that your app needs.
Also see my answers here:
Firebase Query Double Nested
Firebase query if child of child contains a value
Not quite sure what the best practice is if I have two collections, a user collection and a picture collection - I do not want to embed all my pictures into my user collection.
My client searches for pictures under a certain criteria. Let's say he gets 50 pictures back from the search (i.e. one single mongodb query). Each picture is associated to one user. I want the user name displayed as well. I assume there is no way to do a single search performance wise on the user collection returning the names of each user for each picture, i.e. I would have to do 50 searches. Which means, I could only avoid this extra performance load by duplicating data (next to the user_id, also the user_name) in my pictures collection?
Same question the other way around. If my client searches for users and say 50 users are returned from the search through one single query. If I want the last associated picture + title also displayed next to the user data, I would again have to add that to the users collection, otherwise I assume I need to do 50 queries to return the picture data?
Lets say the schema for your picture collection is as such:
Picture Document
{
_id: Objectid(123),
url: 'img1.jpg',
title: 'img_one',
userId: Objectid(342)
}
1) Your picture query will return documents that look like the above. You don't have to make 50 calls to get the user associated with the images. You can simply make 1 other query to the Users Collection using the user ids taken from the picture documents like such:
db.users.find({_id: {$in[userid_1,user_id2,userid_3,...,userid_n]}})
You will receive an array of user documents with the user information. You'll have to handle their display on the client afterwards. At most you'll need 2 calls.
Alternatively
You could design the schema as such:
Picture Document
{
_id: Objectid(123),
url: 'img1.jpg',
title: 'img_one',
userId: Objectid(342),
user_name:"user associated"
}
If you design it this way. You would only require 1 call, but the username won't be in sync with user collection documents. For example lets say a user changes their name. A picture that was saved before may have the old user name.
2) You could design your User Collection as such:
User Document
{
_id: Objectid(342),
name: "Steve jobs",
last_assoc_img: {
img_id: Object(342)
url: 'img_one',
title: 'last image title
}
}
You could use the same principles as mentioned above.
Assuming that you have a user id associated with every user and you're also storing that id in the picture document, then your user <=> picture is a loosely coupled relationship.
In order to not have to make 50 separate calls, you can use the $in operator given that you are able to pull out those ids and put them into a list to run the second query. Your query will basically be in English: "Look at the collection, if it's in the list of ids, give it back to me."
If you intend on doing this a lot and intend for it to scale, I'd either recommend using a relational database or a NoSQL database that can handle joins to not force you into an embedded document schema.
Given an Meteor application that has multiple collections that need to be displayed together in a paged Facebook-style timeline view, I'm trying to decide on the best way to handle the publication of this data.
The requirements are as follows:
Documents from different collections may be intermingled in the timeline view.
The items should be sorted by a common field (the date, for example)
There should be a paged-display limit with a "Load More..." button
To solve this problem I can see two possible approaches...
Approach 1 - Overpublish
Currently I have different collections for each type of data. This poses a problem for the efficient publishing of the information that I need. For example, if the current display limit is 100 then I need to publish 100 elements of each type of collection in order to be sure of displaying the latest 100 elements of the screen.
An example may make this clearer. Assume that the timeline display shows results from collections A, B, C and D. Potentially only one of those collections may have any data, so to be sure that I have enough data to display 100 items I'll need to fetch 100 items from each collection. In that case, however, I could be fetching and sending 400 items instead!
That's really not good at all.
Then, on the client side, I need to handling merging these collections such that I show the documents in order, which probably isn't a trivial task.
Approach 2 - Combine all the collections
The second approach that occurs to me it to have one enormous server side collection of generic objects. That is, instead of having collections A, B, C, and D, I'd instead have a master collection M with a type field that describes the type of data held by the document.
This would allow me to trivially retrieve the the latest documents without over publishing.
However I'm not yet sure what the full repercussions of this approach would be, especially with packages such as aldeed:autoform and aldeed:simple-schema.
My questions are:
Does anyone here have and experience with these two approaches? If
so, what other issues should I be aware of?
Can anyone here suggest
an alternative approach?
I'd use the second approach, but do not put everything in there...
What I mean is that, for your timeline you need events, so you'd create an events collection that stores the basic information for each event (date, owner_id, etc) you'd also add the type of event and id to match another collection. So you'll keep your events just small enough to publish all is needed to then grab more details if there is a need.
You could then, either just publish your events, or publish the cursors of the other collections at the same time using the _id's to not over-publish. That event collection will become very handy for matching documents like if the user wants to see what in his timeline is related to user X or city Y...
I hope it helps you out.
I finally come up with a completely different approach.
I've created a server publication that returns the list of items ids and types to be displayed. The client can then fetch these from the relevant collections.
This allows me to maintain separate collections for each type, thus avoiding issues related to trying to maintain a Master collection type. Our data-model integrity is preserved.
At the same time I don't have to over-publish the data to the client. The workload on the server to calculate the ID list is minimal, and outweighs the disadvantages of the other two approaches by quite a long way in my opinion.
The basic publication looks like this (in Coffeescript):
Meteor.publish 'timeline', (options, limit) ->
check options, Object
check limit, Match.Optional Number
sub = this
limit = Math.min limit ? 10, 200
# We use the peerlibrary:reactive-mongo to enable meteor reactivity on the server
#ids = {}
tracker = Tracker.autorun =>
# Run a find operation on the collections that can be displayed in the timeline,
# and add the ids to an array
collections = ['A', 'B']
items = []
for collectionName in collections
collection = Mongo.Collection.get collectionName
collection.find({}, { fields: { updatedOn: 1 }, limit: limit, sort: { updatedOn: -1 }}).forEach (item) ->
item.collection = collectionName
items.push item
# Sort the array and crop it to the required length
items = items.sort (a,b) -> new Date(a.date) - new Date(b.date)
items = items[0...limit]
newIds = {}
# Add/Remove the ids from the 'timeline' collection
for doc in items
id = doc._id
newIds[id] = true
# Add this id to the publication if we didn't have it before
if not #ids[id]?
#ids[id] = moment doc.updatedOn
sub.added 'timeline', id, { collection: doc.collection, docId: id, updatedOn: doc.updatedOn }
# If the update time has changed then it needs republishing
else if not moment(doc.updatedOn).isSame #ids[id]
#ids[id] = doc.updatedOn
sub.changed 'timeline', id, { collection: doc.collection, docId: id, updatedOn: doc.updatedOn }
# Check for items that are no longer in the result
for id of #ids
if not newIds[id]?
sub.removed 'timeline', id
delete #ids[id]
sub.onStop ->
tracker.stop()
sub.ready()
Note that I'm using peerlibrary:reactive-publish for the server-side autorun.
The queries fetch just the latest ids from each collection, then it places them into a single array, sorts them by date and crops the array length to the current limit.
The resulting ids are then added to the timeline collection, which provides for a reactive solution on the client.
On the client it's a simply a matter of subscripting to this collection, then subscribing the individual item subscriptions themselves. Something like this:
Template.timelinePage.onCreated ->
#autorun =>
#limit = parseInt(Router.current().params['limit']) || 10
sub = #subscribe 'timeline', {}, #limit
if sub.ready()
items = Timeline.find().fetch()
As = _.pluck _.where(items, { collection: 'a' }), 'docId'
#aSub = #subscribe 'a', { _id: { $in: As }}
Bs = _.pluck _.where(items, { collection: 'b' }), 'docId'
#bSub = #subscribe 'b', { _id: { $in: Bs }}
Finally, the template can iterate one the timeline subscription and display the appropriate item based on its type.
I have a posts collection which stores posts related info and author information. This is a nested tree.
Then I have a postrating collection which stores which user has rated a particular post up or down.
When a request is made to get a nested tree for a particular post, I also need to return if the current user has voted, and if yes, up or down on each of the post being returned.
In SQL this would be something like "posts.*, postrating.vote from posts join postrating on postID and postrating.memberID=currentUser".
I know MongoDB does not support joins. What are my options with MongoDB?
use map reduce - performance for a simple query?
in the post document store the ratings - BSON size limit?
Get list of all required posts. Get list of all votes by current user. Loop on posts and if user has voted add that to output?
Is there any other way? Can this be done using aggregation?
NOTE: I started on MongoDB last week.
In MongoDB, the simplest way is probably to handle this with application-side logic and not to try this in a single query. There are many ways to structure your data, but here's one possibility:
user_document = {
name : "User1",
postsIhaveLiked : [ "post1", "post2" ... ]
}
post_document = {
postID : "post1",
content : "my awesome blog post"
}
With this structure, you would first query for the user's user_document. Then, for each post returned, you could check if the post's postID is in that user's "postsIhaveLiked" list.
The main idea with this is that you get your data in two steps, not one. This is different from a join, but based on the same underlying idea of using one key (in this case, the postID) to relate two different pieces of data.
In general, try to avoid using map-reduce for performance reasons. And for this simple use case, aggregation is not what you want.