The problem I'm dealing with is performance issues when needing to perform multiple Firestore queries in order to reach a result similar to to joins in SQL.
In my Flutter app, which uses Firebase and Firestore, I use Cloud Functions for serving my REST API. My API is the only thing that actually interacts with the Firestore database, not the Flutter app itself. I was always taught to design apps like this (not allowing the client to interact directly with the database) and so continued with that approach.
A potential solution I've seen are the official Firestore plugins for Flutter and have seen people designing their clients to interact directly with Firebase, however I haven't found any official documentation explicitly saying this design is preferred or even ok, and since it goes against when have been taught I haven't tried it yet. Can anyone confirm this or point me to documentation confirming this?
Edit
Perhaps it's worth noting that the reasons I've been told not to have the client connect directly to the database are 1. Security and 2. It would require business logic for managing data to be in the client and that should be handled server-side.
Related
I am building flutter mobile app that is intensively using firebase services and firestore.
in app start, the app communicates with firestore to retrieve some basic keys and paramaters that app uses in different services, like APIs Keys, IDs, .. etc.
would like to understand if this approach is secure enough? or there is a possibility that communication (firestore query) to be hacked somehow and the keys are stolen?
Note: I am using simple firebase rule that allow read and write if user is signed in using Firebase Authentication
I can indeed hardcode these keys in the app code, however I prefered this database approach to give myself the chance to change these keys if it is changed by the services providers for any reason.
any answers or links are much apprecaited.
You should assume that any value used inside your client-side application can be found by a malicious user and used for their own purposes.
Once someone has those keys, they can call the APIs that require them differently than what your own application code does, unless you use some other means to prevent this such as Firebase's security rules and App Check.
When using security rules, the best way to prevent somebody from doing something different from your application's use-cases is to encode those use-cases in the security rules too. So instead of just requiring someone to be signed in, expand your rules to validate that only the operations that your own code requires are allowed. Use-case by use-case lock it down, until your cod and rules cover the same set of use-cases.
Also see:
Is it safe to expose Firebase apiKey to the public?
google FireStore security hack from web console
I came across this link trying to figure out how to access the REST api of my firebase database and stream it to my app. I can get/send data manually, but for my app I can't use the firebase api (because it is an app clip), so I'm trying to do it with native tools and rest, but the issue I'm running into is getting the configuration right (per this info) and actually parsing the incoming info and updating my app. I'm so lost and could use some help just figuring out how to make the code in the sample work, especially since I'm using swift ui as well.
Any help is appreciated, thanks!
Apparently App Clips do not allow sockets connections, so (while Google rewrites their sdk to comply), I migrated my database to Realtime Database (since it was a better fit for us usage-wise), and have been using standard rest for sending and getting data, and for the "listener" functionality I have been using IKEventSource that listens to server-side events and updates them accordingly in my app clip. Seems to be a pretty good workaround so far.
Background:
I'm building an API service app. The app is just like any other, you send an HTTP request and receive a response. This seems simple up until I start thinking about user registration, payments, authentication, logging and so on.
Application:
tl;dr simple app diagram
Endpoints listening for HTTP requests and doing all the request related work. This is the core of the service, what the service user would use this app for. Directly not accessible to the end user (unless somehow it knows the url). Python flask server, deployed on google cloud RUN.
API gateway acting like a proxy and a single access point forwarding the requests to the endpoints. This is the service access point for the end users. This part will also be responsible for authentication, limitations, logging and tracking the use of the API endpoints. Python flask server, deployed on google cloud RUN.
Website including documentation, demo and show off of API calls through API gateway, registration, payment (thinking of Stripe) etc. VueJS app on NodeJS server on google cloud compute VM.
Database storing credentials of registered users, payment information and auth keys. Not implemented yet.
Problems:
Is this architecture proper? What could be done differently or improved? How could I further simplify all the interactions between separate parts of the app? Am I not missing any essential parts?
Haven't yet implemented the database part and I'm not sure what should I
use? There are plenty of options on google cloud. Also I could go with something simple and just install a DB with http/JSON interface on google cloud compute VM. How do I chose the DB? Given such an app, what would be the best choice?
Please recommend literature/blogs/other sources of info on similar app
architecture for new developers not familiar with it?
This is pretty open ended, but here are some general comments:
Think about how your UI will work. Are you setting up a static app served directly from cloud storage or do you need something rendered on the server? Personally I prefer separating UI from API when I can but you need to be aware of things like search engine optimization. Even if you need to render some content dynamically your site can still be static. Take a look at static site generators like Gatsby. I haven't had to implement a server rendered UI in years and that makes me happy.
API gateway might be fine, but you don't really need it for anything. It might be simpler to start without it and concentrate on what actually matters. If your APIs are being called by an external client you can't trust the calls anyways and any API key you might be using will be exposed. I'd say don't worry about it for a single app. That being said, if you definitely want to use a GW then use one, just be aware that it is mostly a glorified proxy and not some core part of your architecture.
Make sure your API implementations don't store any local state so you can rely on Cloud Run scaling your services up and down. Definitely don't ever store state directly inside your containers. If you need state on the server it needs to be in some external data store.
Use JWTs or an external IDM (that will generate JWTs) for authentication. Keep session data on the client side as much as possible and pass the JWT in every API call to authenticate the caller. If you are implementing login on your own the only APIs you need to expose without tokens are for auth and password recovery, which you can separate into their own service.
Database selection depends on how well you understand your processes, how transactional your services are and your existing skillset. Overall I would use what you are comfortable with, you can probably succeed with a lot of things. Certain NoSQL flavors can seem simple on the surface but if you don't have a clear understanding on the types of queries you need to run they can get tedious to work with. Generally you should stick to relational databases for OLAP style implementations and consider NoSQL for OLTP. Personally I like MongoDB and it is very popular, probably because it sort of sits in the middle of the pack which makes it fit a lot of applications. Using MongoDB also makes you cloud agnostic since it is available on every platform. Using platform specific database flavors can lock you down to a specific vendor.
Whatever you do, don't start installing things on VMs. You can be almost 100% sure you are doing it wrong if this comes up. Remember, the services you consume don't all have to be managed by Google or even run on GCP. You can get MongoDB capacity directly from MongoDB who manage it on your behalf on all of the Big3 cloud vendors.
At least think about the long term, even if you don't necessarily need to have it impact your architecture right now. If you are expecting your app to be up for years try to make it more platform agnostic than less. This might mean sticking away from some really platform specific serverless features that will force you to jump a couple of extra hoops. If you are using Cloud Run you are using containers which already makes your app pretty portable, don't lock it to one platform by using a lot of platform specific features. That being said, don't stay away from them either. You should always go for the low hanging fruit, so don't try to avoid using things like secrets manager etc. If your app has a short lifespan and you need really fast time to market then don't worry about it.
Just my 2c, what you are doing is very generic and can be done in a lot of different ways.
I am considering using Firebase/Firestore and Mongodb together for data storage. The idea is to use Firebase to store data that users will interact with on mobile and use MongoDB for more backend stuff or web content that won't be accessible on the app.
I've found a lot of people switching between the two, but very few topics discuss using both. Is there any reasons why this isn't done more often?
I have a string for eg: "My name is John" stored in Firebase.
How would I query firebase so I can find all the posts in Firebase that have "John" ?
I can search for the first term in a string now using:
DataService.dataService.BASE_REF.child("Posts").child(selectedComment.commentKey).queryOrderedByChild("userComment").queryStartingAtValue(comment).queryEndingAtValue(comment+"\u{F8FF}").observeSingleEventOfType(.Value, withBlock: { (snapshot) in
where comment = "My"
I read about using Elastic search with Firebase but wanted to check if there was an easier way in Firebase before I looked at ElasticSearch/Flashlight for Firebase,
Unfortunately, Firebase doesn't support searching thru content like that (in any language SDK). From a Google Groups Post in July '16:
As a company that understands search, we're also a company that
understands using the best tool for the job. For fuzzy matching and
contains, a NoSQL, realtime data store isn't the correct tool--these
queries would be slow and scale poorly. BigQuery or ElasticSearch are
the right tool for providing useful results in a scalable and robust
manner.
Right now, this involves deploying a small node script to sync your
search results with the realtime data, as explained in the article
with the sample Flashlight lib. In the future, it will become more
"effortless" as we add integrations between Firebase and Cloud
products, particularly Cloud Functions and BigQuery interoperability.
BigQuery is, as I understand it, not specifically designed for user-facing search.
Elasticsearch (specifically, the Firebase plugin Flashlight) is a potential solution, but as you alluded to, it's an incredible amount of overhead (deploying/managing or renting an ES cluster, configuring the plugin, etc.). If content search is an important enough part of your app to justify that time/$, you may want to consider solutions beyond Firebase for your database needs, as it's by far one of the service's weakest areas.
In my opinion, you have a few options beyond Flashlight:
Algolia, a Search-as-a-service provider, does offer integration with Firebase, but I've never used it & so can't offer much more than to say that it exists.
Another alternative might be maintaining a collection of documents you want to search on another service, like AWS Cloud Search
Depending on the stage of your project & your needs, consider other Backends-as-a-Service that support more in terms of querying. E.g., GraphQL-as-a-service backends, like Scaphold.io, Graph.cool, and Reindex are all built on SQL databases, and (I believe) all support multiple types of querying.