Given Firebase supports realtime push for web, android and iOS, I was tempted to try it out like pub/sub type push based system. Does anyone know if the firebase works and scale to large userbase? for example million+ users of chat conversations and users accessing that data simultaneously, what would be response time as users grow.
Firebase free and basic plans limit connections to 100. Their higher end plans limit connections to 10,000. That's done to prevent abuse.
From the Firebase website in case it was overlooked:
Those restrictions can be lifted though, and their Enterprise solutions can scale to millions of connections and terabytes of data
I cannot speak from personal experience with a million users but given the performance we have seen, it would be scalable to handle that.
Based on the scope of your project, I would engage them directly at support#firebase.com with your specific requirements.
Related
my APP Is Really Depends on push Notifications, for examples iF the app has two actors, owner, and user, if the owner posts something, the user must be notified, based on their locations and Engagement in the app, which means the System first searches the nearest user and notify him, Notification is performed based on the location of the user, the user who is found in the nearest location is notified first, so What I want to ask you is, is it possible to achieve this without using firebase Cloud functions, Since firebase Cloud functions are NOT Free at this time. it asks me Billing account but the payment method is not available in my country
Apologies for my previous reply, I misunderstood the situation,
Also another solution besides one signal, you can use FCM on your own server, via FirebaseAdmin SDK. If you have a NodeJS server, a simple one, you can perform all firebase functions on it. In the end, it's not an absolute necessity to use cloud functions, it's just more convenient for a simple task or a few functions than spinning up your own server just for notifications. I would advise to go with this solution, and you will have access to cloud firestore via the admin SDK, you'll feel right at home.
Firebase cloud functions requires billing info, but you aren't charged. You have 2 MILLION free api calls per month. Then every extra 1 million calls cost less than half a dollar.
Do you know what 2 Million API invocations per month means? That's 32 THOUSAND API calls per day. If your app is generating that much traffic, trust me, you will not be worrying about these costs.
You can use OneSignal if you want, but it's more work for you, and it's outside the Firebase ecosystem.
Go with cloud functions, when you are reaching the limit of the free tier of 2Million monthly API requests, you can start looking for investors in your app.
I am getting very high counts of Entity Writes in my firestore database.
Write permission in most of the paths are restricted, done from back-end server using admin SDK. Only a very few paths have write access- specifically only to the users who are (authenticated & registered & joined and approved in a specific group), so even though the ways to abuse are apparently thin, yet hard to specifically identify.
Only way I see- is to execute Cloud Functions on every write, and have the function log the paths somewhere to analyze. But that introduces further costs and complexity.
Is there any way/recommendation to monitor/profile where (i.e.- path) and who (UID or any identity) are performing the writes? There are tools to do such for RTDB, bu't can't find anything for Firestore.
I am also wondering if there is any way to restrict ip/users automatically in case of abuse (i.e.- high rate of read/write)?
What I'm currently doing is going to firestore console => menu usage => view usage
and I see something like this:
It's not the same as the profiler, but better than nothing.
I'm also keeping an eye on the video on the link below to see if someone provides an answer. People are asking for the profiler too.
https://www.youtube.com/watch?v=9CObBsjk6Tc
I have some concerns about using Firestore directly from mobile clients. I saw that with Firestore, mobile app is controls to db directly. There is only db side control is rules. However I am not yet understand how can I resolve my concerns. I worrying because with decompiling app or any other ways maybe someone can access maliciously.
To resolve my concerns and improve the security I want to:
Limit access per time for user. For example if some one writes or
reads db 30 second ago I want to block their access until 1 minute.
Especially it is important for writing.
I want to have the only document owner write their documents and
block others to write. For do this I don't want to store owner id in
same document because if I put this information to there, readers can
receive this information and maybe some way they can write request
with this information.
In summary, I want to prevent malicious and excessive requests exceeding limits. I want to avoid the risks in db side. How can I resolve my concerns in the two above matter?
Scenerio:
Think that there is harmful someone with name X. X knows how to
decompile mobile app or maybe knows how to request my application's
firebase Firestore account (maybe watched communication I don't know).
X wants to harm my application. First X watching communication and if
we give owner userid in document X receives needed information so
sending request with changing his/her userid or auth.uuid. Secondly X wants to
disable firebase of my application. For do this X sends too much
requests. The limit is overs due to too many requests. Note that: I am
using Firestore directly with mobile application. There is no web
service to communicate.
How can avoid this scenario?
I will build a dashboard system for my apps, where a page will have several widgets that draw charts, tables and glyphs representing potentially unrelated data.
The client will be HTML5 and I can push for only modern web browser.
My big problem is what backend use for this. I want to store "tables" for use in the charts and in real-time update the widgets.
For example, a invoicing widget will show how much $$ have been collected today. In the "table" will have a row for each total of the invoice:
inv = 1; total = 50
Total: 50
and the widget will draw that. When new data is pushed:
inv = 2; total = 100
Total: 150
The widget will show in realtime the total to the end-user.
The data is private for the user company. Eventually I will need to purge too old data (ie: I only need to keep as much data is necessary to proper evaluation of the info need for the end-user. For example, only keep 1 month of invoicing totals).
I'm thinking in use something like http://www.firebase.com/ or http://pusher.com/ but I suspect only solve the "notify in realtime" part of the equation. As far as I understand, they not let me get past data (ie: If the data is update in the weekend and the user open his dashboard to see what happened)
Then I see http://derbyjs.com/ and the possibility to use mongodb.
I wonder which backend/platform will bring me closer to the build of this system. I have experience with python/django/.net/postgress but could accept the use of something else if solve best this kind of app behavior.
Firebase offers both the "notify in relatime" part that you mention, as well as persistent data storage. Take a look at the tutorial, which walks you through building a real-time persisted chat app (the past chat messages are stored in Firebase and are sent back to the client every time you reload). And you can do much more complicated stuff like the real-time charts / widgets that you mention as well.
The big limitation with Firebase right now is that we're in closed beta and the data is currently unprotected (anybody can read and write your data). The security features are coming soon though.
Some other backend platforms you may want to evaluate are: Meteor and Simperium. Firebase and Simperium are cloud services where your data is stored in the cloud and you don't have to manage any servers of your own, while Meteor and DerbyJS are platforms that you have to install and run on your own server.
I would recommend signalR. It's amazing and you can literally do anything with it. Check it out: www.signalr.net and if you have any problems simply go to www.jabbr.net You will find a very helpful community there. I implemented a notification mechanism similar to facebook together with real time monitoring and a small chat in the same web site.
I'm looking to re-code an application to better handle spikes in tweets. I'm moving to Heroku and MongoDB (either MongoLab or MongoHQ) for the database solution.
During certain news events, tweet volume might spike to 15,000 / second. Typically with each tweet, I parse the tweet and store various pieces of data such as user data, etc. My idea is to store the raw tweets in a separate collection, and have a separate process grab raw tweets and parse them. The goal here is when there is a massive spike in tweets, my application isn't trying to parse all of these, but is essentially backlogging the raw tweets in another collection. As the volume slows, the process can take care of the backlog over time.
My question is three fold:
Can MongoDB handle this type of volume with regards to inserts into a collection at a rate of 15,000 tweets per second?
Any idea on the better setup: MongoHQ or MongoLab?
Any feedback on the overall setup?
Thanks!
The write volume that it will handle depends on lots of factors - hardware, indexes, size of each document, etc. Your best bet is to test it in the environment you're planning to use. If the demands of the write load exceed the capacity of a single mongo server, you can always use just multiple shards.
They are very similar, but there are some differences in pricing and the actual site design has a bunch of differences. There's a thread of discussion about it here: https://webmasters.stackexchange.com/questions/20782/mongodb-hosting-mongolab-vs-mongohq-vs-mongomachine
Overall it seems to make sense. Sounds like you will probably want to flesh out some details about how you will be processing the backlog. Will you be polling it by querying periodically, deleting tweets from the backlog as it processes them, etc.
Completely agree on the need to test this. In general, mongo can handle that many writes, but in practice it depends on the size of your set up, other operations, indexes, etc.
I had to do a similar approach for collecting tons of metrics data. I used a lightweight event-machine process to accept incoming requests in parallel, and store them in a simple format, then another process would take those requests and send them up to a central server. The main goal was to make sure no data was lost if the central server was down, but it also allowed me to put in some throttling logic so that the spikes in data wouldn't overwhelm the system.
I'd be interested to see how this works out for you price-wise, vs. a vps like linode. (I'm a huge Heroku fan, but with certain architectures it can get pricey quickly)