I've used Fabric for quite a few applications, however I was curious about the performance when a single application submits potentially hundreds of events per minute.
For this example I'm going to be using a Pedometer application, in which I would want to keep track of the amount of steps users are taking in my application. Considering the average user walks 100 steps per minute, I wouldn't want the application to be sending several dozen updates to the server.
How would Fabric handle this, would it just tell the server "Hey, there were 273 step events in the last 5 minutes with this meta deta" or would it sent 273 step events.
Pedometer applications typically run in the background so how would we get data to Fabric without the user opening the application
Great question! Todd from Fabric. These get batched and sent at time intervals and also certain events (like installs) trigger an upload of the queued events data. You can watch our traffic in Xcode debugger if you are curious about the specifics for your app.
Related
I am noticing latency in REST data the first time I visit a web site that is being served via Azure Mobile Services. Is there a cache or timeout of a connection after a set amount of time, because I am worried about user experience while waiting 7-8 seconds for the data to load (and there is not a lot of data, as I am testing 10 records returned). Once the first connection is made, subsequent visits appear to load quickly... but if I don't visit the site for a while, we are back to 7-8 seconds on first load.
Reason: The reason for this latency is the "shared" mode. When the first call to the service is made, it performs a "cold start" (initializing and starting the virtual server etc)
As you described in your question, after a while when the service is not used, it is put into the "sleep mode" again.
Solution: If you do not want this waiting-time, you can set your service to "reserved" mode, which forces the service to be active all time even when you do not access it for a while. But be aware that this requires you to pay some extra fees.
I'm working on a tracking site that that tracks a player's levels for a game from day to day.
It's going to have to process around 25,000 API calls once a day. I'd like to be able to get this done in 1 hour but I would be okay with processing them all in 2 hours for now.
This is the API I would need to call for each player in my database to get their information: http://hiscore.runescape.com/index_lite.ws?player=Zezima
My site and database are hosted on a VPS.
My thought on how to achieve this is to spin up a handful of Digital Ocean VPS instances when the time comes to make the API calls and have my main VPS distribute the API calls across the DO instances which will make the API calls and insert the results back into my database.
Parallelization is your friend here. Pool your queue listeners and have them run on a machine with adequate CPU and memory.
How fast is your process? Completing 25,000 transactions in one hour means 7 per second. Do you have timing data to help guide the number of instances you'll need?
I'm assuming that your database will allow simultaneous INSERTs. You don't want those getting in each other's way.
The following code fragment
for($i=0;$i<60;$i++){
$u[$i]=$_REQUEST["u".$i];
$pic[$i] =imagecreatefromjpeg("http://graph.facebook.com/".$u[$i]."/picture");
}
is taking more than 90 seconds to execute on my new server. It was taking less than 15 seconds on my shared hosting server. However, on dedicated server it is taking more than 90 seconds.
The data center of my new server is Asia Pacific.
Please advice on how I can reduce this time of fetching images on the graph.
thanks and regards
Why not just request all the pictures' URLs in a single call?
https://graph.facebook.com/?fields=picture&ids=[CSV LIST OF IDS]&access_token=ACCESS_TOKEN
You'll then have a list of all the images and can fetch them all however you so wish
is taking more than 90 seconds to execute on my new server.
Well, for 60 HTTP requests that’s not too bad, I’d say.
It was taking less than 15 seconds on my shared hosting server. However, on dedicated server it is taking more than 90 seconds.
Maybe the connection of your old server was just faster …?
The data center of my new server is Asia Pacific.
Do you know by any chance, which one it was before?
Please advice on how I can reduce this time of fetching images on the graph.
Do you have to request all these images in one go?
Maybe your app’s workflow (which we don’t know anything about yet) would allow for other approaches, like getting user images at a previous time (f.e. when a user starts using your app) and cache them locally, so that you don’t have to do 60+ HTTP requests in one go.
Suppose I have 2 servers.
The first is a service that provides some computations, which can last long time (minutes to hours).
The second server will use this service to have some data computed.
I'm trying to design a REST API for the first server and so far so good. But I'd like to hear some opinion on how to model notifications when the long lasting task is finished.
I considered 2 approaches so far:
Polling - the second server will ask every now and then about the result.
Callback - Second server will setup an uri for the first one to call after it is done. But this smells a bit in REST API.
What do you think?
For your situation I would choose polling. When the second server makes the initial request to create the job on the first server, it should get a response that has the url of the eventual status page. The second server then polls that url every 5-15 minutes to check the status of the job. If the first server makes that url an RSS or Atom feed, then users could also point their RSS readers at the same url and find out themselves if the job is done. It's a real win when both people and machines can get information out of a single source.
In addition to what I've already answered in this similar question, I'd suggest using the Atom Publishing Protocol for the notification (you could publish to your second server).
If you use Python, you can take advantage of RabbitMQ and Celery to do the job. Celery lets you create an item in a queue and then pause execution of whatever you're running it through (i.e.: Django) such that you can consume the output of the queue processor as it becomes available. No need for polling OR callbacks.
I'd like to have something in my app that is just like Twitter but not Twitter. Basically it will be a place people can submit messages and do not need an account. They only way they can submit is through the app. I want other app users to see the submitted messages nearly immediate. I believe push notification can do that sort of work but do I need push notification for this? How does Twitter do it?
-- EDIT --
After reading some of the responses, push might be what I need. People will be submitting messages to my server often. If someone is watching the feed, they might see one new message per minute depending on the query they are using. I'm thinking to go with a MySQL database, (which allows switching to cheaper non Windows servers w/o much hassle) and push notification. Are there any reasons those won't work for my scenario?
You only need push notification if you want the app to be able to receive new messages while closed.
Here's a rough description of one way to do this:
Your app sends a message via HTTP Post to your server.
Your server stores the message in a database, using the iPhones unique ID as an identifier.
Your app connects to the server frequently, asking for new messages.
If there are any new ones, the server hands the message to the app, which displays it.
This is approximately what twitter/iphone twitter apps do.
Your choices are fairly binary:
Use push notification
Use Polling
With Push Notification:
You control when you contact your users... Heavy Load means you can slow updates down to avoid taxing your infrastructure
Contrariwise, you have to push to clients that may not even be there anymore (And thus may need some sort of register model), high load may mean that clients don't get immediate update
You can leverage things like Amazon's EC2 to give you more processing power
Unless you're out of capacity, users are almost certain to be receiving updates as they happen
To pick up messages missed while offline, the SERVER needs to know what message was last successfully received, store older messages and forward many all at once
If you choose to use polling:
You must have a stable address to be polled
You need the ability to have lots of quick query connections checking for new data, then returning that data if required.
If your application becomes popular enough you may find you don't have enough resources
If your resources are taxed your application will go down, rather then just slow down
You don't need to register clients and keep track of their on/offline state
Parallelizing on the fly is a bit trickier
To pick up older messages, the CLIENT needs to know when they last received a message and then request the server send any message since that time
Both can be fast, but they come with different bandwidth and processing profiles. I prefer push for everything that's real-time.
Might want to take a look at XMPP.
Twitter doesn't really push events out to the iPhone in realtime. It's more like polling by the various clients.
If you really want instantaneous for the last mile you'll want to use push.
Twitter uses lots of servers and raid arrays to handle the load of millions of people posting 140 character messages. Twitter clients log in and request a list of updates for all of the people the user is following within a certain time frame.
Push wouldn't be a good candidate for this because it does not persist the "tweets". It is simply a notification mechanism. There is a text messaging app on the App Store (called Ping!) that relies completely on push notification for sending text messages. This seems to work fine, but if the developers are keeping track of the messages, it is all done on their servers. In their case push makes sense as you want to alert the user of a new message. In the case of a twitter clone, however, it would probably just annoy users if they got a new notification every time someone tweeted.
In the end you're better off just implementing it server side and then developing an iPhone client that logs in and retrieves the latest tweets for the people the user is following.