Can someone give more details on the "100 concurrent API calls limitation" for twilio?
We need to send 1000 clients from 15 groups an SMS within one minute. To do that we bought 40 twilio numbers and created our own "sticky sender".
There is a 3 second delay between each SMS sending instead of the expected 1 per second.
How fast can I get these messages sent without getting any delayed or lost messages.
buying more numbers will not help because of the "100 concurrent api calls limitation" but i dont see anywhere in the twilio docs more info on exactly how that works.
It's been a few years since you asked this question, but in case somebody else comes across this post, I wanted to share: Twilio now has documentation for understanding Message Per Second (MPS) throughput (relevant for your use case), as well as Messaging Services with features like Sticky Sender. Finally, you may want to consider your Phone Number strategy - if you need higher throughput, Application-to-person 10-digit Long Codes (A2P 10DLCs), High Throughput Toll Free numbers, or Short Codes may all be options.
Related
I'm writing an application in which users will be able to send money to each other. I've built out most of it, but now comes the most important part, managing transactions as they occur.
What I'd like to do upon a successful transaction is to send an update to the recipient. Right now, my thinking is to do this via SSE or WebSockets. For this particular app, it doesn't appear that I need bi-directional communication, since the response would only be sent to the recipient's instance which should be listening for a response from the server.
I might be answering my own question here, but I also wanted to factor in scale. If my app grows to a million users, for instance, which technology would best be able to handle the number of transactions being processed without failure?
I'm also a little unsure as to how to implement this for the case where there is a multiplicity of users, but I only want a particular user to receive the update.
Any advise would be greatly appreciated.
Thanks!
I am going to implement some sort of home automation system (as my bachelor thesis). I have looked on MQTT protocol and I have two question about it.
I have saw this tutorial:
https://www.youtube.com/watch?v=X04yaaydjFo&list=PLeJ_Vi9u6KisKTSNeRRfqsASNZdHSbo8E&index=13
Which has materials (codes etc) here:
https://randomnerdtutorials.com/raspberry-pi-publishing-mqtt-messages-to-esp8266/
My first question is:
Is the logic how to manage acquired data/message (due to some topic subscription) in broker or clients? From tutorial above it seems, that that logic how to deal with message is in clients. Should it always be there? Or is it possible to has it in broker? Sorry if this question is too "abstract", i am at beginning of programming, so I have no concrete example. Basically what I want is to have as "light" program in client, as is possible (because broker will have lot of memory and computing capacity, while clients will be very limited in both).
My second question is:
Is it posible to put ESP8266 (or just any client) in sleep and awake it, let's say every 5 minutes? Of course it should not be problem if that client only publish (and never subscribe) to topics. But what when I have client, which can read some sensor, which will send in that 5 minutes cycles to broker, and also can control some of its output? Is there way to do that? Or if client is not available (if there is some data to publish to it and it is currently sleeping), the message is just thrown out? My thought was if there is way to ask broker after client awake if there was any published message to them during client sleep?
Thanks for every info! :)
1. Answer
The message processing logic is to be fully implemented by the client. With MQTT there is no "stream" processing as is possible with e.g. Apache Kafka. However, you can of course have intermediary (non-IoT) clients that subscribe to the original topic, prepare a modified message and publish it to a new topic - the one which the IoT device would then subscribe to.
2. Answer
You can tell the broker to retain a message. However, it will at most retain 1 message per topic.
P.S. for the future please stick to 1-question-1-post here on SO.
I have 15 worker clients and one master connected through internet. Job & data are been passed through REST api in json format.
Jobs are not restricted to any particular client. Any worker can query for the available job in regular interval(say 30 seconds), process it and will update the status.
In this scenario, how can I prevent same records been sent to different clients while GET request.
Followings are my solution approach to overcome this issue:
Take top 5 unprocessed records from the database and make it as SENT and expose via REST GET.
But the problem is, it creates inconsistency. Some times, the client doesn't got data due to network connectivity issue. But in server, it will be marked as SENT. So, no other clients can get that data. It will remain as SENT forever.
Get the list from server, and reply back the list of job IDs to Server as received. But in-between this time gap, some other clients also getting same set of Jobs.
You've stumbled upon a fundamental problem in distributed systems: there is no way to know if the other side received your message. You can certainly improve the situation with TCP and ack messages. But if you never get the ACK did the message never arrive, did it arrive but the recipient die before processesing, or did the recipient send he ACK and the ACK get dropped?
That means you need to design your system to handle receiving data more than once.
You offer two partial solutions; if you combine them, your solution starts to look like how SQS works. Mark the item as pending_ack with a timestamp. After client replies, it is marked sent. Any pending_ackss past a certain time period are eligible to be resent.
Pick your time period to allow for slow network and slow clients and it boils down to only sending duplicates when you really don't know if the client died or not.
Maybe you should reconsider the approach to blocking resources. REST architecture - by definition is not obliged to save information about client. Instead, you may want to consider optimistic concurrency control (http://en.wikipedia.org/wiki/Optimistic_concurrency_control).
i would like to take messages from amazon sqs in the same order in which it is inserted into sqs ( first in first out model).
Is their any way to implement it??
I am using zend php for programing.
Unordered message delivery is inherent in the design of SQS. You could try to work around it by numbering the messages and storing the out-of-order messages locally until the missing messages arrive, but its probably not worth the hassle.
SQS is really a bit of an odd duck, it does what it says, but what it does isn't what most people are looking for in a message bus. I really wish Amazon would offer and additional queuing solution more like RabbitMQ. SQS is really only suited for distributing tasks that aren't even remotely coupled, and where things like order and latency aren't important. For instance it would be great for sending completed orders to a shipping center, or perhaps scheduling print jobs.
Their own documentation shows it being used to schedule thumbnail creation, but when I recently used it for this exact purpose I quickly discovered that my users weren't going to be impressed with the latency: which at times is 30-50 seconds.
You can still run RabbitMQ on EC2 nodes, and while not as scalable as SQS it does cluster and should take you pretty far.
You could try IronMQ. It is hosted like SQS, has guaranteed first in first out ordering, no eventual consistency delays, is uber scalable and you can be up and running in minutes.
Here's a PHP library for it: https://github.com/iron-io/iron_mq_php
Disclaimer: I work for Iron.io
The SQS documentation answers this for you (bold is my emphasis to directly answer your question):
Amazon SQS makes a best effort to preserve order in messages, but due
to the distributed nature of the queue, we cannot guarantee you will
receive messages in the exact order you sent them. If your system
requires that order be preserved, we recommend you place sequencing
information in each message so you can reorder the messages upon
receipt.
I have tried to implement the FIFO fashion for receiving the messages in the same order they were sent
For this you can use message sequence no which it sent every time with message and validate at the receiver end
By Using this way you can get desired output in FIFO order
I'm preparing to set up a APNS message server, and I was wondering if anybody has done any analysis on APNS server load that they would be able to share. Minimum server specs, maximum messages per second, anything like that.
Thanks!
edit: I'm planning to implement this with .NET, but info about any platform would be incredibly useful.
For my application (which has about 24,000 downloads) I am seeing an average of of about 1300 messages sent a day.
Those are low numbers, but then my client base isn't that large either. But I figure I might as well contribute some info. :-)
My notification provider is idle most of the time so there is MUCH more capacity available if I need it.
Its also using very little ram at this point (somewhere around 13 mb - I implemented my provider in Python and suspect most of that is taken up by the run time).
I am running on a Media Temple dv (specifically the Base configuration).
I haven't extrapolated out the numbers to find what my theoretical maximum would be, but because of the niche market of my application its not something that worries me at this point. I have lots of capacity to scale with.
Hope that helps a bit.
chris.
One of the Apple devs mentioned that 100,000 messages is not considered a large amount, that doesn't really answer your question, but I wouldn't expect that sending the actual messages would be the bottleneck.
Any server that can handle your database work should be fine for sending the messages out. The protocol is intentionally light-weight.
There are no maximum messages per second.
You should consider that every message must be smaller than 256 Byte. Otherwise Apple will be reject your messages. And you can also check MonoPush. AFAIK they are building their products top of the .Net Framework.