I'm planning to setup some sensors for a project I am working on. I plan to use a simple xmpp client on each device hosting sensors and posting the readings via PEP/PubSub.
I would like to run a program which dynamically displays this sensor data upon receiving the push notifications from the sensor clients.
I intend to communicate on a LAN with 100Mbs connections and several WIFI APs as needed for the sensor locations.
Is XMPP a good choice for a backend?
Would this scale well at say 100 devices running 6-24 sensors each updating every 15-30 seconds?
XMPP would certainly be able to handle this use case, and seems a good fit based on the small set of requirements you have shown. I am assuming that the sensor data is small amounts of textual information.
From what I can see, I would not use PEP though, just straight up pubsub with a single node for all the sensors to publish to. Pretty much as simple as it gets.
I would think that any XMPP server should be able to easily handle that load. I know I was able to publish 800,000-1,000,000 items to a persistent node in Openfire in about 5 minutes (3.7.2 alpha version). A non persistent node would probably do a little better.
Related
I'm researching and trying to building a RC car that can be controlled by the internet. I've started looking into how communication over the web works, but I seem to be going nowhere. My goal for the project is straight forward:
The RC car has an on-board camera and 4g wifi router that enables communication (driving commands, video streaming) over the internet. A Raspberry Pi will serve as the on-board computer.
I will be able to control the car with my PC even across the globe, as long as I'm connected.
I want to preferably do as much by myself as possible without relying too much on other people's code.
So here are my questions:
How does an application communicate over the internet? What is the interface between the application's logic (e.g pressing "w" to go forward), and transmitting/receiving that command over the internet?
How is video data stream handled?
I've looked into WebRTC and WebSockets for communication, but they are aimed at providing real time communication to web browsers and mobile, not something like a raspberry pi, and I'm still in the blind as for exactly what technology should I use, and in general the overview and architecture of real time communication.
All I've achieved so far was an app that sends text messages between devices through a server on my network, with very primitive reading/writing using java Socket.
In short, what does messenger/skype/zoom do in the background when you send a message or video call?
Any guidance would be greatly appreciated.
First things first. You cannot do real-time control over Internet, period. There is absolutely no way to guarantee the delivery latency. Your control commands can arrive with a delay from milliseconds to seconds, or never. No way around it.
Now, you can still do a number of reasonable steps to absorb that unpredictable latency as much as possible and safe-guard your remote robot from the consequences of the unreliable communication.
For example, instead of sending the drive commands directly - as in, acceleration, deceleration, turn angle, etc., you can send a projected trajectory that is calculated from your drive commands locally on a model. Your RC car must be sufficiently smart to do some form of localisation - at the very least, wheel odometry, and with a good enough time sync between the sender and the RC car you'll be able to control the behaviour remotely without nasty consequences of drive commands executed at an unpredictable delay.
You can add a heart-beat to your protocol, to monitor the quality of the communication line, and if hear-beat is delayed or missing, initiate emergency stop.
Also, don't bother with TCP, use UDP only and maintain your own sequence counter to monitor missing packets. Same applies to the telemetry stream, not just command channel.
I am a web developer and I am starting to learn about the world of IoT.
Because of the vaccines arrival to my country (Argentina) I got asked to build 80 temperature sensors to monitor them and I have some questions about it.
What would be the best way to connect all of them to the cloud?
If I use for example aws iot platform, do you know how much it would cost monthly for just sending
and storing temperature logs for each sensor (remember, there are 80 of them)?
Is there any language/environment/protocol that works better for IoT? Because it's a constant flow
of lightweight data...
Is there a better way to connect them to the internet besides using esp32 modules for
each?(I saw a tutorial that said it's possible to connect some more to a single esp32 module)
If you have any advice I'd love to hear it. I know how to code but when it comes to backend and specially server stuff I have a lot to learn.
Costs are directly related to the amount of data you send, process and store. You'd have to check the price lists for each cloud service you plan to use. If we assume that you'll be sending 1 temperature reading (with associated data such as timestamp, device id, ...) every 10 minutes using reasonable protocols (MQTT, JSON) then the total costs for all 80 devices would be perhaps a few dollars per month. The total database storage will accumulate over time and you'll be charged more, but honestly the amount of data under these conditions is ridiculously low.
An ESP32 is cheap, has WiFi and enough performance to send data to cloud. You can connect this micro to AWS IoT or Google Cloud IoT using the relevant libraries from either: AWS library or Google IoT library. These libraries decide the questions of language and protocol on the microcontroller side - it's C and MQTT/HTTPS (but avoid the HTTPS, MQTT is much more practial). You can use JSON for the actual temperature data message. The microcontroller development takes place with either ESP IDF (a bit lower-level C environment) or Arduino (a bit higher-level C/C++ environment). Those use FreeRTOS as the OS on micro (note that the IoT libraries work on almost anything).
A practical alternative to ESP IDF and Arduino (especially for a web dev) is Mongoose OS where you can do much of the development work in JavaScript (not all, though). It has high-level libraries for both AWS and Google IoT (which still use the same underlying MQTT/HTTPS client, I assume).
By far the easiest way to connect the ESP32 modules to Internet is to have each connect to a WiFi AP. If the single WiFi AP doesn't cover all devices, add more until they do. ESP32 does have a mesh networking library, but I would hesitate to recommend it to newbies.
I have an Arduino based device interfaced to a 3G modem which I use to record data from several sensors in a remote environment. I would like to be able to send commands and stream some data from the device every now and then back to my standard network connected PC. If the remote device was connected to a WIFI or other local area network this would be relatively straightforward, but as the device connects over 3G this means that it is behind the 3G carriers NAT and so establishing a connection to the device becomes difficult.
The device can, of course, open a TCP connection to my host PC any time it wishes, the problem is telling the device when i want it to do so. I need some way of getting some kind of message to the device to notify it that I would like it to initiate a connection to my PC.
I've been reading up on NAT traversal techniques that app developers use to initiate P2P comms between 2 devices both behind NATs such as UDP and TCP 'hole punching' but this method seems rather too complex for my arduino system. Another general idea is to have the device polling a web server periodically looking for a signal to initiate a connection, but I'm not sure how much traffic (and data usage costs) this would generate as the device would have to poll every 10 seconds or so in order to make sure it initiates it's connection within a reasonable time frame of the request being set on the web server that it polls.
Is there any commonly used method of achieving something like this? Any general ideas or insight would be much appreciated
Thanks,
James
I think the solution will depend largely on your particular applications and requirements.
There are several ways to achieve this type of functionality and it looks like you have covered some of them already. The most common are:
have the device poll the server. This may be ok depending on the response times you need. If you need to poll as regularly as you suggest above then I imagine power may be more important to you than data rates, especially if you are battery connected. With a typical 3G data plan the polling itself will have a negligible data overhead, I would think.
have the server send a SMS which then triggers the device to contact the server. You need to make sure the SMS variable delivery time is ok for you and you also have to be aware that SMS delivery is not guaranteed so you would have to build in some sort of check for delivery at a higher layer (or into your application).
use some low cost Android based device for your 3G connectivity and leverage the Google push notifications mechanism
It is worth noting that server polling typically gets very bad press as it is seems intuitively wasteful to have the client and the server constantly checking for messages, especially when the actual messages are fairly infrequent. However, underneath most push solutions there is still a pull mechanism in the background, albeit generally a very efficient one that may, for example, piggy back on other messages between the network and the mobile device and hence have minimal power and data overhead. Personally, I would say that if you do not have major concerns with battery/power or with the load polling might generate for your servers, then it is worth exploring if the simplicity benefits of a polling solution outweigh its other disadvantages.
I have an idea for an iPhone app that will make billions of dollars for me, and help me take over the world. I am currently designing the architecture of the application. I had originally assumed that I would need to create and host a database that would allow users to upload their data and share it with other, targeted, users. But I thought it would be better if the users could send their data directly to each other without having it stored in a database. However, I haven't been able to find anything that would suggest this is possible.
Is it possible to send data between iPhones that are not in close proximity (bluetooth) and not on a LAN? For instance...User #1, who is in Tennessee, creates some information and hits "Send to others" and it sends that information directly to User #2, who is in Arizona, and User #3, who is in Maine. Could the text messaging functionality be hacked to do that?
(I realize that the phones belonging to Users #2 and #3 would have to be on, and running the app that would receive the data)
I am new to mobile development, and I am still getting used to the functionality of mobile devices so, I am certain this is a dumb question. But, it is worth asking because, if it can be done, this would dramatically change the architecture (and maintenance costs) of this application.
Thanks, in advance, for whatever advice/pointers you can give.
This is something handled by using the publish - subscribe pattern. You may want to look into a plug 'n play service such as pubnub. That will let you do just that. Phones that should be receiving each other's messages will have listeners set up on the same pubnub channel, and will receive a notification when any of the other phones publish to that channel. Something like pubnub has a pretty simple API/SDK that you can use to get a prototype up and running pretty quickly (and for free, at least at first).
User #1's app sends the message to your server. You server does an APNS push to the phones of users #2, and #3. They don't even need to be running the app.
No need to mark this correct or up-vote it, but I will take a high-paying job in the new world order.
Amazon has an application service called Simple Queue Service (Amazon SQS) which Allows you to create queue's with messages in them tat app's can subscribe to.
To quote their page:
Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly
scalable, hosted queue for storing messages as they travel between
computers. By using Amazon SQS, developers can simply move data
between distributed components of their applications that perform
different tasks, without losing messages or requiring each component
to be always available. Amazon SQS makes it easy to build an automated
workflow, working in close conjunction with the Amazon Elastic Compute
Cloud (Amazon EC2) and the other AWS infrastructure web services.
Amazon SQS works by exposing Amazon’s web-scale messaging
infrastructure as a web service. Any computer on the Internet can add
or read messages without any installed software or special firewall
configurations. Components of applications using Amazon SQS can run
independently, and do not need to be on the same network, developed
with the same technologies, or running at the same time.
They have an iOS API to integrate it into your app, and it is free for up to 100,000 messages per month and then $0.01 per 10,000 messages after that.
More info here:
http://aws.amazon.com/sqs/
Is it possible to send data between iPhones that are not in close
proximity (bluetooth) and not on a LAN?
The two devices obviously need some kind of connectivity.
For instance...User #1, who is in Tennessee, creates some information
and hits "Send to others" and it sends that information directly to
User #2, who is in Arizona, and User #3, who is in Maine. Could the
text messaging functionality be hacked to do that?
The problems here are:
helping each device discover the correct address for the others, and...
ensuring that the devices can reach each other.
Mobile devices are constantly moving around on the network and changing their IP addresses as they go. As you're driving to work, your device may have Internet access through it's 3G connection, except for those times when you're driving through a tunnel and don't have any connectivity at all. Once you enter your building, a wifi connection becomes available, so the device switches to that. But your company's network has a firewall that blocks incoming connections, and it's impossible to know in advance which IP address you'll get from the DHCP server at any given time anyway. The same is true for the folks in Arizona and Maine and wherever else.
A server, on the other hand, is usually located at an address that's both easy to discover (thanks to the Domain Name System), easy to reach, and almost always available. For those reasons, it's common to have mobile devices communicate with each other by going through an intermediate server.
I was wondering what amount of time is required to convey information regarding the tilt and position (not gps) of one particular iphone to another. Could 2 iphones send and receive this information simultaneously? What about 3 iphones? I'm interested in an application that is able to simultaneously send and receive and make conditional decisions based on this information received all within a half a second-ish.
Any shot this is possible? If so, is bluetooth or wifi better?
Thanks a ton,
Jake
This is currently not possible without an intermediate server. (Without a jailbreak, which would make it possible, but extremely difficult)
I'm assuming your purpose is gaming, in which case, the latency associated with a trip to a server and back over a cellular data network, is likely to take too long for any satisfactory gaming experience. I don't believe it would be within half a second.
This will be possible via Bluetooth in the upcoming 3.0 iPhone software, but that is still under NDA, so you are not likely to be able to get any reliable performance numbers until it is released. If I were guessing, I would certainly guess that the latency associated with a direct Bluetooth connection would be FAR under half a second.
All you've got as an option right now is Wi-Fi or the Cell Network. If you use Bonjour over Wi-Fi, you'd have latencies in the milliseconds, but all the phones would have to be connected to the same access point. Take a look at the WiTap example.
It is definitely possible, you'd want to connect your peers over WiFi for best performance and reliability, but Bluetooth would be ok as long as your data packets were constrained to small sizes (< 1k). Check out this documentation and sample code to see how to access UIAccelerometer:
http://developer.apple.com/iphone/library/documentation/UIKit/Reference/UIAccelerometer_Class/Reference/UIAccelerometer.html
http://developer.apple.com/iphone/library/samplecode/AccelerometerGraph/index.html#//apple_ref/doc/uid/DTS40007410
The trick is that the update frequency is controlled in part by the systems needs, so there may be a window (while the system is attempting to update device orientation) wherein your application receives no updates.