I am trying to reverse engineer the API of an IoT device using mitmproxy. My setup is an iphone, computer running on MacOS 10.14, and an IoT device (watering pump) that can only access the wireless network after being plugged into the computer via USB connection, i.e. a not genuinely WiFi device. My phone is configured to point to the computer, which is running mitmproxy on a standard configuration.
When I send instructions from the app controlling the device on my phone to the device itself, presumably these instructions are sent to the computer, to the device cloud server, and then to the device. With these assumptions, one would think that they would see the flow of POST requests in mitmproxy before observing the results of those instructions. That is, if you send an instruction to turn on the pump, you'd think you'd see POST request containing that instruction show up in the mitmproxy flows before you see the pump turn on.
However that is not the case here. What happens is that, when I send instructions from the app, I observe the expected behavior from the IoT device, and then the flow of requests appear on the mitmproxy console seemingly at random. There seems to be no determinate relationship between the instructions I send and the requests that appear; they show up 5 seconds later, 5 minutes later, or 30 minutes later. Is this an intentional security feature? To somehow jam MITMproxies so that hackers cannot easily isolate the knowledge of which packet is performing which instruction? Or is it just something that I am doing wrong? Does anyone have any ideas as to what could be happening hear and potential solutions for making the flow of requests appear in real time? Ty
Related
I am looking for a solution to remotely power cycle embedded devices that are connected via a USB hub to a server. I have a software solution (usbreset.c) that was posted in several responses in stack overflow - rebooting the usb port. This works if the device is still alive, can be detected by the server, but, it just fails to communicate with the server. I have had to use it a few times. however, sometimes the device could hang or freeze on a gui page and there is no serial communication. This requires physically power cycling the device using it's on-off switch. Someone suggested raspberry-pi solution. But as per that solution, I would need one raspberry-pi per device. It's as if using a raspberry-pi to serve as a relay. This is expensive and cannot scale. Could relays work in this situation i.e. the power pins of each device connect to a relay. So, one relay per device. There are remote IP power cycle solutions. so via a web interface it is possible to check which device is offline and power cycle just that one. Such solutions are common in IT. However, is such a solution viable to remote power cycle embedded devices connected at the other end?
I'm researching and trying to building a RC car that can be controlled by the internet. I've started looking into how communication over the web works, but I seem to be going nowhere. My goal for the project is straight forward:
The RC car has an on-board camera and 4g wifi router that enables communication (driving commands, video streaming) over the internet. A Raspberry Pi will serve as the on-board computer.
I will be able to control the car with my PC even across the globe, as long as I'm connected.
I want to preferably do as much by myself as possible without relying too much on other people's code.
So here are my questions:
How does an application communicate over the internet? What is the interface between the application's logic (e.g pressing "w" to go forward), and transmitting/receiving that command over the internet?
How is video data stream handled?
I've looked into WebRTC and WebSockets for communication, but they are aimed at providing real time communication to web browsers and mobile, not something like a raspberry pi, and I'm still in the blind as for exactly what technology should I use, and in general the overview and architecture of real time communication.
All I've achieved so far was an app that sends text messages between devices through a server on my network, with very primitive reading/writing using java Socket.
In short, what does messenger/skype/zoom do in the background when you send a message or video call?
Any guidance would be greatly appreciated.
First things first. You cannot do real-time control over Internet, period. There is absolutely no way to guarantee the delivery latency. Your control commands can arrive with a delay from milliseconds to seconds, or never. No way around it.
Now, you can still do a number of reasonable steps to absorb that unpredictable latency as much as possible and safe-guard your remote robot from the consequences of the unreliable communication.
For example, instead of sending the drive commands directly - as in, acceleration, deceleration, turn angle, etc., you can send a projected trajectory that is calculated from your drive commands locally on a model. Your RC car must be sufficiently smart to do some form of localisation - at the very least, wheel odometry, and with a good enough time sync between the sender and the RC car you'll be able to control the behaviour remotely without nasty consequences of drive commands executed at an unpredictable delay.
You can add a heart-beat to your protocol, to monitor the quality of the communication line, and if hear-beat is delayed or missing, initiate emergency stop.
Also, don't bother with TCP, use UDP only and maintain your own sequence counter to monitor missing packets. Same applies to the telemetry stream, not just command channel.
I have an Arduino based device interfaced to a 3G modem which I use to record data from several sensors in a remote environment. I would like to be able to send commands and stream some data from the device every now and then back to my standard network connected PC. If the remote device was connected to a WIFI or other local area network this would be relatively straightforward, but as the device connects over 3G this means that it is behind the 3G carriers NAT and so establishing a connection to the device becomes difficult.
The device can, of course, open a TCP connection to my host PC any time it wishes, the problem is telling the device when i want it to do so. I need some way of getting some kind of message to the device to notify it that I would like it to initiate a connection to my PC.
I've been reading up on NAT traversal techniques that app developers use to initiate P2P comms between 2 devices both behind NATs such as UDP and TCP 'hole punching' but this method seems rather too complex for my arduino system. Another general idea is to have the device polling a web server periodically looking for a signal to initiate a connection, but I'm not sure how much traffic (and data usage costs) this would generate as the device would have to poll every 10 seconds or so in order to make sure it initiates it's connection within a reasonable time frame of the request being set on the web server that it polls.
Is there any commonly used method of achieving something like this? Any general ideas or insight would be much appreciated
Thanks,
James
I think the solution will depend largely on your particular applications and requirements.
There are several ways to achieve this type of functionality and it looks like you have covered some of them already. The most common are:
have the device poll the server. This may be ok depending on the response times you need. If you need to poll as regularly as you suggest above then I imagine power may be more important to you than data rates, especially if you are battery connected. With a typical 3G data plan the polling itself will have a negligible data overhead, I would think.
have the server send a SMS which then triggers the device to contact the server. You need to make sure the SMS variable delivery time is ok for you and you also have to be aware that SMS delivery is not guaranteed so you would have to build in some sort of check for delivery at a higher layer (or into your application).
use some low cost Android based device for your 3G connectivity and leverage the Google push notifications mechanism
It is worth noting that server polling typically gets very bad press as it is seems intuitively wasteful to have the client and the server constantly checking for messages, especially when the actual messages are fairly infrequent. However, underneath most push solutions there is still a pull mechanism in the background, albeit generally a very efficient one that may, for example, piggy back on other messages between the network and the mobile device and hence have minimal power and data overhead. Personally, I would say that if you do not have major concerns with battery/power or with the load polling might generate for your servers, then it is worth exploring if the simplicity benefits of a polling solution outweigh its other disadvantages.
I am trying to detect if my iPhone is in the same network as my Raspberry Pi. I would like to execute a script when I am at home and my iPhone's presence is registered in my LAN.
It seems that when the phone is in standby not even the iphone-sync port (6207/tcp) is found. "/usr/bin/nmap -n -sT -p62078 [my phone's local IP]" shows no host. I wonder what else I could scan for. Obviously the phone is online and ready to accept facetime calls (data via 3G is deactivated). Could I accomplish something with avahi which I am using on my Raspberry Pi, or are there other ways.
I've just spent a week beating on this problem so I can refrain from sending SMS home alarms to my wife when she's at work.
Pinging won't work because the iPhone won't respond to ICMP when asleep. Reading the ARP cache won't work because a sleeping iPhone will come and go (check it every 30 seconds for a few minutes).
The only way I have found to 'reliably' determine when my two iPhones are on my local (home) network is to use the PCAP dotnet library to look for any packets originating from either of the phones' MAC addresses. For example, if you run Wireshark with the capture filter
ether src <iphone-mac-address>
you will see a surprising amount of network discovery/announcement traffic from the phone. It still has quiescent states, but so far the longest interval I have seen between captured packets is around 10 minutes. You would have to wait until you have not heard from the phone for some interval (I use 15 minutes) before declaring it not-home.
With this technique you will find a phone quickly when it rejoins the home network, assuming your phone is configured for DHCP. I also use port mirroring on my main Ethernet switch to include traffic from my wireless access points.
I don't have a Raspberry Pi solution for this, because my linux expertise is very limited, but someone else may be able to help you along those lines. I have a Windows Service using the PCAP library and so far it works reliably, with the limitation of waiting 15 minutes before deciding an iPhone has left the network.
* update 2-3-2018 *
I have this detection algorithm down to about 5 minutes, using a combination of ping/arp messages directed to each phone, about once per minute. Seems to work great.
You can find a list of devices on your network by investigating your arp cache.
arp -a
Simply write a bash script to run arp -a at a regular interval, and search for the mac address of your phone.
You could go even further with this and perform different actions depending on what brand of device is connected.
The first 3 hexadecimal digits of a mac address are the vendor id.
Take the following mac address:
00:19:E3:AB:CD:EF
00:19:E3: is one of the registered mac address for apple devices.
By comparing the devices on your network with this list, you could detect when for example a '3com' device, or a 'dell' device attaches to your network.
http://www.coffer.com/mac_find/?string=apple
You can do "arp-scan -l -r10" for that (tested this myself), but the problem is if mobile data enabled the iphone will go and suspend wifi if screen is locked to safe battery. so you need to disable mobile data .. then arp-scan will work.
We are in the process of developing an method of caching so that our app can continue to operate in an area with very little/no signal.
Obviously users will try to continue to use functions that require data and we need to handle the inevitable failure of these requests appropriately.
Essentially we are sat in the office, switching airplane mode on and off to simulate entering/exiting signal then adjusting our app to fix any issues this may arise.
What I'd like to know is, is using airplane mode going to give us a reasonable simulation of entering/exiting an area with no data or are there other implications?
I've seen questions raising the issue that the 3G/EDGE connection may not always wake up after airplane mode is switched on - while I appreciate this method is no way as good as actually being out in the field testing, if we can get a reasonable simulation and account for the majority of the problems that arise then I think this is an acceptable tradeoff.
I apologise if this has been asked before, I did do a search on here & on google but couldn't find any appropriate results.
You should try the Network Link Conditioner
There is a WWDC 2012 session called Networking Best Practices that mentions it (but he does not explain how to use it there).
To get it, you have to go to XCode/Open Developer Tool/More Developer Tools.. and download the latest Hardware IO Tools for XCode.
Once you install it from the IO Tools pkg, "Network Link Conditioner" will appear in System Preferences
You can then do something like 100% packet loss to simulate one of those routers that pretends you are connected but actually doesn't work.
On iOS, the network link conditioner is under Settings / Developer (you must have enabled Developer mode in XCode first to see it)
The main problem is that in the Airplane Mode the networking operations fail fast, while spotty mobile signal will lead to timeouts and a-few-bytes-an-hour speeds. This is usually a significant difference from the UI viewpoint. (It might be worth a try to use some bandwidth throttle to starve the testing machine and see how it behaves when the network starts to break?)
A few years back, when testing remote devices which used the cell network to communicate with the 'home base', we did things like move them into a shielded room (make shift), place large shields on three of four sides to force them to connect to a certain tower (and therefore, network), etc. Brute force physical methods. Since this actually cuts off the signal, it may be a more realistic approach.
You may also want to try this through your wlan-router. First, disable data roaming on your iPhone. Then, let the iPhone be connected to the internet through your wlan network. Then, disconnect the gateway on your wlan router while your iPhone is still connected to the wlan network.
This depends on what failure modes you are trying to test.
I use Airplane mode as a first pass check to make sure an app submission isn't quickly rejected.
Other network failure handling checks might include:
3G only (no wifi).
WIFI only (in Airplane mode).
Pulling the power cord on the WIFI access point.
Pulling the network cable from the back of the WIFI access point after connecting to it (Reachability may falsely say yes).
Walking in and out of a basement
elevator (or other Faraday cage) in the middle of a transfer.
Driving between 2 cell towers during a data transfer.
Walking between 2 enabled WIFI access points between connection and data transfer.
Starting the app after more than 30 minutes of device inactivity (radios may be idle).
Running the app while another app (Safari, Mail) is downloading in the background.
etc.