I would like to trigger an event (e.g. play music) on multiple iOS devices at the exact same time (by means of milliseconds)
My approach is to keep a socket connection and send a timestamp to iOS devices (10 seconds later from current time) and trigger the event on iOS devices at that timestamp.
Problem is iOS devices might differ 1 or 2 seconds and that would cause a desynchronize. And even timestamps are pointing out the same time on each devices (AFAIK) they are not on millisecond sensivity.
Is there any way to trigger an event simultaneously on multiple devices, or an approach which should be followed?
Don't send the data over the Internet. You can't assume the connection latency will be low enough for your needs. Use Bluetooth instead. You can do with with GameKit, with dns-sd, or with a library like HHServices.
Pick a device that will act as the controller. Apple provide sample code to do so with GameKit, but it's not difficult to think up your own method. When you want to trigger the action, that controller will send a packet over Bluetooth to the other devices.
I doubt you need lower latency than that, but if you do, have the controller send packets to each connected device to ascertain the latency for each connection, have them send their timestamps to the controller, then the controller should be able to calculate a timestamp for each of them that will occur at the same time.
Related
Our application uses the Twilio voice SDKs for iOS, Android, and Web. Our use case relies on precise device synchronization and time stamping. We are playing an audio stream on multiple adjacent devices (in a Twilio conference call) and we need that audio playback to be in sync. Most of the time, it works great, but every now-and-then, one of the devices falls a little bit behind and throws off the whole experience. We want to detect when a device is falling behind (receiving packets late) so we can temporarily mute it so it does not throw off the user experience we are going for.
We believe that Twilio voice uses real time communication (web RTC) and real-time transport protocol (RTP) under the hood. We also believe RTP has time stamping information for when packets are sent out and when packets are received.
We are looking for any suggestions for how we might read this timestamp information (both sent & received) to detect device synchronization issues.
Our iOS and Android clients are built using Flutter & Dart, so any way to look at this packet information using Dart would be great. If not, we can use native channels through Swift and Kotlin. For the web, we would need a way to look at this timestamp data using javascript.
If possible, we'd like to access this information through the SDK. I don't see anything about timestamps in Twilio's voice documentation. So, if not possible, we might have to sniff for packets on the devices? This way, we could look at the RTP packets coming from Twilio to see what information is available. As long as this does not break Twilio terms of service, of course :)
Even if you could get this information I don't think it will be useful. The timestamp field in RTP has little to do with real time. In voice it's actually a sample offset into the audio stream. With a typical narrowband codec with a fixed bit rate and no silence suppression it's completely predictable from the RTP sequence number. For example, with 20ms packets of G.711 it will increment by exactly 160 each packet.
RTP receivers expect there to be random variation between the receipt time of a packet and its timestamp - known as jitter. This is introduced by delays at the sender, in the network and at the receiver. This is why receivers use jitter buffers to reduce the likelihood of buffer underrun on playing. The definition of jitter for RTCP - the interarrival jitter - is a calculation that measures this. That is - the variation between the (predictable) RTP timestamp and the measured wallclock time at the receiver.
Maybe you need something more like an NTP protocol between your client and your server.
What is the best way to COMPLETELY restart the iOS Bluetooth BTLE central and peripheral managers, after communication stops between two iOS devices (iPad-mini)?
Sometimes after a few minutes, my BTLE communication stops (central can not get response from peripheral, though debug output of each device shows app still running and central app still is trying to scan peripheral, and peripheral is still advertising), and will not start again:
after stopping scanning and stopping advertising, comm still doesn't work;
after re-opening the app, comm still doesn't work(!);
after POWER CYCLING the iPads comm works again, but then after a few minutes dies.
Therefore, something I'm doing is clobbering maybe the iOS core Bluetooth software.
Sometimes there are error messages from the underlying BTLE layer.
BACKGROUND:
Each iPad in this system alternately works as central, to read/write data to others, and then as peripheral, to be read by others. Never at the same time, and with a 1 second delay between transistions.
The bluetooth stack is not the most robust part of iOS. This may have improved in iOS7 but issues have always existed. You most probably are stressing the system so that this hang happens more often. You should create a bug report and send it to Apple. Or create a TSI, as a developer you have two of those every year. They are the only ones who can do something about it.
To solve the issue, restarting the app usually helps but sometimes bluetooth needs to be turned on/off or worst case the device needs to be power cycled. Unfortunately, there is not programmatic way to do any of these.
I need to receive data periodically through a BlueTooth External Accessory.
I implemented an event-driven model of EA's streams. However, the initial transmission from bluetooth is always delayed. For example, if each packet was 15 bytes long, the stream delegate would not fires until about 150 bytes.
Will polling help?
EDIT:
Also I found it hard to recover the session after the app switching back from background to foreground. Trying to open session again would fail. Any idea?
Read every bytes when NSStreamEventHasBytesAvailable arrives.
Did you develop your own Bluetooth accessory? May be the MCU only flushes after sending every 150 bytes.
Also you mentioned initial transmission. Do you know once the Bluetooth device is paired and connected to iPhone, it has to go through some identification process, handshaking some secret certificate. This may take few and even 10 seconds, depending on signal quality. This may be the cause of delay.
My iPhone app sends data persistently over the network, and can handle having the network completely unavailable for a time. However I do my dev on an ipod and have difficulty testing 3G connectivity issues; My question: is there period of non-connectivity, while an app is in the foreground, after which the iphone will stop checking for the network, such that there are no bars and reachability returns 0 until the user "wakes" the phone, e.g. by locking/unlocking? Or can e.g. 1 hr without data network pass and the iphone will recognize that the network is back and reachability will start returning true (I am polling every 1 minute while there is not connection to my server).
This is my experience with wifi: if, while running my app, the ipod hasn't connected to a wifi network for 1/2 hr, it stops looking for networks, and I need to lock and unlock it to stir the device into looking again.
One other thing: the iPhone has the screen dimmed by the proximity sensor while all this persistent network use is happening.
Wi-Fi going down after 30 min of inactivity is a documented behavior. Cellular network is always active, unless turned off by the user. So if it goes down for a while it should get back online automatically when possible even if there was no user activity.
I was wondering what amount of time is required to convey information regarding the tilt and position (not gps) of one particular iphone to another. Could 2 iphones send and receive this information simultaneously? What about 3 iphones? I'm interested in an application that is able to simultaneously send and receive and make conditional decisions based on this information received all within a half a second-ish.
Any shot this is possible? If so, is bluetooth or wifi better?
Thanks a ton,
Jake
This is currently not possible without an intermediate server. (Without a jailbreak, which would make it possible, but extremely difficult)
I'm assuming your purpose is gaming, in which case, the latency associated with a trip to a server and back over a cellular data network, is likely to take too long for any satisfactory gaming experience. I don't believe it would be within half a second.
This will be possible via Bluetooth in the upcoming 3.0 iPhone software, but that is still under NDA, so you are not likely to be able to get any reliable performance numbers until it is released. If I were guessing, I would certainly guess that the latency associated with a direct Bluetooth connection would be FAR under half a second.
All you've got as an option right now is Wi-Fi or the Cell Network. If you use Bonjour over Wi-Fi, you'd have latencies in the milliseconds, but all the phones would have to be connected to the same access point. Take a look at the WiTap example.
It is definitely possible, you'd want to connect your peers over WiFi for best performance and reliability, but Bluetooth would be ok as long as your data packets were constrained to small sizes (< 1k). Check out this documentation and sample code to see how to access UIAccelerometer:
http://developer.apple.com/iphone/library/documentation/UIKit/Reference/UIAccelerometer_Class/Reference/UIAccelerometer.html
http://developer.apple.com/iphone/library/samplecode/AccelerometerGraph/index.html#//apple_ref/doc/uid/DTS40007410
The trick is that the update frequency is controlled in part by the systems needs, so there may be a window (while the system is attempting to update device orientation) wherein your application receives no updates.