How to getStats() in WebRTC iOS SDK in Swift? - swift

I am trying to get ICE Candidate Pair Stats in iOS SDK in Swift. I see this specific interface: statsForTrack but I am unsure about how to use it. Has anybody done this before?

you can use stats function in PeerConnection like this:
peerConnection.stats(for: RTCMediaStreamTrack?, statsOutputLevel: .debug, completionHandler: { reports in
// reports here
}
you can set nil for parameter RTCMediaStreamTrack for get all reports or set special mediaStreamTack to get reports only this mediaStream.
peerConnection.stats(for: nil, statsOutputLevel: .debug, completionHandler: { reports in
// reports all mediaStreams
}

In your class where you creating peerConnection there you can start a timer like below:
statsTimer = [NSTimer scheduledTimerWithTimeInterval:2.0
target:self
selector:#selector(getStats)
userInfo:nil
repeats:YES];
And getStats be like this:
- (void)getStats {
NSArray<RTCMediaStreamTrack *> *tracks = mediaStream.videoTracks;
tracks = [tracks arrayByAddingObjectsFromArray:// Here mediaStream is instance of RTCMediaStream (NSArray<RTCMediaStreamTrack *> *)_publishStream.mediaStream.audioTracks];
for (RTCMediaStreamTrack *track in tracks) {
[peerConnection statsForTrack:track
statsOutputLevel:RTCStatsOutputLevelStandard
completionHandler:^(NSArray<RTCLegacyStatsReport *> * _Nonnull stats) {
for (RTCLegacyStatsReport *stat in stats) {
if ([stat.type isEqualToString:kRTCStatsTypeSSRC]) {
[self processRTCLegacyStatsReport:stat];
[statsBySSRC setObject:#{ kRTCStatsBytesSent: [stat.values objectForKey:kRTCStatsBytesSent],
kRTCStatsLastDate: [NSDate date]
} forKey:[stat.values objectForKey:kRTCStatsTypeSSRC]];
}
}
}];
}
}
- (void)processRTCLegacyStatsReport:(RTCLegacyStatsReport *)statsReport {
NSString *ssrc = [statsReport.values objectForKey:kRTCStatsTypeSSRC];
NSString *mediaType = [statsReport.values objectForKey:kRTCStatsMediaTypeKey];
unsigned long kbps = [self calculateBitrateForStatsReport:statsReport];
L_INFO(#"RTC Publishing %# Stats Type: %#, ID: %# Dict: %#",
mediaType, statsReport.type, statsReport.reportId, statsReport.values);
L_INFO(#"RTC Publishing %# kbps: %lld", mediaType, kbps)
}
- (unsigned long)calculateBitrateForStatsReport:(RTCLegacyStatsReport *)statsReport {
NSString *ssrc = [statsReport.values objectForKey:kRTCStatsTypeSSRC];
unsigned long bytesSent = [[statsReport.values objectForKey:kRTCStatsBytesSent] intValue];
unsigned long lastBytesSent = [[[statsBySSRC objectForKey:ssrc] objectForKey:kRTCStatsBytesSent] intValue];
NSDate *lastStatsDate = [[statsBySSRC objectForKey:ssrc] objectForKey:kRTCStatsLastDate];
NSTimeInterval seconds = [lastStatsDate timeIntervalSinceNow];
unsigned long kbps = (((bytesSent - lastBytesSent) * 8) / fabs(seconds)) / 1000.0;
return kbps;
}

Related

iOS App killed But AppleWatch Can Send Messag To it ? why?

AppleWatch Enterforground, Watch client send dictionary of "#{#"check":#"I'm Back"} "
iOS client receive "#{#"check":#"I'm Back"} " , replay dictionary with static num i as "#{#"check":#"Rcheck_i"}"
kill iOS Client
AppleWatch enterbackground, 5 second , enterforround , AppleWatch receive message from ios client as "#{#"check":#"Rcheck_i"}", what's wrong???
who can help me,the code is ...
//iOS Client
static int i = 0;
- (void) session:(nonnull WCSession *)session didReceiveMessage:(nonnull NSDictionary<NSString *,id> *)message replyHandler:(nonnull void (^)(NSDictionary<NSString *,id> * __nonnull))replyHandler{
dispatch_async(dispatch_get_main_queue(), ^{
i++;
NSString *temKey = [message.allKeys objectAtIndex:0];
NSString *temStr = [NSString stringWithFormat:#"%#_%d",temKey,i];
dispatch_async(dispatch_get_main_queue(), ^{
self.statusLabel.text = temStr;
});
if([message objectForKey:#"check"]){
NSString *temStr = [NSString stringWithFormat:#"Rcheck_%d",i];
replyHandler(#{#"check":temStr});
}
});
}
//AppleWatch Client
static int i = 0;
- (void)willActivate {
[super willActivate];
i++;
if ([WCSession isSupported] ) {
WCSession* session = [WCSession defaultSession];
session.delegate = self;
[session activateSession];
[self performSelector:#selector(checkAction) withObject:nil afterDelay:0.1];
}
else{
[self.lblLockState setText:#"WATCH NOT support"];
}
}
- (void) checkAction{
WCSession* session = [WCSession defaultSession];
[session sendMessage:#{#"check":#"I'm Back"} replyHandler:^(NSDictionary<NSString *,id> * _Nonnull replyHandler) {
dispatch_async(dispatch_get_main_queue(), ^{
NSString *temStr;
if ([replyHandler objectForKey:#"check"]) {
temStr = [replyHandler objectForKey:#"check"] ;//[NSString stringWithFormat:#"CheckSucc_%d",i];
}
else{
temStr = [NSString stringWithFormat:#"CheckSucc??_%d",i];
}
[self.lblLockState setText:temStr];
});
} errorHandler:^(NSError * _Nonnull error) {
dispatch_async(dispatch_get_main_queue(), ^{
NSString *temStr = [NSString stringWithFormat:#"CheckFail_%d",i];
[self.lblLockState setText:temStr];
});
}];
}

How to implement uislider so that it can be used as a scrubber in AVQueueplayer

I have implemented scrubbing using UISlider the problem is when the song is being played the slider doesnt initiate automatically ,it starts moving only when i tap on it and when the next song starts playing it moves automatically . i want the slider to move when the song starts playing ,this is the code which i am using and one more thing when i move the slider the song has to fast forward .I have added the code which i have written .can anyone help me out .Thanks
-(IBAction)slider:(id)sender
{
CMTime interval = CMTimeMake(33, 1000);
self.playbackObserver = [myPlayer addPeriodicTimeObserverForInterval:interval queue:dispatch_get_current_queue() usingBlock:^(CMTime time) {
CMTime endTime = CMTimeConvertScale (myPlayer.currentItem.asset.duration, myPlayer.currentTime.timescale, kCMTimeRoundingMethod_RoundHalfAwayFromZero);
if (CMTimeCompare(endTime, kCMTimeZero) != 0) {
double normalizedTime = (double) myPlayer.currentTime.value / (double) endTime.value;
self.timeSlider.value = normalizedTime;
}
Float64 currentSeconds = CMTimeGetSeconds(myPlayer.currentTime);
int mins = currentSeconds/60.0;
int secs = fmodf(currentSeconds, 60.0);
NSString *minsString = mins < 10 ? [NSString stringWithFormat:#"0%d", mins] : [NSString stringWithFormat:#"%d", mins];
NSString *secsString = secs < 10 ? [NSString stringWithFormat:#"0%d", secs] : [NSString stringWithFormat:#"%d", secs];
currentTimeLabel.text = [NSString stringWithFormat:#"%#:%#", minsString, secsString];
}];
}
EDIT
can u explain where am i going wrong
-(IBAction)playButtonPressed:(UIButton *)sender
{
CMTime interval = CMTimeMake(33, 1000);
self.playbackObserver = [myPlayer addPeriodicTimeObserverForInterval:interval queue:dispatch_get_current_queue() usingBlock:^(CMTime time)
{
CMTime endTime = CMTimeConvertScale (myPlayer.currentItem.asset.duration, myPlayer.currentTime.timescale, kCMTimeRoundingMethod_RoundHalfAwayFromZero);
if (CMTimeCompare(endTime, kCMTimeZero) != 0) {
double normalizedTime = (double) myPlayer.currentTime.value / (double) endTime.value;
self.timeSlider.value = normalizedTime;
}
}];
[myPlayer play];
}
-(IBAction)slider:(id)sender
{
Float64 currentSeconds = CMTimeGetSeconds(myPlayer.currentTime);
int mins = currentSeconds/60.0;
int secs = fmodf(currentSeconds, 60.0);
NSString *minsString = mins < 10 ? [NSString stringWithFormat:#"0%d", mins] : [NSString stringWithFormat:#"%d", mins];
NSString *secsString = secs < 10 ? [NSString stringWithFormat:#"0%d", secs] : [NSString stringWithFormat:#"%d", secs];
currentTimeLabel.text = [NSString stringWithFormat:#"%#:%#", minsString, secsString];
}
Your observer is being called when you tap on the uislider. This is why you have to tap on the slider first. Move the observer outside of the IBAction and possibly put it in the method that is run when the song starts.
-(IBAction)playButtonPressed:(UIButton *)sender
{
CMTime interval = CMTimeMake(33, 1000);
self.playbackObserver = [myPlayer addPeriodicTimeObserverForInterval:interval queue:dispatch_get_current_queue() usingBlock:^(CMTime time)
{
CMTime endTime = CMTimeConvertScale (myPlayer.currentItem.asset.duration, myPlayer.currentTime.timescale, kCMTimeRoundingMethod_RoundHalfAwayFromZero);
if (CMTimeCompare(endTime, kCMTimeZero) != 0) {
double normalizedTime = (double) myPlayer.currentTime.value / (double) endTime.value;
self.timeSlider.value = normalizedTime;
}
[self slider:nil];
}];
[myPlayer play];
}
Or your can look at this answer What gets called when a UISlider value changes? and setup the valueChange as an observer.

Skip over frames while processing video on iOS

I'm trying to process a local video file and simply do some analysis on the pixel data. Nothing is being output. My current code iterates through each frame of the video but I'd actually like to skip ~15 frames at a time to speed things up. Is there a way to skip over frames without decoding them?
In Ffmpeg, I could simply call av_read_frame without calling avcodec_decode_video2.
Thanks in advance! Here's my current code:
- (void) readMovie:(NSURL *)url
{
[self performSelectorOnMainThread:#selector(updateInfo:) withObject:#"scanning" waitUntilDone:YES];
startTime = [NSDate date];
AVURLAsset * asset = [AVURLAsset URLAssetWithURL:url options:nil];
[asset loadValuesAsynchronouslyForKeys:[NSArray arrayWithObject:#"tracks"] completionHandler:
^{
dispatch_async(dispatch_get_main_queue(),
^{
AVAssetTrack * videoTrack = nil;
NSArray * tracks = [asset tracksWithMediaType:AVMediaTypeVideo];
if ([tracks count] == 1)
{
videoTrack = [tracks objectAtIndex:0];
videoDuration = CMTimeGetSeconds([videoTrack timeRange].duration);
NSError * error = nil;
// _movieReader is a member variable
_movieReader = [[AVAssetReader alloc] initWithAsset:asset error:&error];
if (error)
NSLog(#"%#", error.localizedDescription);
NSString* key = (NSString*)kCVPixelBufferPixelFormatTypeKey;
NSNumber* value = [NSNumber numberWithUnsignedInt: kCVPixelFormatType_420YpCbCr8Planar];
NSDictionary* videoSettings = [NSDictionary dictionaryWithObject:value forKey:key];
AVAssetReaderTrackOutput* output = [AVAssetReaderTrackOutput
assetReaderTrackOutputWithTrack:videoTrack
outputSettings:videoSettings];
output.alwaysCopiesSampleData = NO;
[_movieReader addOutput:output];
if ([_movieReader startReading])
{
NSLog(#"reading started");
[self readNextMovieFrame];
}
else
{
NSLog(#"reading can't be started");
}
}
});
}];
}
- (void) readNextMovieFrame
{
//NSLog(#"readNextMovieFrame called");
if (_movieReader.status == AVAssetReaderStatusReading)
{
//NSLog(#"status is reading");
AVAssetReaderTrackOutput * output = [_movieReader.outputs objectAtIndex:0];
CMSampleBufferRef sampleBuffer = [output copyNextSampleBuffer];
if (sampleBuffer)
{ // I'm guessing this is the expensive part that we can skip if we want to skip frames
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
// Lock the image buffer
CVPixelBufferLockBaseAddress(imageBuffer,0);
// Get information of the image
uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(imageBuffer);
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
// do my pixel analysis
// Unlock the image buffer
CVPixelBufferUnlockBaseAddress(imageBuffer,0);
CFRelease(sampleBuffer);
[self readNextMovieFrame];
}
else
{
NSLog(#"could not copy next sample buffer. status is %d", _movieReader.status);
NSTimeInterval scanDuration = -[startTime timeIntervalSinceNow];
float scanMultiplier = videoDuration / scanDuration;
NSString* info = [NSString stringWithFormat:#"Done\n\nvideo duration: %f seconds\nscan duration: %f seconds\nmultiplier: %f", videoDuration, scanDuration, scanMultiplier];
[self performSelectorOnMainThread:#selector(updateInfo:) withObject:info waitUntilDone:YES];
}
}
else
{
NSLog(#"status is now %d", _movieReader.status);
}
}
- (void) updateInfo: (id*)message
{
NSString* info = [NSString stringWithFormat:#"%#", message];
[infoTextView setText:info];
}
If you want less accurate frame processing (not frame by frame) you should use AVAssetImageGenerator.
This class returns a frame for a specified time you asked.
Specifically, build an Array filled with times between the clip's duration with 0.5s difference between each time (iPhone films at about 29.3 fps if you want every 15 frames its about frame for every 30 seconds) and let the image generator returns your frames.
For each frame you can see the time you requested and the actual time of the frame. It's default value is around 0.5s tolerance from the time you asked but you can also change that by changing the properties:
requestedTimeToleranceBefore
and
requestedTimeToleranceAfter
I hope I answered your question,
Good luck.

NSString initWithFormat producing value with (null)

I'm having some issues creating a string using 'initWithFormat'. Here is the code I'm using:
- (void)convertSpeedUnits
{
NSString *speedUnits = [[NSUserDefaults standardUserDefaults] stringForKey:kSpeedUnits];
double speed;
if ([speedUnits isEqualToString:#"Knots"])
{
speed = ms2knots(currentSpeedMS);
}
else if ([speedUnits isEqualToString:#"MPH"])
{
speed = ms2kph(currentSpeedMS);
}
else if ([speedUnits isEqualToString:#"KPH"])
{
speed = ms2mph(currentSpeedMS);
}
NSString *speedLabel = [[NSString alloc] initWithFormat:#"%.2f %s", speed, speedUnits];
currentSpeed.text = speedLabel;
[speedLabel release];
}
I would expect speedLabel to be something like this...
'1.12 Knots' or '1.12 MPH' or '1.12 KPH'
however what I'm getting is the following
'1.12 (null)'
speedUnits is a NSString, so you should use %# and not %s:
NSString *speedLabel = [[NSString alloc] initWithFormat:#"%.2f %#", speed, speedUnits];

iphone - difference between time(app hiding) and time(app appearing)

how do i get the date from appEnterBackground and take away from appEnterForeground, then show the difference in a label.
This is my code so far..
**.h**
NSTimeInterval appEnteredBackground;
NSTimeInterval appEnteredForeground;
NSTimeInterval difference;
**.m**
- (void)applicationDidEnterBackground:(UIApplication *)application
{
appEnteredBackground = [NSDate timeIntervalSinceReferenceDate];
}
- (void)applicationWillEnterForeground:(UIApplication *)application
{
appEnteredForeground = [NSDate timeIntervalSinceReferenceDate];
difference = appEnteredForeground - appEnteredBackground;
NSLog(#"Duration is %#",[NSDate dateWithTimeIntervalSinceReferenceDate: difference]);
NSLog(#"Duration is %#", [NSString stringWithFormat:#"%f", difference]);
NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
[dateFormatter setDateStyle:NSDateFormatterMediumStyle];
NSString *time = [NSString stringWithFormat:#"%f", difference]; **//ERROR HERE (variable not used)**
[dateFormatter release];
}
Any help would be fantastic
I think that you do not have errors here. You are correctly calculating the time difference, then you build a string with its "description" but do not use it.
To see if it all works correctly try this before the end of the method:
NSLog(#"Computed time was: %#", time);
You should care about that time variable, since you are not actively using it.
To measure this type of event (or difference of events), you need something with resolution of less than a second. There is a "c" interface to get the machine time, which is the hardware free running "ticks" inside the processor.
You can get really fine timing (milliseconds, microseconds if you believe them) using this StopWatch class.
StopWatch.h
#import <Foundation/Foundation.h>
#interface StopWatch : NSObject
{
uint64_t _start;
uint64_t _stop;
uint64_t _elapsed;
}
-(void) Start;
-(void) Stop;
-(void) StopWithContext:(NSString*) context;
-(double) seconds;
-(NSString*) description;
+(StopWatch*) stopWatch;
-(StopWatch*) init;
#end
StopWatch.m
#import "StopWatch.h"
#include <mach/mach_time.h>
#implementation StopWatch
-(void) Start
{
_stop = 0;
_elapsed = 0;
_start = mach_absolute_time();
}
-(void) Stop
{
_stop = mach_absolute_time();
if(_stop > _start)
{
_elapsed = _stop - _start;
}
else
{
_elapsed = 0;
}
_start = mach_absolute_time();
}
-(void) StopWithContext:(NSString*) context
{
_stop = mach_absolute_time();
if(_stop > _start)
{
_elapsed = _stop - _start;
}
else
{
_elapsed = 0;
}
NSLog([NSString stringWithFormat:#"[%#] Stopped at %f",context,[self seconds]]);
_start = mach_absolute_time();
}
-(double) seconds
{
if(_elapsed > 0)
{
uint64_t elapsedTimeNano = 0;
mach_timebase_info_data_t timeBaseInfo;
mach_timebase_info(&timeBaseInfo);
elapsedTimeNano = _elapsed * timeBaseInfo.numer / timeBaseInfo.denom;
double elapsedSeconds = elapsedTimeNano * 1.0E-9;
return elapsedSeconds;
}
return 0.0;
}
-(NSString*) description
{
return [NSString stringWithFormat:#"%f secs.",[self seconds]];
}
+(StopWatch*) stopWatch
{
StopWatch* obj = [[[StopWatch alloc] init] autorelease];
return obj;
}
-(StopWatch*) init
{
[super init];
return self;
}
#end
The class has a static stopWatch method that returns an autoreleased object.
Once you call start, use the seconds method to get the elapsed time. Call start again to restart it. Or stop to stop it. You can still read the time (call seconds) anytime after calling stop.
Example In A Function (Timing call of execution)
-(void)SomeFunc
{
StopWatch* stopWatch = [StopWatch stopWatch];
[stopWatch Start];
... do stuff
[stopWatch StopWithContext:[NSString stringWithFormat:#"Created %d Records",[records count]]];
}