How to ignore the 60fps limit in javafx? - javafx-8

I need to create a 100fps animation that display 3d data from a file that contains 100 frames per second. But the AnimationTimer in javaFx allows me to get 60fps only. How to get over it?

Removing the JavaFX Frame Rate Cap
You can remove the 60fps JavaFX frame rate cap by setting a system property, e.g.,
java -Djavafx.animation.fullspeed=true MyApp
Which is an undocumented and unsupported setting.
Removing the JavaFX frame rate cap may make your application considerably less efficient in terms of resource usage (e.g. a JavaFX application without a frame rate cap will consume more CPU than an application with the frame rate cap in place).
Configuring the JavaFX Frame Rate Cap
Additionally, there is another undocumented system property you could try:
javafx.animation.framerate
I have not tried it.
Debugging JavaFX Frames (Pulses)
There are other settings like -Djavafx.pulseLogger=true which you could enable to help you debug the JavaFX architecture and validate that your application is actually running at the framerate you expect.
JavaFX 8 has a Pulse Logger (-Djavafx.pulseLogger=true system property) that "prints out a lot of crap" (in a good way) about the JavaFX engine's execution. There is a lot of provided information on a per-pulse basis including pulse number (auto-incremented integer), pulse duration, and time since last pulse. The information also includes thread details and events details. This data allows a developer to see what is taking most of the time.
Warning
Normal warnings for using undocumented features apply, as Richard Bair from the JavaFX team notes:
Just a word of caution, if we haven't documented the command line switches, they're fair game for removal / modification in subsequent releases :-)

Fullspeed=true will give you a high framerate without control (and thereby decrease performance of your app as it renders too much), framerate doesn't work indeed.
Use:
-Djavafx.animation.pulse=value
You can check your framerate with the following code. I checked if it actually works too by setting the pulserate to 2, 60 and 120 (I have a 240Hz monitor) and you see a difference in how fast the random number changes.
private final long[] frameTimes = new long[100];
private int frameTimeIndex = 0 ;
private boolean arrayFilled = false ;
Label label = new Label();
root.getChildren().add(label);
AnimationTimer frameRateMeter = new AnimationTimer() {
#Override
public void handle(long now) {
long oldFrameTime = frameTimes[frameTimeIndex] ;
frameTimes[frameTimeIndex] = now ;
frameTimeIndex = (frameTimeIndex + 1) % frameTimes.length ;
if (frameTimeIndex == 0) {
arrayFilled = true ;
}
if (arrayFilled) {
long elapsedNanos = now - oldFrameTime ;
long elapsedNanosPerFrame = elapsedNanos / frameTimes.length ;
double frameRate = 1_000_000_000.0 / elapsedNanosPerFrame ;
label.setText(String.format("Current frame rate: %.3f" +", Random number:" + Math.random(), frameRate));
}
}
};
frameRateMeter.start();

Related

Flutter compute function takes time to start execute

I am trying to use Flutters compute function to do some real time heavy image processing using a C++ code and dart ffi.
I tried wrapping the call to the heavy function in a compute to avoid messing with the ui thread and I took some time measurements to see what takes the most time to execute.
the code looks like this:
double _work(CheckPhotoData p) {
DateTime s = DateTime.now();
Pointer<Double> rPointer = Pointer.fromAddress(p.rPointerAddress);
Pointer<Double> gPointer = Pointer.fromAddress(p.gPointerAddress);
Pointer<Double> bPointer = Pointer.fromAddress(p.bPointerAddress);
final a = NativeCCode.checkPhoto(rPointer, gPointer, bPointer, p.w, 1);
print("ACTUAL NativeCCode.checkPhoto took: " + DateTime.now().difference(s).inMilliseconds.toString());
return a;
}
class CheckPhotoWrapper {
static Future<double> checkPhotoWrapper(Uint8List photo) async {
final CheckPhotoData deconstructData = _deconstructData(photo);
DateTime s = DateTime.now();
double res = await compute(_work, deconstructData);
print("compute took: " + DateTime.now().difference(s).inMilliseconds.toString());
return res;
}
...
}
After running the code I got this output:
ACTUAL NativeCCode.checkPhoto took: 106
compute took: 514
(this means that compute took 408ms more than the code it runs)
From what I understand from these results, the actual compute method from dart:async is taking much more time then the actual code its executing and causes a big overhead impacting the performance.
Even worse, my app UI is stuck when the processing starts.
Is there a way to reduce the overhead that compute introduces or a different approach this issue that I couldn't figure out?
Thanks for any idea or a solution to my problem.
Note:
I ran the test on debug mode on a physical device.
CheckPhotoData is a simple class containing the parameters to my _work function.
I am using flutter version 2.2.3, Channel stable
The overhead seems to be caused by debug mode. I saw a similar compute delay of several hundred milliseconds in my app (using Flutter 2.10.2), but when running in release mode it's less than 10 milliseconds.

How to minimize latency when reading audio with ALSA?

When trying to acquire some signals in the frequency domain, I've encountered the issue of having snd_pcm_readi() take a wildly variable amount of time. This causes problems in the logic section of my code, which is time dependent.
I have that most of the time, snd_pcm_readi() returns after approximately 0.00003 to 0.00006 seconds. However, every 4-5 call to snd_pcm_readi() requires approximately 0.028 seconds. This is a huge difference, and causes the logic part of my code to fail.
How can I get a consistent time for each call to snd_pcm_readi()?
I've tried to experiment with the period size, but it is unclear to me what exactly it does even after re-reading the documentation multiple times. I don't use an interrupt driven design, I simply call snd_pcm_readi() and it blocks until it returns -- with data.
I can only assume that the reason it blocks for a variable amount of time, is that snd_pcm_readi() pulls data from the hardware buffer, which happens to already have data readily available for transfer to the "application buffer" (which I'm maintaining). However, sometimes, there is additional work to do in kernel space or on the hardware side, hence the function call takes longer to return in these cases.
What purpose does the "period size" serve when I'm not using an interrupt driven design? Can my problem be fixed at all by manipulation of the period size, or should I do something else?
I want to achieve that each call to snd_pcm_readi() takes approximately the same amount of time. I'm not asking for a real time compliant API, which I don't imagine ALSA even attempts to be, however, seeing a difference in function call time on the order of being 500 times longer (which is what I'm seeing!) then this is a real problem.
What can be done about it, and what should I do about it?
I would present a minimal reproducible example, but this isn't easy in my case.
Typically when reading and writing audio, the period size specifies how much data ALSA has reserved in DMA silicon. Normally the period size specifies your latency. So for example while you are filling a buffer for writing through DMA to the I2S silicon, one DMA buffer is already being written out.
If you have your period size too small, then the CPU doesn't have time to write audio out in the scheduled execution slot provided. Typically people aim for a minimum of 500 us or 1 ms in latency. If you are doing heavy forms of computation, then you may want to choose 5 ms or 10 ms of latency. You may choose even more latency if you are on a non-powerful embedded system.
If you want to push the limit of the system, then you can request the priority of the audio processing thread be increased. By increasing the priority of your thread, you ask the scheduler to process your audio thread before all other threads with lower priority.
One method for increasing priority taken from the gtkIOStream ALSA C++ OO classes is like so (taken from the changeThreadPriority method) :
/** Set the current thread's priority
\param priority <0 implies maximum priority, otherwise must be between sched_get_priority_max and sched_get_priority_min
\return 0 on success, error code otherwise
*/
static int changeThreadPriority(int priority){
int ret;
pthread_t thisThread = pthread_self(); // get the current thread
struct sched_param origParams, params;
int origPolicy, policy = SCHED_FIFO, newPolicy=0;
if ((ret = pthread_getschedparam(thisThread, &origPolicy, &origParams))!=0)
return ALSA::ALSADebug().evaluateError(ret, "when trying to pthread_getschedparam\n");
printf("ALSA::Stream::changeThreadPriority : Current thread policy %d and priority %d\n", origPolicy, origParams.sched_priority);
if (priority<0) //maximum priority
params.sched_priority = sched_get_priority_max(policy);
else
params.sched_priority = priority;
if (params.sched_priority>sched_get_priority_max(policy))
return ALSA::ALSADebug().evaluateError(ALSA_SCHED_PRIORITY_ERROR, "requested priority is too high\n");
if (params.sched_priority<sched_get_priority_min(policy))
return ALSA::ALSADebug().evaluateError(ALSA_SCHED_PRIORITY_ERROR, "requested priority is too low\n");
if ((ret = pthread_setschedparam(thisThread, policy, &params))!=0)
return ALSA::ALSADebug().evaluateError(ret, "when trying to pthread_setschedparam - are you su or do you have permission to set this priority?\n");
if ((ret = pthread_getschedparam(thisThread, &newPolicy, &params))!=0)
return ALSA::ALSADebug().evaluateError(ret, "when trying to pthread_getschedparam\n");
if(policy != newPolicy)
return ALSA::ALSADebug().evaluateError(ALSA_SCHED_POLICY_ERROR, "requested scheduler policy is not correctly set\n");
printf("ALSA::Stream::changeThreadPriority : New thread priority changed to %d\n", params.sched_priority);
return 0;
}

Should I disconnect nodes that can't be used anymore?

I'm experimenting with Web Audio, and I made a function to play a note.
var context = new (window.AudioContext || window.webkitAudioContext)()
var make_triangle = function(destination, frequency, start, duration) {
var osc = context.createOscillator()
osc.type = "triangle"
osc.frequency.value = frequency
var gain = context.createGain()
osc.connect(gain)
gain.connect(destination)
// timing
osc.start(start)
osc.stop(start + 2*duration) // this line is discussed later
gain.gain.setValueAtTime(0.5, start)
gain.gain.linearRampToValueAtTime(0, start+duration)
}
Usage is something like this:
make_triangle(context.destination, 440, context.currentTime+1, 1)
This works just fine.
Firefox has a Web Audio tab in its developer console. When I play the sound, the Oscillator and Gain show up in the graph. Without the osc.stop(start + 2*duration) line, these linger forever. With the osc.stop(start + 2*duration) line, the Oscillator goes away, but the Gain stays connected to the AudioDestination forever.
I don't want to cause a memory leak or performance hit from having lots of old things still connected. To what extent do I need to clean up after creating nodes? Should I stop the oscillator? Disconnect everything? Both?
The Web Audio API is designed for oscillators to stop as soon as their 'note' finishes [1]. If you use the line osc.stop(start + 2*duration) then the oscillator will be disconnected from the gain and destroyed immediately.
If you don't plan on reusing the gain node that your oscillator was connected to, then I would suggest disconnecting it so that it can be garbage-collected.
Simply adding this callback to the oscillator within your make_triangle function would suffice:
...
osc.onended = function() {
gain.disconnect();
};
};
The callback is triggered as soon as the oscillator has ended its lifetime (ie: when the stop method has been called or scheduled with a timing parameter)
If you try this with Firefox's Web Audio tab open, you'll see that disconnected gain nodes are eventually garbage-collected (as long as nothing else is connected to the gain node).
Tip
Also, it's not a bad idea to have a single gain node that you keep connected to the AudioContext so that other nodes can connect up to it. This 'final-gain' is useful for mixing all other connected nodes and keeping them from clipping (ie: exceeding an amplitude of 1). You could pass this node into your make_triangle function as a parameter like so:
var context = new (window.AudioContext || window.webkitAudioContext)()
// Gain that will remain connected to `AudioContext`
var finalGain = context.createGain();
finalGain.connect(context.destination);
var make_triangle = function(destination, frequency, start, duration) {
var osc = context.createOscillator()
osc.type = "triangle"
osc.frequency.value = frequency
var gain = context.createGain()
osc.connect(gain)
gain.connect(destination)
// timing
osc.start(start)
osc.stop(start + 2*duration) // destroy osc and disconnect from gain
gain.gain.setValueAtTime(0.5, start)
gain.gain.linearRampToValueAtTime(0, start + duration)
// Added for cleanup:
osc.onended = function() {
gain.disconnect();
};
};
// Connect a new triangle osc to the finalGain
make_triangle(finalGain, 440, context.currentTime, 1);
If you don't want the oscillator to live forever, you definitely need to schedule it to stop eventually. Otherwise it will play forever, consuming resources. A really smart implementation might be be able to do something clever, but I wouldn't depend on that because that's not required.
When the oscillator stops it should automatically disconnect itself from any downstream nodes. If there are no other references to the oscillator or any of the downstream nodes either, then they should all be eventually collected without you having to do anything.
It is a bug in the implementation if this doesn't happen.
It could be a bug in Firefox's WebAudio developer tab that the gain node still appears. But it could also be a bug in Firefox's implementation.

CreateJS - Measuring FPS in RAF (requestAnimationFrame) mode

I am using CreateJS in the RAF mode
createjs.Ticker.timingMode = createjs.Ticker.RAF;
How do I integrate with for instance stats.js measure the browser's FPS?
Or is there any different, recommended way to measure the FPS with CreateJS?
Thanks!
You should be able to integrate using the instructions on the stats.js GitHub page
Essentially:
c.Ticker.on("tick", tick, this);
function tick(evt) {
stats.begin();
// do stuff, like stage.update();
stats.end();
}
Alternatively, you could look at Ticker.getMeasuredFPS and Ticker.getMeasuredTickTime.

libspotify C sending zeros at the end of track

I'm using libspotify SDK, C library for win32.
I think to have a right setup, every session callback is registered. I don't understand why i can't receive the call for end_of_track, while music_delivery continues to be called with zero padding 22050 long frames.
I attempt to start playing first loading the track with sp_session_load; till it returns SP_ERROR_IS_LOADING I post a message on my message queue (synchronization method I've used, PostMessage win32 API) in order to reload again with same API sp_session_load. As soon as it returns SP_ERROR_OK I use the sp_session_play and the music_delivery starts immediately, with correct frames.
I don't know why at the end of track the libspotify runtime then start sending zero padded frames, instead of calling end_of_track callback.
In other conditions it works perfectly: I've used the sp_track obtained from a album browse, so the track is fully loaded at the moment I load to the current session for playing: with this track, it works fine with end_of_track called correctly. In the case with padding error, I search the track using its Spotify URI and got the results; in this case the track metadata are not still ready (at the play attempt) so I used that kind of "polling" on sp_session_load with PostMessage.
Can anybody help me?
I ran into the same problem and I think the issue was that I was consuming the data too fast without giving other threads time to do any work since I was spending all of my time in the music_delivery callback. I found that if I add some throttling and notify the main thread that it can wake up to do some processing, the extra zeros at the end of track is reduced to one delivery of 22,050 frames (or 500ms at 44.1kHz).
Here is an example of what I added to my callback, heavily borrowed from the jukebox.c example provided with the SDK:
/* Buffer 1 second of data, then notify the main thread to do some processing */
if (g_throttle > format->sample_rate) {
pthread_mutex_lock(&g_notify_mutex);
g_notify_do = 1;
pthread_cond_signal(&g_notify_cond);
pthread_mutex_unlock(&g_notify_mutex);
// Reset the throttle counter
g_throttle = 0;
return 0;
}
As I said, there was still 22,050 frames of zeros delivered before the track stopped, but I believe libspotify may purposely do this to ensure that the duration calculated by the number of frames received (song_duration_ms = total_frames_delivered / sample_rate * 1000) is greater than or equal to the duration reported by sp_track_duration. In my case, the track I was trying to stream was 172,000ms in duration, without the extra padding the duration calculated is 171,796ms, but with the padding it was 172,296ms.
Hope this helps.