I'm using a rxdart ZipStream within my app to combine two streams of incoming bluetooth data. Those streams are used along with "bufferCount" to collect 500 elements each before emitting. Everything works fine so far, but if the stream subscription gets cancelled at some point, there might be a number of elements in those buffers that are omitted after that. I could wait for a "buffer cycle" to complete before cancelling the stream subscription, but as this might take some time depending on the configured sample rate, I wonder if there is a solution to get those buffers as they are even if the number of elements might be less than 500.
Here is some simplified code for explanation:
subscription = ZipStream.zip2(
streamA.bufferCount(500),
streamB.bufferCount(500),
(streamABuffer, streamBBuffer) {
return ...;
},
).listen((data) {
...
});
Thanks in advance!
So for anyone wondering: As bufferCount is implemented with BufferCountStreamTransformer which extends BackpressureStreamTransformer, there is a dispatchOnClose property that defaults to true. That means if the underlying stream whose emitted elements are buffered is closed, then the remaining elements in that buffer are emitted finally. This also applies to the example above. My fault was to close the stream and to cancel the stream subscription instantly. With awaiting the stream's closing and cancelling the stream subscription afterwards, everything works as expected.
Related
Similar to this Flutter question I want to nest Streams.
In Flutter, this can be achieved easily by nesting StreamBuilders, however, I do not want to use widgets. Instead, I want to solve the problem in Dart alone. (nesting here means that one stream depends on the values from another stream and these should be combined)
Let me illustrate the problem:
Stream streamB(String a);
streamA: 'Hi' --- 'Hello' ---- 'Hey'
As you can see, I have a streamA that continuously emits events and a streamB that arises from the events that streamA emits. In a streamC, I want to be updated about every event from streamB.
Regular stream mapping
If I had valueB instead of streamB, I could simply use streamA.map((event) => valueB(event)), however, Stream.map can only handle synchronous values.
There is also Stream.asyncMap, however, that only works for Futures.
Then, there is also Stream.expand, but that works only for synchronous iterables.
Stream.asyncExpand
There is actually an Stream.asyncExpand method:
streamC = streamA.asyncExpand((event) => streamB(event));
However, this has the problem that the result stream (streamC) will only move on to the next event in the source stream (streamA) if the sub stream (streamB) of the first event has closed. In the case of say Cloud Firestore, this will never work because the sub stream will not close.
Stream.concurrentAsyncExpand
Luckily, there is the stream_transform package!
streamC = streamA.concurrentAsyncExpand((event) => streamB(event));
This package provides a concurrent async expand functionality. This way, the result stream does not wait for the sub streams to close.
However, this has the downside that the previous sub streams are not automatically closed when a new event in the source stream is received.
Thus, this is also not useful for Cloud Firestore.
Stream.switchMap
Also from the stream_transform package:
streamC = streamA.switchMap((event) => streamB(event));
This solves the problem I outlined above.
getUserMedia(constrains).then(stream => {
var recorder = new MediaRecorder(stream)
})
recorder.start()
recorder.pause()
// get new stream getUserMedia(constrains_new)
// how to update recorder stream here?
recorder.resume()
Is it possible? I've try to create MediaStream and use addTrack and removeTrack methods to change stream tracks but no success (recorder stops when I try to resume it with updated stream)
Any ideas?
The short answer is no, it's not possible. The MediaStream recording spec explicitly describes this behavior: https://w3c.github.io/mediacapture-record/#dom-mediarecorder-start. It's bullet point 15.3 of that algorithm which says "If at any point, a track is added to or removed from stream’s track set, the UA MUST immediately stop gathering data ...".
But in case you only want to record audio you can probably use an AudioContext to proxy your streams. Create a MediaStreamAudioDestinationNode and use the stream that it provides for recording. Then you can feed your streams with MediaStreamAudioSourceNodes and/or MediaStreamTrackAudioSourceNodes into the audio graph and mix them in any way you desire.
Last but not least there are currently plans to add the functionality you are looking for to the spec. Maybe you just have to wait a bit. Or maybe a bit longer depending on the browser you are using. :-)
https://github.com/w3c/mediacapture-record/issues/167
https://github.com/w3c/mediacapture-record/pull/186
I am writing a small CEP program using Siddhi. I can add a callback whenever a given filter outputs a data like this
executionPlanRuntime.addCallback("query1", new QueryCallback() {
#Override
public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) {
EventPrinter.print(inEvents);
System.out.println("data received after processing");
}
});
but is there is a way to know that the filter has finished processing and it won't give any more of the above callback. Something like didFinish. I think that would be the ideal place for shutting down SiddhiManager and ExecutionPlanRuntime instances.
No. There in no such functionality and can't be supported in the future also. Rationale behind that is, in real time stream processing queries will process the incoming stream and emit an output stream. There is no concept as 'finished processing'. Query will rather process event as long as there is input.
Since your requirement is to shutdown SiddhiManager and ExecutionPlanRuntime, recommended way is to do this inside some cleaning method of your program. Or else you can write some java code inside callback to count responses or time wait and call shutdown. Hope this helps!!
I'm confused by the behavior of a shared stream that is created using Rx.Observable.just.
For example:
var log = function(x) { console.log(x); };
var cold = Rx.Observable
.just({ foo: 'cold' });
cold.subscribe(log); // <-- Logs three times
cold.subscribe(log);
cold.subscribe(log);
var coldShare = Rx.Observable
.just({ foo: 'cold share' })
.share();
coldShare.subscribe(log); // <-- Only logs once
coldShare.subscribe(log);
coldShare.subscribe(log);
Both streams only emit one event, but the un-shared one can be subscribed to three times. Why is this?
I need to "fork" a stream but share its value (and then combine the forked streams).
How can I share the value of a stream but also subscribe to it multiple times?
I realize that this is probably related to the concept of "cold" and "hot" observables. However:
Is the stream created by Rx.Observable.just() cold or hot?
How is one supposed to determine the answer to the previous question?
Is the stream created by Rx.Observable.just() cold or hot?
Cold.
How is one supposed to determine the answer to the previous question?
I guess the documentation is the only guide.
How can I share the value of a stream but also subscribe to it multiple times?
You are looking for the idea of a connectable observable. By example:
var log = function(x) { console.log(x); };
var coldShare = Rx.Observable
.just({ foo: 'cold share' })
.publish();
coldShare.subscribe(log); // Does nothing
coldShare.subscribe(log); // Does nothing
coldShare.subscribe(log); // Does nothing
coldShare.connect(); // Emits one value to its three subscribers (logs three times)
var log = function(x) {
document.write(JSON.stringify(x));
document.write("<br>");
};
var coldShare = Rx.Observable
.just({ foo: 'cold share' })
.publish();
coldShare.subscribe(log); // <-- Only logs once
coldShare.subscribe(log);
coldShare.subscribe(log);
coldShare.connect();
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/4.0.7/rx.all.min.js"></script>
The example above logs three times. Using publish and connect, you essentially "pause" the observable until the call to connect.
See also:
How do I share an observable with publish and connect?
Are there 'hot' and 'cold' operators?
I don-t understand your first question, but about the last one, as I have been having problem getting that one too:
Rxjs implementation of Observables/Observers is based on the observer pattern, which is similar to the good old callback mechanism.
To exemplify, here is the basic form of creating an observable (taken from the doc at https://github.com/Reactive-Extensions/RxJS/blob/master/doc/api/core/operators/create.md)
var source = Rx.Observable.create(function (observer) {
observer.onNext(42);
observer.onCompleted();
// Note that this is optional, you do not have to return this if you require no cleanup
return function () {
console.log('disposed');
};
});
Rx.Observable.create takes as argument a function (say factory_fn to be original) which takes an observer. Your values are generated by a computation of your choice in the body of factory_fn, and because you have the observer in parameter you can process/push the generated values when you see fit. BUT factory_fn is not executed, it is just registered (like a callback would). It will be called everytime there is a subscribe(observer) on the related observable (i.e. the one returned by Rx.Observable.create(factory_fn).
Once subscription is done (creation callback called), values flow to your observer according to the logic in the factory function and it remains that way till your observable completes or the observer unsubscribes (supposing you did implement an action to cancel value flow as the return value of factory_fn).
What that basically means is by default, Rx.Observables are cold.
My conclusion after using quite a bit of the library, is that unless it is duely documented, the only way to know FOR SURE the temperature of an observable is to eye the source code. Or add a side effect somewhere, subscribe twice and see if the side effect happens twice or only once (which is what you did). That, or ask on stackoverflow.
For instance, Rx.fromEvent produce hot observables, as you can see from the last line in the code (return new EventObservable(element, eventName, selector).publish().refCount();). (code here : https://github.com/Reactive-Extensions/RxJS/blob/master/src/core/linq/observable/fromevent.js). The publish operator is among those operators which turns a cold observable into a hot one. How that works is out of scope so I won-t detail it here.
But Rx.DOM.fromWebSocket does not produce hot observables (https://github.com/Reactive-Extensions/RxJS-DOM/blob/master/src/dom/websocket.js). Cf. How to buffer stream using fromWebSocket Subject
Confusion often comes I think from the fact that we conflate the actual source (say stream of button clicks) and its representation (Rx.Observable). It is unfortunate when that happens but what we imagine as hot sources can end up being represented by a cold Rx.Observable.
So, yes, Rx.Observable.just creates cold observables.
I'm using libspotify SDK, C library for win32.
I think to have a right setup, every session callback is registered. I don't understand why i can't receive the call for end_of_track, while music_delivery continues to be called with zero padding 22050 long frames.
I attempt to start playing first loading the track with sp_session_load; till it returns SP_ERROR_IS_LOADING I post a message on my message queue (synchronization method I've used, PostMessage win32 API) in order to reload again with same API sp_session_load. As soon as it returns SP_ERROR_OK I use the sp_session_play and the music_delivery starts immediately, with correct frames.
I don't know why at the end of track the libspotify runtime then start sending zero padded frames, instead of calling end_of_track callback.
In other conditions it works perfectly: I've used the sp_track obtained from a album browse, so the track is fully loaded at the moment I load to the current session for playing: with this track, it works fine with end_of_track called correctly. In the case with padding error, I search the track using its Spotify URI and got the results; in this case the track metadata are not still ready (at the play attempt) so I used that kind of "polling" on sp_session_load with PostMessage.
Can anybody help me?
I ran into the same problem and I think the issue was that I was consuming the data too fast without giving other threads time to do any work since I was spending all of my time in the music_delivery callback. I found that if I add some throttling and notify the main thread that it can wake up to do some processing, the extra zeros at the end of track is reduced to one delivery of 22,050 frames (or 500ms at 44.1kHz).
Here is an example of what I added to my callback, heavily borrowed from the jukebox.c example provided with the SDK:
/* Buffer 1 second of data, then notify the main thread to do some processing */
if (g_throttle > format->sample_rate) {
pthread_mutex_lock(&g_notify_mutex);
g_notify_do = 1;
pthread_cond_signal(&g_notify_cond);
pthread_mutex_unlock(&g_notify_mutex);
// Reset the throttle counter
g_throttle = 0;
return 0;
}
As I said, there was still 22,050 frames of zeros delivered before the track stopped, but I believe libspotify may purposely do this to ensure that the duration calculated by the number of frames received (song_duration_ms = total_frames_delivered / sample_rate * 1000) is greater than or equal to the duration reported by sp_track_duration. In my case, the track I was trying to stream was 172,000ms in duration, without the extra padding the duration calculated is 171,796ms, but with the padding it was 172,296ms.
Hope this helps.