Similar to this Flutter question I want to nest Streams.
In Flutter, this can be achieved easily by nesting StreamBuilders, however, I do not want to use widgets. Instead, I want to solve the problem in Dart alone. (nesting here means that one stream depends on the values from another stream and these should be combined)
Let me illustrate the problem:
Stream streamB(String a);
streamA: 'Hi' --- 'Hello' ---- 'Hey'
As you can see, I have a streamA that continuously emits events and a streamB that arises from the events that streamA emits. In a streamC, I want to be updated about every event from streamB.
Regular stream mapping
If I had valueB instead of streamB, I could simply use streamA.map((event) => valueB(event)), however, Stream.map can only handle synchronous values.
There is also Stream.asyncMap, however, that only works for Futures.
Then, there is also Stream.expand, but that works only for synchronous iterables.
Stream.asyncExpand
There is actually an Stream.asyncExpand method:
streamC = streamA.asyncExpand((event) => streamB(event));
However, this has the problem that the result stream (streamC) will only move on to the next event in the source stream (streamA) if the sub stream (streamB) of the first event has closed. In the case of say Cloud Firestore, this will never work because the sub stream will not close.
Stream.concurrentAsyncExpand
Luckily, there is the stream_transform package!
streamC = streamA.concurrentAsyncExpand((event) => streamB(event));
This package provides a concurrent async expand functionality. This way, the result stream does not wait for the sub streams to close.
However, this has the downside that the previous sub streams are not automatically closed when a new event in the source stream is received.
Thus, this is also not useful for Cloud Firestore.
Stream.switchMap
Also from the stream_transform package:
streamC = streamA.switchMap((event) => streamB(event));
This solves the problem I outlined above.
Related
I'm using a rxdart ZipStream within my app to combine two streams of incoming bluetooth data. Those streams are used along with "bufferCount" to collect 500 elements each before emitting. Everything works fine so far, but if the stream subscription gets cancelled at some point, there might be a number of elements in those buffers that are omitted after that. I could wait for a "buffer cycle" to complete before cancelling the stream subscription, but as this might take some time depending on the configured sample rate, I wonder if there is a solution to get those buffers as they are even if the number of elements might be less than 500.
Here is some simplified code for explanation:
subscription = ZipStream.zip2(
streamA.bufferCount(500),
streamB.bufferCount(500),
(streamABuffer, streamBBuffer) {
return ...;
},
).listen((data) {
...
});
Thanks in advance!
So for anyone wondering: As bufferCount is implemented with BufferCountStreamTransformer which extends BackpressureStreamTransformer, there is a dispatchOnClose property that defaults to true. That means if the underlying stream whose emitted elements are buffered is closed, then the remaining elements in that buffer are emitted finally. This also applies to the example above. My fault was to close the stream and to cancel the stream subscription instantly. With awaiting the stream's closing and cancelling the stream subscription afterwards, everything works as expected.
I'm running a live video streaming server. There's an Array[Byte] video source. Note that I can't get 2 connections to my video source. I want every client connecting to my server to receive this same stream, with a buffer discarding the old frames.
I tried using a BroadcastHub like this :
val source =
Source.fromIterator(() => myVideoStreamingSource.zipWithIndex)
val runnableGraph =
source.toMat(BroadcastHub.sink(bufferSize = 2))(Keep.right)
runnableGraph.run().to(Sink.foreach { index =>
println(s"client A reading frame #$index")
}).run()
runnableGraph.run().to(Sink.foreach { index =>
println(s"client B reading frame #$index")
}).run()
I get :
client A reading frame #0
client B reading frame #1
client A reading frame #2
client B reading frame #3
We see that the main stream is partitioned between the two clients, whereas I'd expect my two client being able to see all the source stream's frames.
Did I miss something, or is there any other solution ?
The issue is the combination of Iterator with BroadcastHub. I assume you myVideoStreamingSource is something like:
val myVideoStreamingSource = Iterator("A","B","C","D","E")
I'll now quote from BroadcastHub.Sink:
Every new materialization of the [[Sink]] results in a new, independent hub, which materializes to its own [[Source]] for consuming the [[Sink]] of that materialization.
The issue here for you, is that it does not yet consume the data from the iterator.
The thing with iterator, is that once you consumed its data, you won't get back to the beginning again. Add to that the fact that both graphs run in parallel, it looks like it "divides" the elements between the two. But actually that is completely random. For example, if you add a sleep of 1 second between the Client A and Client B, so the only client that will print will be A.
In order to get that work, you need to create a source that is reversible. For example, Seq, or List. The following will do:
val myVideoStreamingSource = Seq("A","B","C","D","E")
val source = Source.fromIterator(() => myVideoStreamingSource.zipWithIndex.iterator)
I am learning BLoC pattern in Flutter without using any package. What I know is data are sent as Sink and output is given as Stream in BLoC pattern. StreamController is there to handle these thing. Looks like both method make changes to the input data but I am confused about their purpose. Went through the documentation but couldn't understand.
Class CartBloc{
final _cart = Cart();
Sink<Product> get addition => _additionalController.sink;
final _additionController = StreamController<Product>();
Stream<int> get itemCount => _itemCountSubject.stream;
final _itemCountSubject = BehaviorSubject<int>();
CartBloc(){
_additionaController.stream.listen(_handle);
}
void _handle(Product product){
_cart.add(product);
_itemCountSubject.add(_cart.itemCount);
}
}
Above code is from Build reactive mobile apps with Flutter (Google I/O '18). They mentioned listen is necessary because we are not just capturing data but doing some other operation through the _handle() method. Now my question is can't we implement same with stream.transform(). If yes what will be the equivalent implementation and if no what is the reason.
stream.transform(transformer) will create a new stream that is somehow modified from the original. It does not start listening on the stream, and so no values will flow through the stream until it has a listener.
Think of transform as a way to change a stream, usually for some other bit of code to listen to. Think of listen as a way to react to values that come through a stream.
I have been successfully using FileIO to stream the contents of a file, compute some transformations for each line and aggregate/reduce the results.
Now I have a pretty specific use case, where I would like to stop the stream when a condition is reached, so that it is not necessary to read the whole file but the process finishes as soon as possible. What is the recommended way to achieve this?
If the stop condition is "on the outside of the stream"
There is a advanced building-block called KillSwitch that you could use to do this: http://doc.akka.io/japi/akka/2.4.7/akka/stream/KillSwitches.html The stream would get shut down once the kill switch is notified.
It has methods like abort(reason) / shutdown etc, see here for it's API: http://doc.akka.io/japi/akka/2.4.7/akka/stream/SharedKillSwitch.html
Reference documentation is here: http://doc.akka.io/docs/akka/2.4.8/scala/stream/stream-dynamic.html#kill-switch-scala
Example usage would be:
val countingSrc = Source(Stream.from(1)).delay(1.second,
DelayOverflowStrategy.backpressure)
val lastSnk = Sink.last[Int]
val (killSwitch, last) = countingSrc
.viaMat(KillSwitches.single)(Keep.right)
.toMat(lastSnk)(Keep.both)
.run()
doSomethingElse()
killSwitch.shutdown()
Await.result(last, 1.second) shouldBe 2
If the stop condition is inside the stream
You can use takeWhile to express any condition really, though sometimes take or limit may be also enough "take 10 lnes".
If your logic is very advanced, you could build a special stage that handles that special logic using statefulMapConcat that allows to express literally anything - so you could complete the stream whenever you want to "from the inside".
I'm confused by the behavior of a shared stream that is created using Rx.Observable.just.
For example:
var log = function(x) { console.log(x); };
var cold = Rx.Observable
.just({ foo: 'cold' });
cold.subscribe(log); // <-- Logs three times
cold.subscribe(log);
cold.subscribe(log);
var coldShare = Rx.Observable
.just({ foo: 'cold share' })
.share();
coldShare.subscribe(log); // <-- Only logs once
coldShare.subscribe(log);
coldShare.subscribe(log);
Both streams only emit one event, but the un-shared one can be subscribed to three times. Why is this?
I need to "fork" a stream but share its value (and then combine the forked streams).
How can I share the value of a stream but also subscribe to it multiple times?
I realize that this is probably related to the concept of "cold" and "hot" observables. However:
Is the stream created by Rx.Observable.just() cold or hot?
How is one supposed to determine the answer to the previous question?
Is the stream created by Rx.Observable.just() cold or hot?
Cold.
How is one supposed to determine the answer to the previous question?
I guess the documentation is the only guide.
How can I share the value of a stream but also subscribe to it multiple times?
You are looking for the idea of a connectable observable. By example:
var log = function(x) { console.log(x); };
var coldShare = Rx.Observable
.just({ foo: 'cold share' })
.publish();
coldShare.subscribe(log); // Does nothing
coldShare.subscribe(log); // Does nothing
coldShare.subscribe(log); // Does nothing
coldShare.connect(); // Emits one value to its three subscribers (logs three times)
var log = function(x) {
document.write(JSON.stringify(x));
document.write("<br>");
};
var coldShare = Rx.Observable
.just({ foo: 'cold share' })
.publish();
coldShare.subscribe(log); // <-- Only logs once
coldShare.subscribe(log);
coldShare.subscribe(log);
coldShare.connect();
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/4.0.7/rx.all.min.js"></script>
The example above logs three times. Using publish and connect, you essentially "pause" the observable until the call to connect.
See also:
How do I share an observable with publish and connect?
Are there 'hot' and 'cold' operators?
I don-t understand your first question, but about the last one, as I have been having problem getting that one too:
Rxjs implementation of Observables/Observers is based on the observer pattern, which is similar to the good old callback mechanism.
To exemplify, here is the basic form of creating an observable (taken from the doc at https://github.com/Reactive-Extensions/RxJS/blob/master/doc/api/core/operators/create.md)
var source = Rx.Observable.create(function (observer) {
observer.onNext(42);
observer.onCompleted();
// Note that this is optional, you do not have to return this if you require no cleanup
return function () {
console.log('disposed');
};
});
Rx.Observable.create takes as argument a function (say factory_fn to be original) which takes an observer. Your values are generated by a computation of your choice in the body of factory_fn, and because you have the observer in parameter you can process/push the generated values when you see fit. BUT factory_fn is not executed, it is just registered (like a callback would). It will be called everytime there is a subscribe(observer) on the related observable (i.e. the one returned by Rx.Observable.create(factory_fn).
Once subscription is done (creation callback called), values flow to your observer according to the logic in the factory function and it remains that way till your observable completes or the observer unsubscribes (supposing you did implement an action to cancel value flow as the return value of factory_fn).
What that basically means is by default, Rx.Observables are cold.
My conclusion after using quite a bit of the library, is that unless it is duely documented, the only way to know FOR SURE the temperature of an observable is to eye the source code. Or add a side effect somewhere, subscribe twice and see if the side effect happens twice or only once (which is what you did). That, or ask on stackoverflow.
For instance, Rx.fromEvent produce hot observables, as you can see from the last line in the code (return new EventObservable(element, eventName, selector).publish().refCount();). (code here : https://github.com/Reactive-Extensions/RxJS/blob/master/src/core/linq/observable/fromevent.js). The publish operator is among those operators which turns a cold observable into a hot one. How that works is out of scope so I won-t detail it here.
But Rx.DOM.fromWebSocket does not produce hot observables (https://github.com/Reactive-Extensions/RxJS-DOM/blob/master/src/dom/websocket.js). Cf. How to buffer stream using fromWebSocket Subject
Confusion often comes I think from the fact that we conflate the actual source (say stream of button clicks) and its representation (Rx.Observable). It is unfortunate when that happens but what we imagine as hot sources can end up being represented by a cold Rx.Observable.
So, yes, Rx.Observable.just creates cold observables.