Let me explain what I want to achieve first.
Lets say I have the following data incoming form the event stream
var data = new string[] {
"hello",
"Using",
"ok:michael",
"ok",
"begin:events",
"1:232",
"2:343",
"end:events",
"error:dfljsdf",
"fdl",
"error:fjkdjslf",
"ok"
};
When I subscribe the data source, I would like to get the following result
"ok:michael"
"ok"
"begin:events 1:232 2:343 end:events"
"error:dfljsdf"
"error:fjkdjslf"
"ok"
Basically, I want to get whichever data that start with ok or error and the data between begin and end.
I have tried this so far..
var data = new string[] {
"hello",
"Using",
"ok:michael",
"ok",
"begin:events",
"1:232",
"2:343",
"end:events",
"error:dfljsdf",
"fdl",
"error:fjkdjslf",
"ok"
};
var dataStream = Observable.Generate(
data.GetEnumerator(),
e => e.MoveNext(),
e => e,
e => e.Current.ToString(),
e => TimeSpan.FromSeconds(0.1));
var onelineStream = from d in dataStream
where d.StartsWith("ok") || d.StartsWith("error")
select d;
// ???
// may be need to buffer? I want to get data like "begin:events 1:232 2:343 end:events"
// but it is not working...
var multiLineStream = from list in dataStream.Buffer<string, string, string>(
bufferOpenings: dataStream.Where(d => d.StartsWith("begin")),
bufferClosingSelector: b => dataStream.Where(d => d.StartsWith("end")))
select String.Join(" ", list);
// merge two stream????
// but I have no clue how to merge these twos :(
mergeStream .Subscribe(d =>
{
Console.WriteLine(d);
Console.WriteLine();
});
Since I'm very new to Reactive programming, I can't make myself to think in reactive way. :(
Thanks in advance.
You were so, so very close to the right answer!
Essentially you had the onelineStream & multiLineStream queries just about right.
Merging them together is very easy. Just do this:
onelineStream.Merge(multiLineStream)
However, where your queries fell short was in the Observable.Generate that you used to introduce the delay between values. This creates a observable that, if you have multiple subscribers, kind of "fans out" the values.
Given your data and your definition for dataStream look how this code behaves:
dataStream.Select(x => "!" + x).Subscribe(Console.WriteLine);
dataStream.Select(x => "#" + x).Subscribe(Console.WriteLine);
You get these values:
!hello
#Using
!ok:michael
#ok
#1:232
!begin:events
#2:343
!end:events
!fdl
#error:dfljsdf
!error:fjkdjslf
#ok
Notice that some got handled by one subscription and the others got handled by the other. This means that even though your onelineStream & multiLineStream queries were just about right they would only see some of the data each and thus not behave as you expect.
You can also get race conditions that can skip and duplicate values. So it's best to avoid this kind of observable.
A better approach to introduce a delay between values is to do this:
var dataStream = data.ToObservable().Do(_ => Thread.Sleep(100));
Now this creates a "cold" observable, meaning that every new subscriber will get a fresh subscription of the observable so starting from the first value.
Your multiLineStream query will not work correctly on a cold observable.
To make the data stream a "hot" observable (which shares values amongst the subscribers) we use the Publish operator.
So, multiLineStream now looks like this:
var multiLineStream =
dataStream.Publish(ds =>
from list in ds.Buffer(
ds.Where(d => d.StartsWith("begin")),
b => ds.Where(d => d.StartsWith("end")))
select String.Join(" ", list));
You can then get your results like so:
onelineStream.Merge(multiLineStream).Subscribe(d =>
{
Console.WriteLine(d);
Console.WriteLine();
});
This is what I got:
ok:michael
ok
begin:events 1:232 2:343 end:events
error:dfljsdf
error:fjkdjslf
ok
Let me know if that works for you.
Related
I am trying to learn reactive programming, so forgive me if I ask a silly question. I'm also open to advice on changing my design.
I am working in scala-swing to display the results of a simulator. With one setting, a chart is displayed as a histogram; with the other setting the chart is displayed as the cumulative sum. (I'm probably using the wrong word; in the first setting you might have bin1=2, bin2=5, bin3=3; in the second setting the first height is 2, the second is 2 + 5, the third is 2 + 5 + 3, etc.). The simulator can be slow, so I originally used a Future to compute it, and the set the data into the chart. I decided to try a reactive approach, so my requirements are: 1. I don't want to recreate the data when I change the display mode, and 2. I want to set the Observable once for the chart and have the chart listen to the same Observable permanently.
I got this to work when I started the chain with a PublishSubject and the Future set the data into the start of the chain. When the display mode changed, I created a new PublishSubject().map(newRenderingLogic).subscribe(theChartsObservable). I am now trying to do what looks like the "right way," but it's not working correctly. I've tried to simplify what I have done:
val textObservable: Subject[String] = PublishSubject()
textObservable.subscribe(text => {
println(s"Text: ${text}")
})
var textSubscription: Option[Subscription] = None
val start = Observable.from(Future {
"Base text"
}).cache
var i = 0
val button = new Button() {
text = "Click"
reactions += {
case event => {
i += 1
if (textSubscription.isDefined) {
textSubscription.get.unsubscribe()
}
textSubscription = Some(start.map(((j: Int) => { (base: String) => s"${base} ${j}" })(i)).subscribe(textObservable))
}
}
}
On start, an Observable is created and logic to print some text is added to it. Then, an Observable with the generated data is created and a cache is added so that the result is replayed if the next subscription comes in after its results are generated. Then, a button is created. Then on button clicks a middle observable is chained with unique logic (it's a function that creates a function to append the value of i into the string, run with the current value of i; I tried to make something that couldn't just be reused) that is supposed to change with each click. Then the first Observable is subscribed to it so that the results of the whole chain end up being printed.
In theory, the cache operation takes care of not regenerating the data, and this works once, but onComplete is called on textObservable and then it can't be used again. It works if I subscribe it like this:
textSubscription = Some(start.map(((j: Int) => { (base: String) => s"${base} ${j}" })(i)).subscribe(text => textObservable.onNext(text)))
because the call to onComplete is intercepted, but this looks wrong and I wanted to know if there was a more typical way to do this, or architect it. It makes me think that I don't understand how this is supposed to be done if there isn't an out-of-the-box operation to do this.
Thank you.
I'm not 100% sure if I got the essence of your question right, but: if you have an Observable that may complete and you want to turn it into an Observable that never completes, you can just concatenate it with Observable.never.
For example:
// will complete after emitting those three elements:
val completes = Observable.from(List(1, 2, 3))
// will emit those three elements, but will never complete:
val wontComplete = completes ++ Observable.never
I need help i have list of message in which there are 3 fields. message, uid & time.
Its showing data but its mixed like when we enter data some goes in middle of array some goes in start some in end like this.
But i need to show time wise mean the latest or newest time off array will come first or in end like this. Attach the data format also. This is simply how i am getting message from service
getAllMessages(){
this.messageService.getAllMessages().subscribe((data)=>{
this.data = data;
console.log(this.data);
});
}
You need to use a pipe and map operator for this (map needs to be imported), that will then leverage sort method:
import { map } from 'rxjs/operators';
...
getAllMessages() {
this.messageService.getAllMessages().pipe(
map(messages => messages.sort((a:any, b:any) => b.time.seconds - a.time.seconds)
).subscribe(messages => this.data = messages)
}
I assumed you have seconds already so you do not have to transform date obj of any sort in this example. If it is not the case you can adjust this code accordingly I hope.
UPDATE
I think I've figured out the solution. I explain it in this video. Basically, use timeoutWith, and some tricks with zip (within zip).
https://youtu.be/0A7C1oJSJDk
If I have a single observable like this:
A-1-2--B-3-4-5-C--D--6-7-E
I want to put the "numbers" as lower priority; it should wait until the "letters" is filled up (a group of 2 for example) OR a timeout is reached, and then it can emit. Maybe the following illustration (of the desired result) can help:
A------B-1-----C--D-2----E-3-4-5-6-7
I've been experimenting with some ideas... one of them: first step is to split that stream (groupBy), one containing letters, and the other containing numbers..., then "something in the middle" happen..., and finally those two (sub)streams get merged.
It's that "something in the middle" what I'm trying to figure out.
How to achieve it? Is that even possible with RxJS (ver 5.5.6)? If not, what's the closest one? I mean, what I want to avoid is having the "numbers" flooding the stream, and not giving enough chance for the "letters" to be processed in timely manner.
Probably this video I made of my efforts so far can clarify as well:
Original problem statement: https://www.youtube.com/watch?v=mEmU4JK5Tic
So far: https://www.youtube.com/watch?v=HWDI9wpVxJk&feature=youtu.be
The problem with my solution so far (delaying each emission in "numbers" substream using .delay) is suboptimal, because it keeps clocking at slow pace (10 seconds) even after the "characters" (sub)stream has ended (not completed -- no clear boundary here -- just not getting more value for indeterminate amount of time). What I really need is, to have the "numbers" substream raise its pace (to 2 seconds) once that happen.
Unfortunately I don't know RxJs5 that much and use xstream myself (authored by one of the contributor to RxJS5) which is a little bit simpler in terms of the number of operators.
With this I crafted the following example:
(Note: the operators are pretty much the same as in Rx5, the main difference is with flatten wich is more or less like switch but seems to handle synchronous streams differently).
const xs = require("xstream").default;
const input$ = xs.of("A",1,2,"B",3,4,5,"C","D",6,7,"E");
const initialState = { $: xs.never(), count: 0, buffer: [] };
const state$ = input$
.fold((state, value) => {
const t = typeof value;
if (t === "string") {
return {
...state,
$: xs.of(value),
count: state.count + 1
};
}
if (state.count >= 2) {
const l = state.buffer.length;
return {
...state,
$: l > 0 ? xs.of(state.buffer[0]) : xs.of(value) ,
count: 0,
buffer: state.buffer.slice(1).concat(value)
};
}
return {
...state,
$: xs.never(),
buffer: state.buffer.concat(value),
};
}, initialState);
xs
.merge(
state$
.map(s => s.$),
state$
.last()
.map(s => xs.of.apply(xs, s.buffer))
)
.flatten()
.subscribe({
next: console.log
});
Which gives me the result you are looking for.
It works by folding the stream on itself, looking at the type of values and emitting a new stream depending on it. When you need to wait because not enough letters were dispatched I emit an emptystream (emits no value, no errors, no complete) as a "placeholder".
You could instead of emitting this empty stream emit something like
xs.empty().endsWith(xs.periodic(timeout)).last().mapTo(value):
// stream that will emit a value only after a specified timeout.
// Because the streams are **not** flattened concurrently you can
// use this as a "pending" stream that may or may not be eventually
// consumed
where value is the last received number in order to implement timeout related conditions however you would then need to introduce some kind of reflexivity with either a Subject in Rx or xs.imitate with xstream because you would need to notify your state that your "pending" stream has been consumed wich makes the communication bi-directionnal whereas streams / observables are unidirectionnal.
The key here the use of timeoutWith, to switch to the more aggresive "pacer", when the "events" kicks in. In this case the "event" is "idle detected in the higher-priority stream".
The video: https://youtu.be/0A7C1oJSJDk
I have a ReactiveList with keywords. The user can add or remove keyword from that list. The app needs to verify if the user typed one of the keywords.
There was already a similar post but it doesn't take in account a flexible list:
Using Reactive Extension for certain KeyPress sequences?
var keyElements = new ReactiveList<KeyElement>();
IObservable<IObservable<int>> rangeToMax = Observable.Merge(keyElements.ItemsAdded, keyElements.ItemsRemoved).Select(obs => Observable.Range(2, keyElements.Select(ke => ke.KeyTrigger.Length).Max()));
IObservable<IObservable<string>> detectedKeyTrigger = rangeToMax
.Select(n => _keyPressed.Buffer(n, 1))
.Merge().Where(m => keyElements.Where(ke => ke.KeyTrigger == m).Any());
//Here I want to end up with IObservable<string> instead of IObservable<IObservable<string>>
I can get rid of the outer IObservable by reassigning the detectedKeyTrigger each time an element in the reactive list changes, but then I lose all my subscriptions.
So, how can I end up with just an Observable of strings?
First off, both Max and Any have overloads which takes a selector and a predicate respectively. This negates the need of the Select.
Next, I changed the Observable.Merge to use the Changed property of ReactiveList which is the Rx version of INotifyCollectionChanged. I also changed the Select to produce an IEnumerable of ints instead; it just felt more Right™.
var keyElements = new ReactiveList<KeyElement>();
IObservable<IEnumerable<int>> rangeToMax = keyElements.Changed
.Select(_ => Enumerable.Range(2, keyElements.Max(keyElement => keyElement.KeyTrigger.Length));
IObservable<IObservable<string>> detectedKeyTrigger = rangeToMax.
.Select(range => range
.Select(length => _keyPressed.Buffer(length, 1).Select(chars => new string(chars.ToArray()))) // 1
.Merge() // 2
.Where(m => keyElements.Any(ke => ke.KeyTrigger == m)) // 3
.Switch(); // 4
Create an IObservable<string> which emits the last n characters typed by the user. Create such an observable for each of the possible lengths of an combo
Merge the observables in the IEnumerable<IObservable<string>> into one Observable<string>
Only let strings which mach one of the KeyTriggers through
As rangeToMax.Select produces an IObservable<IObservable<string>> we use Switch to only subscribe to the most recent IObservable<string> the IObservable<IObservable<string>> produces.
I've got an observable watching a log that is continuously being written too. Each line is a new onNext call. Sometimes the log outputs a single log item over multiple lines. Detecting this is easy, I just can't find the right RX call.
I'd like to find a way to collect the single log items into a List of lines, and onNext the list when the single log item is complete.
Buffer doesn't seem right as this isn't time based, it's algorithm based.
GroupBy might be what I want, but the documentation is confusing for it. It also seems that the observables it creates probably won't have onComplete called until the completion of the source observable.
This solution can't delay the log much (preferably not at all). I need to be reading the log as close to real time as possible, and order matters.
Any push in the right direction would be great.
This is a typical reactive parsing problem. You could use Rxx Parsers, or for a native solution you can build your own state machine with either Scan or by defining an async iterator. Scan is preferable for simple parsers and often uses a Scan-Where-Select pattern.
Async iterator state machine example: Turnstile
Scan parser example (untested):
IObservable<string> lines = ReadLines();
IObservable<IReadOnlyList<string>> parsed = lines.Scan(
new
{
ParsingItem = (IEnumerable<string>)null,
Item = (IEnumerable<string>)null
},
(state, line) =>
// I'm assuming here that items never span lines partially.
IsItem(line)
? IsItemLastLine(line)
? new
{
ParsingItem = (IEnumerable<string>)null,
Item = (state.ParsingItem ?? Enumerable.Empty<string>()).Concat(line)
}
: new
{
ParsingItem = (state.ParsingItem ?? Enumerable.Empty<string>()).Concat(line),
Item = (List<string>)null
}
: new
{
ParsingItem = (IEnumerable<string>)null,
Item = new[] { line }
})
.Where(result => result.Item != null)
.Select(result => result.Item.ToList().AsReadOnly());