quick question about an operator I've written for myself.
Please excuse my poor-man's marble diagrams:
zip
aa--bb--cc--dd--ee--ff--------gg
--11----22--33--------44--55----
================================
--a1----b2--c3--------d4--e5----
combineLatest
aa--bb--cc--dd--ee--ff--------gg
--11----22--33--------44--55----
================================
--a1b1--c2--d3--e3--f3f4--f5--g5
zipLatest
aa--bb--cc--dd--ee--ff--------gg
--11----22--33--------44--55----
================================
--a1----c2--d3--------f4------g5
zipLatest (the one I wrote) fires at almost the same times as zip, but without the queueing zip includes.
I've already implemented it, I'm just wondering if this already exists.
I know I wrote a similar method in the past, to discover by random chance that I'd written the sample operator without knowing it.
So, does this already exist in the framework, or exist as a trivial composition of elements I haven't thought of?
Note: I don't want to rely on equality of my inputs to deduplicate (a la distinctUntilChanged).
It should work with a signal that only outputs "a" on an interval.
To give an update on the issue: There is still no operator for this requirement included in RxJS 6 and none seems to be planned for future releases. There are also no open pull requests that propose this operator.
As suggested here, a combination of combineLatest, first and repeat will produce the expected behaviour:
combineLatest(obs1, obs2).pipe(first()).pipe(repeat());
combineLatest will wait for the emission of both Observables - throwing away all emissions apart from the latest of each. first will complete the Observable after the emission and repeat resubscribes on combineLatest, causing it to wait again for the latest values of both observables.
The resubscription behaviour of repeat is not fully documented, but can be found in the GitHub source:
source.subscribe(this._unsubscribeAndRecycle());
Though you specifically mentions not to use DistinctUntilChanged, you can use it with a counter to distinct new values:
public static IObservable<(T, TSecond)> ZipLatest<T, TSecond>(this IObservable<T> source, IObservable<TSecond> second)
{
return source.Select((value, id) => (value, id))
.CombineLatest(second.Select((value, id) => (value, id)), ValueTuple.Create)
.DistinctUntilChanged(x => (x.Item1.id, x.Item2.id), new AnyEqualityComparer<int, int>())
.Select(x => (x.Item1.value, x.Item2.value));
}
public class AnyEqualityComparer<T1, T2> : IEqualityComparer<(T1 a, T2 b)>
{
public bool Equals((T1 a, T2 b) x, (T1 a, T2 b) y) => Equals(x.a, y.a) || Equals(x.b, y.b);
public int GetHashCode((T1 a, T2 b) obj) => throw new NotSupportedException();
}
Note that I've used Int32 here - because that's what Select() gives me - but it might be to small for some use cases. Int64 or Guid might be a better choice.
Related
I'm experimenting with Reactive Extensions on various platforms, and one thing that annoys me a bit are the glitches.
Even though for UI code these glitches might not be that problematic, and usually one can find an operator that works around them, I still find debugging code more difficult in the presence of glitches: the intermediate results are not important to debug, but my mind does not know when a result is intermediate or "final".
Having worked a bit with pure functional FRP in Haskell and synchronous data-flow systems, it also 'feels' wrong, but that is of course subjective.
But when hooking RX to non-UI actuators (like motors or switches), I think glitches are more problematic. How would one make sure that only the correct value is send to the external actuators?
Maybe this can be solved by some 'dispatcher' that knows when some 'external sensor' fired the initiating event, so that all internal events are handled before forwarding the final result(s) to the actuators. Something like described in the flapjax paper.
The question(s) I hope to get answers for are:
Is there something in RX that makes fixing glitches for synchronous notifications impossible?
If not, does a (preferably production quality) library or approach exists for RX that fixes synchronous glitches? Especially for the single-threaded Javascript this might make sense?
If no general solution exists, how would RX be used to control external sensors/actuators without glitches at the actuators?
Let me give an example
Suppose I want to print a sequence of tuples (a,b) where the contract is
a=n b=10 * floor(n/10)
n is a natural number stream = 0,1,2....
So I expect the following sequence
(a=0, b=0)
(a=1, b=0)
(a=2, b=0)
...
(a=9, b=0)
(a=10, b=10)
(a=11, b=10)
...
In RX, to make things more interesting, I will use filter for computing the b stream
var n = Observable
.Interval(TimeSpan.FromSeconds(1))
.Publish()
.RefCount();
var a = n.Select(t => "a=" + t);
var b = n.Where(t => t % 10 == 0)
.Select(t => "b=" + t);
var ab = a.CombineLatest(b, Tuple.Create);
ab.Subscribe(Console.WriteLine);
This gives what I believed to be a glitch (temporary violation of the invariant/contract):
(a=0, b=0)
(a=1, b=0)
(a=2, b=0)
...
(a=10, b=0) <-- glitch?
(a=10, b=10)
(a=11, b=10)
I realize that this is the correct behavior of CombineLatest, but I also thought this was called a glitch because in a real pure FRP system, you do not get these intermediate-invariant-violating results.
Note that in this example, I would not be able to use Zip, and also WithLatestFrom would give an incorrect result.
Of course I could just simplify this example into one monadic computation, never multi-casting the n stream occurrences (this would mean not being able to filter but just map), but that's not the point: IMO in RX you always get a 'glitch' whenever you split and rejoin an observable stream:
s
/ \
a b
\ /
t
For example, in FlapJAX you don't get these problems.
Does any of this make sense?
Thanks a lot,
Peter
Update: Let me try to answer my own question in an RX context.
First of all, it seems my understanding of what a "glitch" is, was wrong. From a pure FRP standpoint, what looked like glitches in RX to me, seems actually correct behavior in RX.
So I guess that in RX we need to be explicit about the "time" at which we expect to actuate values combined from sensors.
In my own example, the actuator is the console, and the sensor the interval n.
So if I change my code
ab.Subscribe(Console.WriteLine);
into
ab.Sample(n).Subscribe(Console.WriteLine);
then only the "correct" values are printed.
This does mean that when we get an observable sequence that combines values from sensors, that we must know all the original sensors, merge them all, and sample the values with that merged signal before sending any values to actuators...
So an alternative approach would be to "lift" IObservable into a "Sensed" structure that remembers and merges the originating sensors, for example like this:
public struct Sensed<T>
{
public IObservable<T> Values;
public IObservable<Unit> Sensors;
public Sensed(IObservable<T> values, IObservable<Unit> sensors)
{
Values = values;
Sensors = sensors;
}
public IObservable<Unit> MergeSensors(IObservable<Unit> sensors)
{
return sensors == Sensors ? Sensors : Sensors.Merge(sensors);
}
public IObservable<T> MergeValues(IObservable<T> values)
{
return values == Values ? Values : Values.Merge(values);
}
}
And then we must transfer all RX method to this "Sensed" structure:
public static class Sensed
{
public static Sensed<T> Sensor<T>(this IObservable<T> source)
{
var hotSource = source.Publish().RefCount();
return new Sensed<T>(hotSource, hotSource.Select(_ => Unit.Default));
}
public static Sensed<long> Interval(TimeSpan period)
{
return Observable.Interval(period).Sensor();
}
public static Sensed<TOut> Lift<TIn, TOut>(this Sensed<TIn> source, Func<IObservable<TIn>, IObservable<TOut>> lifter)
{
return new Sensed<TOut>(lifter(source.Values), source.Sensors);
}
public static Sensed<TOut> Select<TIn, TOut>(this Sensed<TIn> source, Func<TIn, TOut> func)
{
return source.Lift(values => values.Select(func));
}
public static Sensed<T> Where<T>(this Sensed<T> source, Func<T, bool> func)
{
return source.Lift(values => values.Where(func));
}
public static Sensed<T> Merge<T>(this Sensed<T> source1, Sensed<T> source2)
{
return new Sensed<T>(source1.MergeValues(source2.Values), source1.MergeSensors(source2.Sensors));
}
public static Sensed<TOut> CombineLatest<TIn1, TIn2, TOut>(this Sensed<TIn1> source1, Sensed<TIn2> source2, Func<TIn1, TIn2, TOut> func)
{
return new Sensed<TOut>(source1.Values.CombineLatest(source2.Values, func), source1.MergeSensors(source2.Sensors));
}
public static IDisposable Actuate<T>(this Sensed<T> source, Action<T> next)
{
return source.Values.Sample(source.Sensors).Subscribe(next);
}
}
My example then becomes:
var n = Sensed.Interval(TimeSpan.FromMilliseconds(100));
var a = n.Select(t => "a=" + t);
var b = n.Where(t => t % 10 == 0).Select(t => "b=" + t);
var ab = a.CombineLatest(b, Tuple.Create);
ab.Actuate(Console.WriteLine);
And again only the "desired" values are passed to the actuator, but with this design, the originating sensors are remember in the Sensed structure.
I'm not sure if any of this makes "sense" (pun intended), maybe I should just let go of my desire for pure FRP, and live with it. After all, time is relative ;-)
Peter
I am in the process of learning RX and have run across a sample on the Intro to Rx site that I have a question about. Here is the example which implements the same functionality as the Window with count extension method:
public static IObservable<IObservable<T>> MyWindow<T>(
this IObservable<T> source,
int count)
{
var shared = source.Publish().RefCount();
var windowEdge = shared
.Select((i, idx) => idx % count)
.Where(mod => mod == 0)
.Publish()
.RefCount();
return shared.Window(windowEdge, _ => windowEdge);
}
I understand the purpose of the var shared = source.Publish().RefCount() line to 'share' the source with the window edge query. What I don't understand is why the windowEdge query was also defined with the .Publish().RefCount()? Can someone please help me understand why this would be necessary?
Good Question!
Long Answer
Aside from performance reasons, the reason windowEdge is ref-counted has to do with the use of Select.
In this example, Select is using the index argument (idx), who's value is determined uniquely for each new subscriber. Therefore, if we did not ref-count windowEdge, each new subscriber would receive an event upon the next item yielded, since mod == 0 will always be true.
This means without ref-counting that each window would consist of exactly two values (assuming no other race conditions are introduced). Example:
When the first event fires, we create a new window and feed in the event, at which point we also use the window-closing selector to obtain an observable which will yield when the window should close. The next event fires, and is sent to the current window. That event also happens to be the first event that is sent to our window-closing observable (because mod == 0 is always true). The window-closing observable has now fired, and the window is closed, leaving us with a window which contains exactly two elements. Repeat.
TLDR
The use of ref-count for windowEdge is necessary to ensure we're only incrementing idx once per "MyWindow" observable.
Say for example I have an enumerable
dim e = Enumerable.Range(0, 1024)
I'd like to be able to do
dim o = e.ToObservable(Timespan.FromSeconds(1))
So that the observable would generate values every second
until the enumerable is exhausted. I can't figure a simple way to
do this.
You can use Interval together with Zip to get the desired functionality:
var sequence = Observable.Interval(TimeSpan.FromSeconds(1))
.Zip(e.ToObservable(), (tick, index) => index)
I have also looked for the solution and after reading the intro to rx made my self one:
There is an Observable.Generate() overload which I have used to make my own ToObservable() extension method, taking TimeSpan as period:
public static class MyEx {
public static IObservable<T> ToObservable<T>(this IEnumerable<T> enumerable, TimeSpan period)
{
return Observable.Generate(
enumerable.GetEnumerator(),
x => x.MoveNext(),
x => x,
x => x.Current,
x => period);
}
public static IObservable<T> ToObservable<T>(this IEnumerable<T> enumerable, Func<T,TimeSpan> getPeriod)
{
return Observable.Generate(
enumerable.GetEnumerator(),
x => x.MoveNext(),
x => x,
x => x.Current,
x => getPeriod(x.Current));
}
}
Already tested in LINQPad. Only concerning about what happens with the enumerator instance after the resulting observable is e.g. disposed. Any corrections appreciated.
You'd need something to schedule notifying observers with each value taken from the Enumerable.
You can use the recursive Schedule overload on an Rx scheduler.
Public Shared Function Schedule ( _
scheduler As IScheduler, _
dueTime As TimeSpan, _
action As Action(Of Action(Of TimeSpan)) _
) As IDisposable
On each scheduled invocation, simply call enumerator.MoveNext(), and call OnNext(enumerator.Current), and finally OnCompleted when MoveNext() returns false. This is pretty much the bare-bones way of doing it.
An alternative was to express your requirement is to restate it as "for a sequence, have a minimum interval between each value".
See this answer. The test case resembles your original question.
You could always do this very simple approach:
dim e = Enumerable.Range(0, 1024)
dim o = e.ToObservable().Do(Sub (x) Thread.Sleep(1000))
When you subscribe to o the values take a second to be produced.
I can only assume that you are using Range to dumb down your question.
Do you want every value that the Enumerable pushes to be delayed by a second?
var e = Enumerable.Range(0, 10);
var o = Observable.Interval(TimeSpan.FromSeconds(1))
.Zip(e, (_,i)=>i);
Or do you want only the last value of the Enumerable at each second to be pushed. i.e. reading from Enumerable that is evaluating as you enumerate it (perhaps some IO). In which case CombineLatest is more useful than Zip.
Or perhaps you just want to get a value every second, in which case just use the Observable.Interval method
var o = Observable.Interval(TimeSpan.FromSeconds(1));
If you explain your problem space then the community will be able to better help you.
Lee
*Excuse the C# answer, but I dont know what the equivalent VB.NET code would be.
Is there a way to have an observable sequence to resume execution with the next element in the sequence if an error occurs?
From this post it looks like you need to specify a new observable sequence in Catch() to resume execution, but what if you needed to just continue processing with the next element in the sequence instead? Is there a way to achieve this?
UPDATE:
The scenario is as follows:
I have a bunch of elements that I need to process. The processing is made up of a bunch of steps. I have
decomposed the steps into tasks that I would like to compose.
I followed the guidelines for ToObservable() posted here
to convert by tasks to an observables for composition.
so basically I'm doing somethng like so -
foreach(element in collection)
{
var result = from aResult in DoAAsync(element).ToObservable()
from bResult in DoBAsync(aResult).ToObservable()
from cResult in DoCAsync(bResult).ToObservable()
select cResult;
result.subscribe( register on next and error handlers here)
}
or I could something like this:
var result =
from element in collection.ToObservable()
from aResult in DoAAsync(element).ToObservable()
from bResult in DoBAsync(aResult).ToObservable()
from cResult in DoCAsync(bResult).ToObservable()
select cResult;
What is the best way here to continue processing other elements even if let's say the processing of
one of the elements throws an exception. I would like to be able to log the error and move on ideally.
Both James & Richard made some good points, but I don't think they have given you the best method for solving your problem.
James suggested using .Catch(Observable.Never<Unit>()). He was wrong when he said that "will ... allow the stream to continue" because once you hit an exception the stream must end - that is what Richard pointed out when he mentioned the contract between observers and observables.
Also, using Never in this way will cause your observables to never complete.
The short answer is that .Catch(Observable.Empty<Unit>()) is the correct way to change a sequence from one that ends with an error to one that ends with completion.
You've hit on the right idea of using SelectMany to process each value of the source collection so that you can catch each exception, but you're left with a couple of issues.
You're using tasks (TPL) just to turn a function call into an observable. This forces your observable to use task pool threads which means that the SelectMany statement will likely produce values in a non-deterministic order.
Also you hide the actual calls to process your data making refactoring and maintenance harder.
I think you're better off creating an extension method that allows the exceptions to be skipped. Here it is:
public static IObservable<R> SelectAndSkipOnException<T, R>(
this IObservable<T> source, Func<T, R> selector)
{
return
source
.Select(t =>
Observable.Start(() => selector(t)).Catch(Observable.Empty<R>()))
.Merge();
}
With this method you can now simply do this:
var result =
collection.ToObservable()
.SelectAndSkipOnException(t =>
{
var a = DoA(t);
var b = DoB(a);
var c = DoC(b);
return c;
});
This code is much simpler, but it hides the exception(s). If you want to hang on to the exceptions while letting your sequence continue then you need to do some extra funkiness. Adding a couple of overloads to the Materialize extension method works to keep the errors.
public static IObservable<Notification<R>> Materialize<T, R>(
this IObservable<T> source, Func<T, R> selector)
{
return source.Select(t => Notification.CreateOnNext(t)).Materialize(selector);
}
public static IObservable<Notification<R>> Materialize<T, R>(
this IObservable<Notification<T>> source, Func<T, R> selector)
{
Func<Notification<T>, Notification<R>> f = nt =>
{
if (nt.Kind == NotificationKind.OnNext)
{
try
{
return Notification.CreateOnNext<R>(selector(nt.Value));
}
catch (Exception ex)
{
ex.Data["Value"] = nt.Value;
ex.Data["Selector"] = selector;
return Notification.CreateOnError<R>(ex);
}
}
else
{
if (nt.Kind == NotificationKind.OnError)
{
return Notification.CreateOnError<R>(nt.Exception);
}
else
{
return Notification.CreateOnCompleted<R>();
}
}
};
return source.Select(nt => f(nt));
}
These methods allow you to write this:
var result =
collection
.ToObservable()
.Materialize(t =>
{
var a = DoA(t);
var b = DoB(a);
var c = DoC(b);
return c;
})
.Do(nt =>
{
if (nt.Kind == NotificationKind.OnError)
{
/* Process the error in `nt.Exception` */
}
})
.Where(nt => nt.Kind != NotificationKind.OnError)
.Dematerialize();
You can even chain these Materialize methods and use ex.Data["Value"] & ex.Data["Selector"] to get the value and selector function that threw the error out.
I hope this helps.
The contract between IObservable and IObserver is OnNext*(OnCompelted|OnError)? which is upheld by all operators, even if not by the source.
Your only choice is to re-subscribe to the source using Retry, but if the source returns the IObservable instance for every description you won't see any new values.
Could you supply more information on your scenario? Maybe there is another way of looking at it.
Edit: Based on your updated feedback, it sounds like you just need Catch:
var result =
from element in collection.ToObservable()
from aResult in DoAAsync(element).ToObservable().Log().Catch(Observable.Empty<TA>())
from bResult in DoBAsync(aResult).ToObservable().Log().Catch(Observable.Empty<TB>())
from cResult in DoCAsync(bResult).ToObservable().Log().Catch(Observable.Empty<TC>())
select cResult;
This replaces an error with an Empty which would not trigger the next sequence (since it uses SelectMany under the hood.
I'm writing some special purpose data structures in Java, intended for use in the browser, (compiled to JavaScript with GWT).
I'm trying to match the performance of some of the built-in JDK classes I'm noticing things run reasonably fast, but when I compare my code trace to some of the emulated JDK code, mine has lots of calls to dynamicCast and canCastUnsafe, while the JDK emulated classes do not. And it just about accounts for the difference in performance too...
Any GWT gurus out there know how to avoid this? It's amounting to a 20% overhead :-(
Details:
Here's the profile output (captured in Firebug) for 10,000 insertions of random integers, between 0 and 100,000 into two different data structures:
Google's TreeMap implementation for java.util.TreeMap (a red-black tree):
Profile (4058.602ms, 687545 calls)
Function Calls Percent Own Time
$insert_1 129809 41.87% 1699.367ms
$compare_0 120290 16% 649.209ms
$isRed 231166 13.33% 540.838ms
compareTo_0 120290 8.96% 363.531ms
$put_2 10000 6.02% 244.493ms
wrapArray 10000 3.46% 140.478ms
createFromSeed 10000 2.91% 118.038ms
$TreeMap$Node 10000 2.38% 96.706ms
initDim 10000 1.92% 77.735ms
initValues 10000 1.49% 60.319ms
$rotateSingle 5990 0.73% 29.55ms
TreeMap$Node 10000 0.47% 18.92ms
My Code (An AVL tree):
Profile (5397.686ms, 898603 calls)
Function Calls Percent Own Time
$insert 120899 25.06% 1352.827ms
$compare 120899 17.94% 968.17ms
dynamicCast 120899 14.12% 762.307ms <--------
$balanceTree 120418 13.64% 736.096ms
$setHeight 126764 8.93% 482.018ms
compareTo_0 120899 7.76% 418.716ms
canCastUnsafe 120899 6.99% 377.518ms <--------
$put 10000 2.59% 139.936ms
$AVLTreeMap$Node 9519 1.04% 56.403ms
$moveLeft 2367 0.36% 19.602ms
AVLTreeMap$State 9999 0.36% 19.429ms
$moveRight 2378 0.34% 18.295ms
AVLTreeMap$Node 9519 0.34% 18.252ms
$swingRight 1605 0.26% 14.261ms
$swingLeft 1539 0.26% 13.856ms
Additional observations:
Same problem for another data structure I made (SkipList).
dynamicCast is being applied in the compare function:
cmp = dynamicCast(right.key, 4).compareTo$(key);
dynamicCast goes away if the class does not implement Map (ie: just removing " implements Map" from the class. Doesn't matter if it's accessed through the interface or directly. This results in the same line compiling to:
cmp = right.key.compareTo$(key);
This is the relevant section of Java source from SkipList:
private int compare(Node a, Object o) {
if (comparator != null)
return comparator.compare((K) a.key, (K) o);
return ((Comparable<K>) a.key).compareTo((K) o);
}
public V get(Object k) {
K key = (K) k;
Node<K, V> current = head;
for (int i = head.height - 1; i >= 0; i--) {
Node<K, V> right;
while ((right = current.right[i]) != null) {
int cmp = compare(right, key);
...
}
}
}
Unfortunately I'm still not exactly clear on the cause, but from my experience, it seems from explicit casts, like:
((Comparable) obj).compareTo(other)
The Javascript generated looks like:
dynamicCast(obj, 1).compareTo(other);
Where 1 is a generated typeId representing the target of the cast. dynamicCast in turn calls canCastUnsafe and if false, it throws a ClassCastException. The value of this has been debated, since this would already be caught in hosted mode.
It can be sidestepped with JSNI:
public static native int compare(Object a, Object b) /*-{
return a.#java.lang.Comparable::compareTo(Ljava/lang/Object;)(b);
}-*/;
Dunno if you've seen this thread in the GWT Contributor's forum...
Basically, it starts with the same problem you've identified, proposes some new compiler flags, and goes on to show how to use some JSNI to get around the casts.
Edit In the GWT trunk there's a new compiler flag. See the wiki...
An updated answer for GWT version 2.1 and later:
Since GWT 2.1 (at least that's the first mention), the GWT compiler has a new compiler argument called -XdisableCastChecking that disables all runtime checking of casts.
Note, this option is marked as experimental (probably because this would make class cast exceptions very hard to debug).
In my app dynamicCast was called thousands of times in a short profile run, and were the the 3rd most time consuming method in the Firebug profiler. Using this compiler argument significantly reduced the number of "Long Duration Events" messages in the Chrome Speed Tracer.
See GWT Compiler Options for this and other Compiler arguments.
It's definitely a compiler problem: I have the problem on the following line:
final DefaultIconedSuggestBox<SuggestValueProxy, IconedValueHolderItem<SuggestValueProxy>> fieldValueWidget = getCategoryWidget().getFieldValueWidget();
I don't really know how I can workaround it: this line happens in a moment I'm changing from a module to another (it is maybe related to the code splitter issue: even though I'm not using code split: I'm just loading another page with another module)
Does the use of java 1.5 generics and wildcards could avoid this ?