I have written a subject that I use to push out messages that I receive from RabbitMQ.
I was expecting to always receive all messages on subscribe before finally is triggered but I am usually not receiving all messages and can see that that subscribe is called after finally is called.
_messageSummarySubject = new Subject<MessageSummary>();
_subscriber = _messageSummarySubject
.Finally(Finisehd)
.Buffer(TimeSpan.FromMilliseconds(100))
.SubscribeOn(Bootstrapper.Resolve<ISchedulerService>().DispatcherScheduler)
.ObserveOn(Bootstrapper.Resolve<ISchedulerService>().DispatcherScheduler)
.Subscribe(
x =>
{
//This is sometime called after finally
});
ThreadPool.QueueUserWorkItem(s => SomeAction(_messageSummarySubject));
SomeAction is basically calling
subject.OnNext(messageSummary);
a few times in a thread and once the thread is closed calling subject.OnCompleted();
Is this the correct behavior or am I doing something wrong?
Related
I am new for Vert.x async programming. I have event_bus consumer, once it receives a message, then it needs to call a Async. method async_method_to_access_database to get database records from Database. async_method_to_access_database will return a Future. The code looks like following.
Here I have one questions:
Inside event_bus consumer, could it run a Async. method? I doubt whether consumer handler supports to run Async. method? In Vertx, I understand Vertilce contains event loop, it supports Async. method, I am not sure if event_bus consumer handler supports it?
EventBus eb = vertx.eventBus();
MessageConsumer<String> consumer = eb.consumer("my_address");
consumer.handler(message -> {
Future<List<DataRecord>> listFuture = async_method_to_access_database();
listFuture.onSuccess(ar->{
doSomething();
});
});
You can have async code in the handler. The important part to understand that you have two ways of sending messages on the eventbus: request/publish.
With publish you just fire-and-forget
And with request you will register a handler wich waits until the consumer answers the message by calling message.reply().
In both cases your async code in the consumer will be executed, but when you use send, you have the option to have additional logic on the sender side, (e.g: repeat on error, validate response, etc)
EventBus eb = vertx.eventBus();
// sender
eb.request("my_address","testmessage", h -> {
if(h.succeeded()){
System.out.println("successful access to db");
}else{
System.out.println(h.cause());
}
});
// consumer
MessageConsumer<String> consumer = eb.consumer("my_address");
consumer.handler(message -> {
Future<List<DataRecord>> listFuture = async_method_to_access_database();
listFuture.onComplete(ar->{
if (ar.succeeded()) {
message.reply("updated elements successfully" + listFuture.size());
}
message.fail(1,"critical error") // will trigger error in sender
});
});
I am new to RX and have been investigating error handling and the use of Retry; I have the following (yes I know it's not a 'real' unit test but it gives me place to fiddle!!) and was wondering how I go about keeping the Retry but be able to log any Exception?
[Test]
public void Test()
{
var scheduler = new TestScheduler();
var source = scheduler.CreateHotObservable(
new Recorded<Notification<long>>(10000000, Notification.CreateOnNext(0L)),
new Recorded<Notification<long>>(20000000, Notification.CreateOnNext(1L)),
new Recorded<Notification<long>>(30000000, Notification.CreateOnNext(2L)),
new Recorded<Notification<long>>(30000001, Notification.CreateOnError<long>(new Exception("Fail"))),
new Recorded<Notification<long>>(40000000, Notification.CreateOnNext(3L)),
new Recorded<Notification<long>>(40000000, Notification.CreateOnCompleted<long>())
);
source.Retry().Subscribe(
l => Console.WriteLine($"OnNext {l}"),
exception => Console.WriteLine(exception.ToString()), // Would be logging this in production
() => Console.WriteLine("OnCompleted"));
scheduler.Start(
() => source,
0,
TimeSpan.FromSeconds(1).Ticks,
TimeSpan.FromSeconds(5).Ticks);
}
Which results in...
OnNext 0
OnNext 1
OnNext 2
OnNext 3
OnCompleted
...which is exactly what I want to happen apart from fact I would like to log the Exception which occurs between 2 and 3.
Is there a way to allow the Subscriber to see the Exception in OnError (and log it) and then re-subscribe so it sees 3?
Thx!
You could achieve that with this:
source
.Do(_ => { }, exception => Console.WriteLine(exception.ToString()), () => {})
.Retry()
.Subscribe(
l => Console.WriteLine($"OnNext {l}"),
// exception => Console.WriteLine(exception.ToString()), // Would be logging this in production
() => Console.WriteLine("OnCompleted")
);
Just to clarify what's going on here: OnError is a terminating signal. If the error reached the subscription, that would terminate the rest of the stream. .Retry terminates the subscription, swallows the OnError, and then re-subscribes, melding the two subscriptions together. For example look at this:
source
.StartWith(-1)
.Retry()
.Subscribe(
l => Console.WriteLine($"OnNext {l}"),
() => Console.WriteLine("OnCompleted")
);
Your output would be
OnNext -1
OnNext 0
OnNext 1
OnNext 2
OnNext -1
OnNext 3
OnCompleted
The OnNext -1 shows up twice, because it shows up whenever you subscribe (which Retry does after the OnError.
Your test observable is frankly a bad test. It breaks the "Rx Contract" which is that notifications follow the following pattern:
OnNext* (OnCompleted | OnError)?
That is, 0 or more OnNext notifications, followed by an optional OnError or an optional OnCompleted. No notifications of any type should follow either an OnError or an OnCompleted.
I have an Akka Actor that I want to send "control" messages to.
This Actor's core mission is to listen on a Kafka queue, which is a polling process inside a loop.
I've found that the following simply locks up the Actor and it won't receive the "stop" (or any other) message:
class Worker() extends Actor {
private var done = false
def receive = {
case "stop" =>
done = true
kafkaConsumer.close()
// other messages here
}
// Start digesting messages!
while (!done) {
kafkaConsumer.poll(100).iterator.map { cr: ConsumerRecord[Array[Byte], String] =>
// process the record
), null)
}
}
}
I could wrap the loop in a Thread started by the Actor, but is it ok/safe to start a Thread from inside an Actor? Is there a better way?
Basically you can but keep in mind that this actor will be blocking and a thumb of rule is to never block inside actors. If you still want to do this, make sure that this actor runs in a separate thread pool than the native one so you don't affect Actor System performances. One another way to do it would be to send messages to itself to poll new messages.
1) receive a order to poll a message from kafka
2) Hand over the
message to the relevant actor
3) Send a message to itself to order
to pull a new message
4) Hand it over...
Code wise :
case object PollMessage
class Worker() extends Actor {
private var done = false
def receive = {
case PollMessage ⇒ {
poll()
self ! PollMessage
}
case "stop" =>
done = true
kafkaConsumer.close()
// other messages here
}
// Start digesting messages!
def poll() = {
kafkaConsumer.poll(100).iterator.map { cr: ConsumerRecord[Array[Byte], String] =>
// process the record
), null)
}
}
}
I am not sure though that you will ever receive the stop message if you continuously block on the actor.
Adding #Louis F. answer; depending on the configuration of your actors they will either drop all messages that they receive if at the given moment they are busy or put them in a mailbox aka queue and the messages will be processed later (usually in FIFO manner). However, in this particular case you are flooding the actor with PollMessage and you have no guarantee that your message will not be dropped - which appears to happen in your case.
I'm using RxJava to pull out values from RabbitMQ. Here's the code:
val amqp = new RabbitQueue("queueName")
val obs = Observable[String](subscr => while (true) subscr onNext amqp.next)
obs subscribe (
s => println(s"String from rabbitmq: $s"),
error => amqp.connection.close
)
It works fine but now I have a requirement that a value should be pulled at most once per second while all the values should be preserved (so debounce won't do since it drops intermediary values).
It should be like amqp.next blocks thread so we're waiting... (RabbitMQ got two messages in queue) pulled a 1st message... wait 1 second... pulled a 2nd message... wait indefinitely for the next message...
How can I achieve this using rx methods?
Alternatively you could create a observable from a timer like that. I personally find this more elegant.
RabbitQueue amqp = new RabbitQueue("queueName");
Observable.timer(0, 1, TimeUnit.SECONDS)
.map(tick -> amp.next())
.subscribe(...)
One option may be to use the Schedulers API in combination with a PublishSubject as the observable.
Unfortunately, I don't know Scala syntax but here is the Java version you should be able to convert:
RabbitQueue amqp = new RabbitQueue("queueName");
Scheduler.Worker worker = Schedulers.newThread().createWorker();
PublishSubject<String> obs = PublishSubject.create();
worker.schedulePeriodically(new Action0() {
#Override
public void call() {
obs.onNext(amqp.next);
}
}, 1, 1, TimeUnit.SECONDS);
Your subscribe code from above would remain the same:
obs subscribe (
s => println(s"String from rabbitmq: $s"),
error => amqp.connection.close
)
I'm curious if it's possible to safely implement a self-cancelling poller without using a var to keep the instance of akka.actor.Cancellable
So far, I came up with something like what you see in the example below. However, I'm curious if it's safe to assume that the "tick" message will never be dispatched before the hotswap happens, i.e the line that schedules the poller:
tick(1, 5, context.system.scheduler.schedule(Duration.Zero, 3 seconds, self, "tick"))
is basically the same as:
val poll = context.system.scheduler.schedule(Duration.Zero, 3 seconds, self, "tick")
tick(1, 5, poll)
So, I would think that in some cases the first tick would be received before the hotswap has a chance to happen... Thoughts?
import akka.actor.{Cancellable, ActorSystem}
import akka.actor.ActorDSL._
import concurrent.duration._
object PollerDemo {
def run() {
implicit val system = ActorSystem("DemoPoller")
import system.dispatcher
actor(new Act{
become {
case "tick" => println("UH-OH!")
case "start" =>
become {
tick(1, 5, context.system.scheduler.schedule(Duration.Zero, 3 seconds, self, "tick"))
}
}
def tick(curr:Long, max:Long, poll:Cancellable):Receive = {
case "tick" => {
println(s"poll $curr/$max")
if(curr > max)
cancel(poll)
else
become{ tick(curr + 1, max, poll) }
}
}
def cancel(poll:Cancellable) {
println("cancelling")
poll.cancel()
println(s"cancelled successfully? ${poll.isCancelled}")
println("shutting down")
context.system.shutdown()
}
}) ! "start"
system.awaitTermination(1 minute)
}
}
My guess is that your code will be okay. Remember, actors only process their mailbox one at a time. When you receive the start message, you setup a timer that will deliver another message to the mailbox and then you swap the receive implementation. Because you do the receive swap while you are still processing the start message, then you will have already changed the actors's receive behavior before it processes the next message in the mailbox. So when it moves on to process the tick message you can be sure that it will be using the new receive behavior.
You could verify this by sending an additional tick message inside the first become like so:
become {
self ! "tick"
tick(1, 5, context.system.scheduler.schedule(Duration.Zero, 3 seconds, self, "tick"))
}
Here we are really eliminating the timer from the equation when asking if a message sent during the become block will be processed by the old receive or the new receive. I did not run this, but from my understanding or akka, both of these ticks should be handled by the new receive.
You really can't do pure functional programming with actors. Sending them messages is a side-effect. Since their receive function doesn't return a result, all the actor can do when receiving a message is to side effect. Just about every single thing your code does is for side-effects
You might be avoiding vars in your implementation of the code, but become is mutating a var in the Actor superclass. context.system.scheduler.schedule is clearly side-effecting and mutating state somewhere. Every single thing that cancel does is a side effect. system.awaitTermination(1 minute) is not a function...