I have the following design I'd like to create, but I'm not sure which Rx pattern matches it. The goal is more or less along the lines of a Single, but with a conditional check.
There is one Observable<String>, and the possibility of any number of observers.
If a request is first made, the observable will execute some network request taking in the string, then emit a callback (much like a completable/single)
Any subsequent call with the same key will return the same result immediately
However, if 5 minutes has passed and the same call is made, we will refetch the data as it may have expired, then emit it to any listeners. This result will be saved for another 5 minutes, and the cycle repeats.
All data is stored based on the key sent, much like a flyweight pattern. Expiration is based off of the last request time of the specific key.
My initial thought was to just make my own class with a concurrent hashmaps. However, this will mean I have to handle a lot of the threading mechanisms myself. I feel like RxJava will be a great solution to this, but I'm not sure if such patterns exist. Does anyone have an idea?
I get that the purpose of a Single<T> is meant to only retrieve a single response, so my terms may not be correct.
The following is my attempt, which I will be updating as I go
/**
* Created by Allan Wang on 07/01/18.
*
* Reactive flyweight to help deal with prolonged executions
* Each call will output a [Single], which may be new if none exist or the old one is invalidated,
* or reused if an old one is still valid
*
* Types:
* T input argument for caller
* C condition condition to check against for validity
* R response response within reactive output
*/
abstract class RxFlyweight<in T : Any, C : Any, R : Any> {
/**
* Given an input emit the desired response
* This will be executed in a separate thread
*/
protected abstract fun call(input: T): R
/**
* Given an input and condition, check if
* we may used cache data or if we need to make a new request
* Return [true] to use cache, [false] otherwise
*/
protected abstract fun validate(input: T, cond: C): Boolean
/**
* Given an input, create a new condition to be used
* for future requests
*/
protected abstract fun cache(input: T): C
private val conditionals = mutableMapOf<T, C>()
private val sources = mutableMapOf<T, Single<R>>()
private val lock = Any()
/**
* Entry point to give an input a receive a [Single]
* Note that the observer is not bound to any particular thread,
* as it is dependent on [createNewSource]
*/
operator fun invoke(input: T): Single<R> {
synchronized(lock) {
val source = sources[input]
// update condition and retrieve old one
val condition = conditionals.put(input, cache(input))
// check to reuse observable
if (source != null && condition != null && validate(input, condition))
return source
val newSource = createNewSource(input).cache()
sources.put(input, newSource)
return newSource
}
}
/**
* Open source creator
* Result will then be created with [Single.cache]
* If you don't have a need for cache,
* you likely won't have a need for flyweights
*/
open protected fun createNewSource(input: T): Single<R> =
Single.fromCallable { call(input) }
.timeout(20, TimeUnit.SECONDS)
.subscribeOn(Schedulers.io())
fun reset() {
synchronized(lock) {
sources.clear()
conditionals.clear()
}
}
}
Related
Here's update function from Coroutines StateFlow. I have two questions:
How is it atomic? We have multiple operations inside, how can atomicity be guaranteed without mutual exclusion?
Why is it in while(true) loop? At what condition the loop is necessary?
Can the loop be endless?
/**
* Updates the [MutableStateFlow.value] atomically using the specified [function] of its value.
*
* [function] may be evaluated multiple times, if [value] is being concurrently updated.
*/
public inline fun <T> MutableStateFlow<T>.update(function: (T) -> T) {
while (true) {
val prevValue = value
val nextValue = function(prevValue)
if (compareAndSet(prevValue, nextValue)) {
return
}
}
}
The goal of this function is to create a stream that emits values periodically until it encounters one that matches a predicate.
Here is some skeleton code that I've come up with:
class Watcher<T : Any>(
/**
* Emits the data associated with the provided id
*/
private val callable: (id: String) -> T,
/**
* Checks if the provided value marks the observable as complete
*/
private val predicate: (id: String, value: T) -> Boolean
) {
private val watchPool: MutableMap<String, Observable<T>> = ConcurrentHashMap()
fun watch(id: String): Observable<T> {
// reuse obesrvable if exists
val existing = watchPool[id]
if (existing != null)
return existing
val value = callable(id)
if (predicate(id, value)) return Observable.just(value)
// create new observable to fetch until complete,
// then remove from the map once complete
val observable = Observable.fromCallable<T> {
callable(id)
}.repeatWhen { /* What to put here? */ }.doOnComplete {
watchPool.remove(id)
}.distinctUntilChanged()
watchPool[id] = observable
return observable
}
}
As an example, if I have the following enums:
enum class Stage {
CREATED, PROCESSING, DELIVERING, FINISHED
}
And some callable that will retrieve the right stage, I should be able to pass the callable and a predicate checking if stage == FINISHED, and poll until I get the FINISHED event.
The issue I have is in generating an observable when the event received is not a final event. In that case, the observable should continue to poll for events until either it receives an event matching the predicate or until it has no more subscribers.
This observable should:
Not poll until it receives at least one subscriber
Poll every x seconds
Mark itself as complete if predicate returns true
Complete itself if it ever goes from >0 subscribers to 0 subscribers
The use of watch pools is simply to ensure that two threads watching the same id will not poll twice the number of times. Removal of observables from the map is also just so it doesn't pile up. For the same reason, observables that emit just one variable are not stored for reference.
How do I go about adding the functionality for the points added above?
I will link to one existing RxJava Github issue that I found useful, but from what I'm aware, it doesn't allow for predicates dealing with the value emitted by the callable.
I ended up with just using takeUntil, and using the observal's interval method to poll.
abstract class RxWatcher<in T : Any, V : Any> {
/**
* Emits the data associated with the provided id
* At a reasonable point, emissions should return a value that returns true with [isCompleted]
* This method should be thread safe, and the output should not depend on the number of times this method is called
*/
abstract fun emit(id: T): V
/**
* Checks if the provided value marks the observable as complete
* Must be thread safe
*/
abstract fun isCompleted(id: T, value: V): Boolean
/**
* Polling interval in ms
*/
open val pollingInterval: Long = 1000
/**
* Duration between events in ms for which the observable should time out
* If this is less than or equal to [pollingInterval], it will be ignored
*/
open val timeoutDuration: Long = 5 * 60 * 1000
private val watchPool: MutableMap<T, Observable<V>> = ConcurrentHashMap()
/**
* Returns an observable that will emit items every [pollingInterval] ms until it [isCompleted]
*
* The observable will be reused if there is polling, so the frequency remains constant regardless of the number of
* subscribers
*/
fun watch(id: T): Observable<V> {
// reuse observable if exists
val existing = watchPool[id]
if (existing != null)
return existing
val value = emit(id)
if (isCompleted(id, value)) return Observable.just(value)
// create new observable to fetch until complete,
// then remove from the map once complete
val observable = Observable.interval(pollingInterval, TimeUnit.MILLISECONDS, Schedulers.io()).map {
emit(id)
}.takeUntil {
isCompleted(id, it)
}.doOnComplete {
watchPool.remove(id)
}.distinctUntilChanged().run {
if (timeoutDuration > pollingInterval) timeout(timeoutDuration, TimeUnit.MILLISECONDS)
else this
}
watchPool[id] = observable
return observable
}
/**
* Clears the observables from the watch pool
* Note that existing subscribers will not be affected
*/
fun clear() {
watchPool.clear()
}
}
I hope the title accurately represents what I'm asking here. For my assignment here, I have a Speakerphone object, which is a singleton, and two interfaces, Listener and Talker with which Speakerphone needs to communicate.
There are two shoutMessage methods, the second of which takes a Class object as a parameter. Here's where I'm confused - I don't know if the "Class object" in the instructions is referring to the Listener interface, or the Speakerphone object. Can anyone tell me if the code in this method is correct, based on the instructions here?
public class Speakerphone extends Object {
private static Speakerphone sSpeaker = null;
private static HashSet<Listener> mListener = new HashSet<Listener>();
private static HashSet<Talker> mTalker = new HashSet<Talker>();
private Speakerphone() {
}
/*
* get
*
* #return the singleton instance of Speakerphone (Speakerphone)
*/
/************************************************
* ASSIGNMENT:
* Implement the get method
/************************************************/
public static Speakerphone get(){
if (sSpeaker == null){
sSpeaker = new Speakerphone();
}
return sSpeaker;
}
/*
* shoutMessage
*
* Sends the message to all of the Listeners tracked by Speakerphone
*
* #param talker a Talker whose message will be sent (Talker)
* #return nothing
*/
/************************************************
* ASSIGNMENT:
* Implement the shoutMessage method
/************************************************/
public void shoutMessage(Talker talker){
Iterator<Talker> talkIterator = mTalker.iterator();
while(talkIterator.hasNext())
talkIterator.next().getMessage();
}
/*
* shoutMessage
*
* Sends the message to all of the Listeners which are instances of
* the cls parameter
*
* #param talker a Talker whose message will be sent (Talker)
* #param cls a Class object representing the type which the Listener
* should extend from in order to receive the message (Class)
* #return nothing
*
* HINT: see Class.isAssignableFrom()
* http://docs.oracle.com/javase/7/docs/api/java/lang/Class.html#isAssignableFrom(java.lang.Class)
*/
/************************************************
* ASSIGNMENT:
* Implement the shoutMessage method
/************************************************/
public void shoutMessage(Talker talker, Class<?> cls){
if (Listener.class.isAssignableFrom(cls)){
Iterator<Talker> talkIterator = mTalker.iterator();
while(talkIterator.hasNext())
talkIterator.next().getMessage();
}
}
}
So, the class parameter in the shoutMessage method appears to be there so you can select out only listeners of a certain type. Class in Java is an object that describes a data type - a meta of meta so to speak.
Based on the comments in the code it looks like your shoutMessage method might be doing the opposite of what you want. Here's what I think it needs to do (based on the comments of the method):
Iterate through all of the listeners in your mListener HashSet. For each listener in the HashSet, do the following
As you iterate through, use Listener.class.isAssignableFrom to determine if the current listener is of class cls.
If it is of the type cls then call talker.getMessage() and pass that message to the current listener (by calling "receiveMessage" or "on" or whatever function is defined in your Listener interface).
If it is not of the type cls then ignore.
I have an issue where a ScheduledThreadPoolExecutor ends up with 3 million future tasks. I am trying to see what type of task so I can go to where that task is scheduled, but I am not sure how to get any info from this screen(I have tried right clicking those future tasks and selecting various choices in the menu). It seems like there is something missing in the gui like the links to the actual runnables or something...
any ideas on how to drill into further?
Some General Stuff
You need to know, that if you have a portable heap dump (phd, see types here), then it does not contain actual data (primitives), so then you can make your findings only based on reference map (which types hold a reference to which other types).
You can give a try to OQL. This is an SQL like language, with which you can query your objects.
One example:
select * from java.lang.String s where s.#retainedHeapSize>10000
This gives back all strings, that are bigger than ~10k.
You can make also some functions (like this aggregating here).
You could give a try to it.
As for the current problem
If you check the FutureTask source (here is JDK6 below):
public class FutureTask<V> implements RunnableFuture<V> {
/** Synchronization control for FutureTask */
private final Sync sync;
...
public FutureTask(Callable<V> callable) {
if (callable == null)
throw new NullPointerException();
sync = new Sync(callable);
}
...
public FutureTask(Runnable runnable, V result) {
sync = new Sync(Executors.callable(runnable, result));
}
The actual Runnable is referred by the Sync object:
private final class Sync extends AbstractQueuedSynchronizer {
private static final long serialVersionUID = -7828117401763700385L;
/** State value representing that task is running */
private static final int RUNNING = 1;
/** State value representing that task ran */
private static final int RAN = 2;
/** State value representing that task was cancelled */
private static final int CANCELLED = 4;
/** The underlying callable */
private final Callable<V> callable;
/** The result to return from get() */
private V result;
/** The exception to throw from get() */
private Throwable exception;
/**
* The thread running task. When nulled after set/cancel, this
* indicates that the results are accessible. Must be
* volatile, to ensure visibility upon completion.
*/
private volatile Thread runner;
Sync(Callable<V> callable) {
this.callable = callable;
}
So in the GUI open the Sync object (not open in your picture), and then you can check the Runnables.
I dont know if you can change the code or not, but in general it is better always limit the size of the queue used by an executor, since this way you can avoid leaks. Or you can use some persisted queue. If you apply a limit you can define the rejection policy like for example reject, run in caller and so on. See http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html for details.
I use Acceleo in order to generate code with a model I have made. I managed to protect my methods in order to protect them usinig "#generated NOT" in case I need to regenerate my code with Acceleo. The problem is that adding #generated NOT protect all the method content, that is to say the body, the signature and JavaDocs.
The thing is that I only need to keep the method body, or at least the method body and its signature, but I need the doc to be updated. How can I do this ?
Just for information here is an example of a potential generated class :
/*
* #generated
*/
public class ActeurRefEntrepriseServicesImpl implements ActeurRefEntrepriseServices {
#Autowired
HelloWorldService helloWorldService;
/**
* Service which say hello
*
* #param name
* user name
* #return print Hello username
*
* #generated NOT
*/
#Override
public void sayHello(final String name) {
helloWorldService.print(name);
}
}
Baptiste,
The #generated tags use the standard EMF protection rules : "#generated" means that the body of the block for which it is set will be generated, anything else means no re-generation. If you set something as "#generated" in any of your metamodels' generated code, you will see that there, too, the javadoc is preserved whatever the edits you do.
In short, you cannot tell EMF to re-generate anything other than the code itself.
If you need to have the body protected but not the javadoc, you have to shift from the "#generated" protection to Acceleo's [protected] blocks. i.e, change your template from :
[template generatedMethod(methodName : String)]
/**
* Some doc.
* #param param1
* param documentation.
* #generated
*/
[generateSignature(methodName)/] {
[generateBody()/]
}
[/template]
to something using a protected block :
[template generatedMethod(methodName : String)]
/**
* Some doc.
* #param param1
* param documentation.
*/
[protected (methodName)]
[generateSignature(methodName)/] {
[generateBody()/]
}
[/protected]
[/template]
With this paradigm, anything that is outside of the protected area will be regenerated, everything else will remain untouched by a regeneration.
See also the full documentation available from the Acceleo website.
If you absolutely need to use the "#generated" protection method for your model, you will need to tamper with the JMerger API from EMF and alter the launcher Acceleo generated for you in order to use your own merging strategy (see the getGenerationStrategy method from that launcher). Note that this is by no means an easy task.