In my project scala-adapters I display log entries that are sent over a websocket.
As I have no control on how many entries are sent, I am looking for a strategy to avoid that the screen freezes.
I created a ScalaFiddle to simulate that: https://scalafiddle.io/sf/kzr28tq
This function with these parameters works perfectly:
setInterval(1000) { // note the absence of () =>
entries.value += (0 to 100).map(_.toString).mkString("")
}
If the interval gets smaller and the String longer - the screen freezes, e.g. with:
setInterval(100) { // note the absence of () =>
entries.value += (0 to 10000).map(_.toString).mkString("")
}
Is there a solution to solve that on the client side - or do I have to solve that on the server side?
You can try:
#dom
def render = {
<div>
{
for (entry <- entries) yield {
entryDiv(entry).bind
}
}
</div>
}
The problem is that you bind entries too early. Binding.scala does its magic by CPS transform, every .bind triggers re-evaluation of all code after, so you should bind a variable as late as possible.
And for Vars, use for comprehension instead of bind directly, to avoid updating the whole list. When you use += to modify the content of Vars, Binding.scala "patches" the list internally, but if you do .bind on the Vars instance directly to get the whole list, the framework cannot do any optimization for you.
Here is the updated ScalaFiddle: https://scalafiddle.io/sf/kzr28tq/3
Related
i'm a RxJava newcomer, and i'm having some trouble wrapping my head around how to do the following.
i'm using Retrofit to invoke a network request that returns me a Single<Foo>, which is the type i ultimately want to consume via my Subscriber instance (call it SingleFooSubscriber)
Foo has an internal property items typed as List<String>.
if Foo.items is not empty, i would like to invoke separate, concurrent network requests for each of its values. (the actual results of these requests are inconsequential for SingleFooSubscriber as the results will be cached externally).
SingleFooSubscriber.onComplete() should be invoked only when Foo and all Foo.items have been fetched.
fetchFooCall
.subscribeOn(Schedulers.io())
// Approach #1...
// the idea here would be to "merge" the results of both streams into a single
// reactive type, but i'm not sure how this would work given that the item emissions
// could be far greater than one. using zip here i don't think it would every
// complete.
.flatMap { foo ->
if(foo.items.isNotEmpty()) {
Observable.zip(
Observable.fromIterable(foo.items),
Observable.just(foo),
{ source1, source2 ->
// hmmmm...
}
).toSingle()
} else {
Single.just(foo)
}
}
// ...or Approach #2...
// i think this would result in the streams for Foo and items being handled sequentially,
// which is not really ideal because
// 1) i think it would entail nested streams (i get the feeling i should be using flatMap
// instead)
// 2) and i'm not sure SingleFooSubscriber.onComplete() would depend on the completion of
// the stream for items
.doOnSuccess { data ->
if(data.items.isNotEmpty()) {
// hmmmm...
}
}
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ data -> /* onSuccess() */ },
{ error -> /* onError() */ }
)
any thoughts on how to approach this would be greatly appreciated!
bonus points: in trying to come up with a solution to this, i've begun to question the decision to use the Single reactive type vs the Observable reactive type. most (all, except this one Foo.items case?) of my streams actually revolve around consuming a single instance of something, so i leaned toward Single to represent my streams as i thought it would add some semantic clarity around the code. anybody have any general guidance around when to use one vs the other?
You need to nest flatMaps and then convert back to Single:
retrofit.getMainObject()
.flatMap(v ->
Flowable.fromIterable(v.items)
.flatMap(w ->
retrofit.getItem(w.id).doOnNext(x -> w.property = x)
)
.ignoreElements()
.toSingle(v)
)
Here is my code.
var link = scala.collection.mutable.LinkedHashMap[String, String]()
var fieldTypeMapRDD = fixedRDD.mapPartitionsWithIndex((idx, itr) => itr.map(s => (s(8), s(9))))
fieldTypeMapRDD.foreach { i =>
println(i)
link.put(i._1, i._2)
}
println(link.size)// here size is zero
I want to access link out side loop .Please help.
Why your code is not supposed to work:
Before your foreach task is started, whole your function's closure inside foreach block is serialized and sent first to master, then to each of workers. This means each of them will have its own instance of mutable.LinkedHashMap as copy of link.
During foreach block each worker will put each of its items inside its own link copy
After your task is done you have still empty local link and several non-empty former copies on each of worker nodes.
Moral is clear: don't use local mutable collections with RDD. It's just not going to work.
One way to get whole collection to local machine is collect method.
You can use it as:
val link = fieldTypeMapRDD.collect.toMap
or in case of need to preserve the order:
import scala.collection.immutable.ListMap
val link = ListMap(fieldTypeMapRDD.collect:_*)
But if you are really into mutable collections, you can modify your code a bit. Just change
fieldTypeMapRDD.foreach {
to
fieldTypeMapRDD.toLocalIterator.foreach {
See also this question.
I'm trying to extract the parameter from a Lift Menu.param within a snippet so that I can use it to create a named Comet. However, I get a NullPointerException when I try to pass the parameter to the snippet using SnippetDisptach in my Boot.scala, as suggested here:
http://comments.gmane.org/gmane.comp.web.lift/44299
I've created the Menu item as follows:
object AnItemPage {
// create a parameterized page
def menu = Menu.param[Item]("Item", "Item",
s => fetchItem(s), item => item._id.toString) / "item"
private def fetchItem(s:String) : Box[Item] = synchronized {
ItemDAO.findById(ObjectId.massageToObjectId(s))
}
}
I've added the menu to SiteMap. I've also created a Snippet which I would like to pick up the Item parameter. (I'm using fmpwizard's InsertNamedComet library here):
class AddCometItemPage(boxedItem: Box[Item]) extends InsertNamedComet with DispatchSnippet{
val item : Item = boxedItem.openOr(null)
override lazy val name= "comet_item_" + item._id.toString
override lazy val cometClass= "UserItemCometActor"
def dispatch = null
}
My next step is to crate an instance of this class as demonstrated by David Pollak here:
http://comments.gmane.org/gmane.comp.web.lift/44299
This is what I have added to my Boot.scala:
LiftRules.snippetDispatch.append {
case "item_page" => new AddCometItemPage(AnItemPage.menu.currentValue)
}
My item.html references this snippet:
<div class="lift:item_page">
I get the following null pointer exception when I compile and run this:
Exception occurred while processing /item/5114eb4044ae953cf863b786
Message: java.lang.NullPointerException
net.liftweb.sitemap.Loc$class.siteMap(Loc.scala:147)
net.liftweb.sitemap.Menu$ParamMenuable$$anon$9.siteMap(Menu.scala:170)
net.liftweb.sitemap.Loc$class.allParams(Loc.scala:123)
net.liftweb.sitemap.Menu$ParamMenuable$$anon$9.allParams(Menu.scala:170)
net.liftweb.sitemap.Loc$class.net$liftweb$sitemap$Loc$$staticValue(Loc.scala:87)
net.liftweb.sitemap.Menu$ParamMenuable$$anon$9.net$liftweb$sitemap$Loc$$staticValue(Menu.scala:170)
net.liftweb.sitemap.Loc$$anonfun$paramValue$2.apply(Loc.scala:85)
net.liftweb.sitemap.Loc$$anonfun$paramValue$2.apply(Loc.scala:85)
net.liftweb.common.EmptyBox.or(Box.scala:646)
net.liftweb.sitemap.Loc$class.paramValue(Loc.scala:85)
net.liftweb.sitemap.Menu$ParamMenuable$$anon$9.paramValue(Menu.scala:170)
net.liftweb.sitemap.Loc$$anonfun$currentValue$3.apply(Loc.scala:114)
net.liftweb.sitemap.Loc$$anonfun$currentValue$3.apply(Loc.scala:114)
net.liftweb.common.EmptyBox.or(Box.scala:646)
net.liftweb.sitemap.Loc$class.currentValue(Loc.scala:114)
net.liftweb.sitemap.Menu$ParamMenuable$$anon$9.currentValue(Menu.scala:170)
bootstrap.liftweb.Boot$$anonfun$lift$8.apply(Boot.scala:107)
bootstrap.liftweb.Boot$$anonfun$lift$8.apply(Boot.scala:106)
net.liftweb.util.NamedPF$$anonfun$applyBox$1.apply(NamedPartialFunction.scala:97)
net.liftweb.util.NamedPF$$anonfun$applyBox$1.apply(NamedPartialFunction.scala:97)
net.liftweb.common.Full.map(Box.scala:553)
net.liftweb.util.NamedPF$.applyBox(NamedPartialFunction.scala:97)
net.liftweb.http.LiftRules.snippet(LiftRules.scala:711)
net.liftweb.http.LiftSession$$anonfun$net$liftweb$http$LiftSession$$findSnippetInstance$1.apply(LiftSession.scala:1506)
net.liftweb.http.LiftSession$$anonfun$net$liftweb$http$LiftSession$$findSnippetInstance$1.apply(LiftSession.scala:1506)
net.liftweb.common.EmptyBox.or(Box.scala:646)
net.liftweb.http.LiftSession.net$liftweb$http$LiftSession$$findSnippetInstance(LiftSession.scala:1505)
net.liftweb.http.LiftSession$$anonfun$locateAndCacheSnippet$1$1$$anonfun$apply$88.apply(LiftSession.scala:1670)
net.liftweb.http.LiftSession$$anonfun$locateAndCacheSnippet$1$1$$anonfun$apply$88.apply(LiftSession.scala:1669)
Has anybody any idea where I'm going wrong? I've not been able to find a lot of information on Menu.param.
Thank you very much for your help.
f
I have never tried what you are doing, so I am not sure the best way to accomplish it. The way you are using the Loc Param, you are extracting a variable from a URL pattern. In your case, http://server/item/ITEMID where ITEMID is the string representation of an Item, and which is the value that gets passed to the fetchItem function. The function call will not have a value if you just arbitrarily call it, and from what I can see you are requesting a value that is not initialized.
I would think there are two possible solutions. The first would be to use S.location instead of AnItemPage.menu.currentValue. It will return a Box[Loc[Any]] representing the Loc that is currently being accessed (with the parameters set). You can use that Loc to retrive currentValue and set your parameter.
The other option would be to instantiate the actor in your snippet. Something like this:
item.html
<div data-lift="AnItemPage">
<div id="mycomet"></div>
</div>
And then in your AnItemPage snippet, something like this:
class AnItemPage(item: Item) {
def render = "#mycomet" #> new AddCometItemPage(item).render
}
I haven't tested either of those, so they'll probably need some tweaking. Hopefully it will give you a general idea.
I am having some trouble using become in my Akka actor. Basically, my actor has a structure like so:
// This is where I store information received by the actor
// In my real application it has more fields, though.
case class Information(list:List[AnyRef]) {
def received(x:AnyRef) = {
Information(list :+ x)
}
}
class MyActor extends Actor {
// Initial receive block that simply waits for a "start" signal
def receive = {
case Start => {
become(waiting(Information(List())))
}
}
// The main waiting state. In my real application, I have multiple of
// these which all have a parameter of type "Information"
def waiting(info:Information):Receive = {
// If a certain amount of messages was received, I decide what action
// to take next.
if(someCondition) {
decideNextState(x)
}
return {
case Bar(x) => {
//
// !!! Problem occurs here !!!
//
// This is where the problem occurs, apparently. After a decision has been
// made, (i.e. decideNextState was invoked), the info list should've been
// cleared. But when I check the size of the info list here, after a decision
// has been made, it appears to still contain all the messages received
// earlier.
//
become(waiting(info received x))
}
}
}
def decideNextState(info:Information) {
// Some logic, then the received information list is cleared and
// we enter a new state.
become(waiting((Information(List())))
}
}
Sorry for the long code snippet, but I couldn't really make it any smaller.
The part where the problem occurs is marked in the comments. I am passing a parameter to the method that returns the Receive partial function which is then passed to the become method. However, the created partial function seems to somehow preserve state from an earlier invocation. I find the problem a bit difficult to explain, but I did my best to do so in the comments in the code, so please read those and I'll answer anything that is unclear.
Your logic is a little convoluted but I'll take a shot at what could be the problem:
If someCondition is true then your actor steps into a state, let's call it S1 characterized by a value Information(List()). And then you return (by the way, avoid using return unless it is absolutely necessary) a receive method which will put your actor into a state S2 characterized by a list Information(somePreviousList :+ x). So at this point your stack of states has S1 on top. But when you receive a Bar(x) message the state S2 will be pushed, thus covering S1 and you actually transition into a state characterized by an Information with the old values + your new x.
Or something like that, the recursion in your actor is a bit mesmerizing.
But I'll suggest rewriting that code since it seems that the state which changes is something of type Information and you are manipulating this state using Akka's actor state transitions which is not at all the best tool to do that. become and unbecome are meant to be used to transition from different states of the actor's behavior. That is, an actor can have a different behavior at any time and you use become and unbecome to change between these behaviors.
Why not do something like this ?
class MyActor extends Actor {
private var info = Information(List.empty)
def receive = {
case Start => info = Information(List()) //a bit redundant, but it's just to match 1:1 with your code
case Bar(x) => {
if (someCondition) {
info = Information(List.empty)
}
info = info received x
}
}
}
I might not have captured your entire idea, but you get the picture.
I've been using .NET Reactive Extensions to observe log events as they come in. I'm currently using a class that derives from IObservable and uses a ReplaySubject to store the logs, that way I can filter and replay the logs (for example: Show me all the Error logs, or show me all the Verbose logs) without losing the logs I've buffered.
The problem is, even though I've set a buffer size on the subject:
this.subject = new ReplaySubject<LogEvent>(10);
The memory usage of my program goes through the roof when I use OnNext to add to the observable collection on an infinite loop:
internal void WatchForNewEvents()
{
Task.Factory.StartNew(() =>
{
while (true)
{
dynamic parameters = new ExpandoObject();
// TODO: Add parameters for getting specific log events
if (this.logEventRepository.GetManyHasNewResults(parameters))
{
foreach (var recentEvent in this.logEventRepository.GetMany(parameters))
{
this.subject.OnNext(recentEvent);
}
}
// Commented this out for now to really see the memory go up
// Thread.Sleep(1000);
}
});
}
Does the buffer size on ReplaySubject not work? It doesn't seem to be clearing the buffer when the buffer size is reached. Any help much appreciated!
UPDATE:
I add subscribers like this (Is this wrong?):
public IDisposable Subscribe(IObserver<LogEvent> observer)
{
return this.subject.Subscribe(observer);
}
...which is called like:
// Inserts into UI ListView
this.logEventObservable.Subscribe(evt => this.InsertNewLogEvent(evt));
I'm not sure if this is the definitive answer, but I suspect that you're hitting an issue because of concurrency around the scheduler you're using. The constructor you're calling on ReplaySubject looks like this:
public ReplaySubject(int bufferSize)
: this(bufferSize, TimeSpan.MaxValue, Scheduler.CurrentThread)
{ }
The Scheduler.CurrentThread worries me. Try changing it to Scheduler.ThreadPool and see if that helps.
Also, as a side note, you seem to be mixing Rx with TPL and old fashioned thread sleeping. It's usually best to avoid doing that. You could change your WatchForNewEvents code to look like this:
dynamic parameters = new ExpandoObject();
var newEvents =
from n in Observable.Interval(TimeSpan.FromSeconds(1.0))
where this.logEventRepository.GetManyHasNewResults(parameters)
from recentEvent in
this.logEventRepository.GetMany(parameters).ToObservable()
select recentEvent;
newEvents.Subscribe(this.subject);
That's a nice compact Rx-y way of doing things.