Ok, I've been stuggling with this one for a while, and have spent a lot of time trying different things to do something that I have done very easily using PHP.
I am trying to iterate over a list while keeping track of a variable locally, while spitting out HTML attempting to populate a table.
Attempt #1:
#{
var curDate : Date = null
for(ind <- indicators){
if(curDate == null || !curDate.equals(ind.getFirstFound())){
curDate = ind.getFirstFound()
<tr><th colspan='5' class='day'>#(ind.getFirstFound())</th></tr>
<tr><th>Document ID</th><th>Value</th><th>Owner</th><th>Document Title / Comment</th></tr>
}
}
}
I attempt too user a scala block statement to allow me to keep curDate as a variable within the created scope. This block correctly maintains curDate state, but does not allow me to output anything to the DOM. I did not actually expect this to compile, due to my unescaped, randomly thrown in HTML, but it does. this loop simply places nothing on the DOM, although the decision structure is correctly executed on the server.
I tried escaping using #Html('...'), but that produced compile errors.
Attempt #2:
A lot of google searches led me to the "for comprehension":
#for(ind <- indicators; curDate = ind.getFirstFound()){
#if(curDate == null || !curDate.equals(ind.getFirstFound())){
#(curDate = ind.getFirstFound())
}
<tr><th colspan='5' class='day'>#(ind.getFirstFound())</th></tr>
<tr><th>Document ID</th><th>Value</th><th>Owner</th><th>Document Title / Comment</th></tr>
}
Without the if statement in this block, this is the closest I got to doing what I actually wanted, but apparently I am not allowed to reassign a non-reference type, which is why I was hoping attempt #1's reference declaration of curDate : Date = null would work. This attempt gets me the HTML on the page (again, if i remove the nested if statement) but doesn't get me the
My question is, how do i implement this intention? I am very painfully aware of my lack of Scala knowledge, which is being exacerbated by Play templating syntax. I am not sure what to do.
Thanks in advance!
Play's template language is very geared towards functional programming. It might be possible to achieve what you want to achieve using mutable state, but you'll probably be best going with the flow, and using a functional solution.
If you want to maintain state between iterations of a loop in functional programming, that can be done by doing a fold - you start with some state, and on each iteration, you get the previous state and the next element, and you then return the new state based on those two things.
So, looking at your first solution, it looks like what you're trying to do is only print an element out if it's date is different from the previous one, is that correct? Another way of putting this is you want to filter out all the elements that have a date that's the same date as the previous one. Expressing that in terms of a fold, we're going to fold the elements into a sequence (our initial state), and if the last element of the folded sequence has a different date to the current one, we add it, otherwise we ignore it.
Our fold looks like this:
indicators.foldLeft(Vector.empty[Indicator]) { (collected, next) =>
if (collected.lastOption.forall(_.getFirstFound != next.getFirstFound)) {
collected :+ next
} else {
collected
}
}
Just to explain the above, we're folding into a Vector because Vector has constant time append and last, List has n time. The forall will return true if there is no last element in collected, otherwise if there is, it will return true if the passed in lambda evaluates to true. And in Scala, == invokes .equals (after doing a null check), so you don't need to use .equals in Scala.
So, putting this in a template:
#for(ind <- indicators.foldLeft(Vector.empty[Indicator]) { (collected, next) =>
if (collected.lastOption.forall(_.getFirstFound != next.getFirstFound)) {
collected :+ next
} else {
collected
}
}){
...
}
Related
I am working on a package that deals with citations and most of them are of the form
Author, Year, Journal, Volume, Page, DOI
So a a string with series of fields separated by commas.Unfortunately some (~5%) are missing one (or often more than one) of these fields.
To do useful things with them I need to be able to check if two are the same ignoring a field if it is missing. I have an __eq__(self, other) defined that does this with a series of if statements like this:
elif hasattr(self, 'V') and hasattr(other, 'V') and getattr(self, 'V') != getattr(other, 'V'):
return False
The one constant about the citations is that author is present and at least one of year or journal is too.
I feel like there should be a much faster way of doing this, but have not been able to come up with one. Is there a faster way of doing this as that would really help with processing?
Rather than using a long if/elif chain and explicitly returning Fase if a condition finds a mismatch, you can chain together the comparisons directly in a Boolean expression with and:
return (self.author == other.author and
self.year == other.year and
self.journal == other.journal
...)
You can keep using your getattr calls if you want, but I'd suggest moving the logic to fill in None for missing values into the initialization code, rather than needing to repeat it anywhere you check the attributes. That way you'll always have the attributes you expect, ones that don't have meaningful data will simply have None as their value.
Note that the current behavior of your getattr code doesn't quite match the description you give of your desired behavior ("check if two are the same ignoring a field if it is missing"). The current code will return False if an attribute is present in one citation and not in another, even if the rest of the values match up. If you want the behavior to match your description, use something like this:
not (hasattr(self, "year") and (hasattr(other, "year")) or self.year == other.year
Or (if you do year = None in __init__ if there's no year specified):
None in (self.year, other.year) or self.year == other.year
I have a Spark driver that goes like this:
EDIT - earlier version of the code was different & didn't work
var totalResult = ... // RDD[(key, value)]
var stageResult = totalResult
do {
stageResult = stageResult.flatMap(
// Some code that returns zero or more outputs per input,
// and updates `acc` to number of outputs
...
).reduceByKey((x, y) => x.sum(y))
totalResult = totalResult.union(stageResult)
} while(stageResult.count() > 0)
I know from properties of my data that this will eventually terminate (I'm essentially aggregating up the nodes in a DAG).
I'm not sure of a reasonable caching strategy here - should I cache stageResult each time through the loop? Am I setting up a horrible tower of recursion, since each totalResult depends on all previous incarnations of itself? Or will Spark figure that out for me? Or should I put each RDD result in an array and take one big union at the end?
Suggestions will be welcome here, thanks.
I would rewrite this as follows:
do {
stageResult = stageResult.flatMap(
//Some code that returns zero or more outputs per input
).reduceByKey(_+_).cache
totalResult = totalResult.union(stageResult)
} while(stageResult.count > 0)
I am fairly certain(95%) that the stageResult DAG used in the union will be the correct reference (especially since count should trigger it), but this might need to be double checked.
Then when you call totalResult.ACTION, it will put all of the cached data together.
ANSWER BASED ON UPDATED QUESTION
As long as you have the memory space, then I would indeed cache everything along the way as it stores the data of each stageResult, unioning all of those data points at the end. In fact, each union does not rely on the past as that is not the semantics of RDD.union, it merely puts them together at the end. You could just as easily change your code to use a val due to RDD immutability.
As a final note, maybe the DAG visualization will help understand why there would not be recursive ramifications:
In my current project I have to deal with more complex forms. Fields (i'll name them 'A' and 'B') are automatically filled if a specific field (i'll name that one 'C') received user input. But also if the user inputs data into field A, the fields B and C are automatically filled out.
(This is only a simple example, the current logic is a bit more complicated)
What I have to take care of is that no cycles happen (C -> A -> C -> A -> ...). So I need to now if the current value change was due to user input or another field that had received input and then triggered the value change of the current field. And I also need to now in the second case which field exactly triggered the value change because then I must trigger other specific actions corresponding from who/what triggered that value change.
Is there a general approach in Vaadin to deal with this kind of form
structure? The problem at the moment is that I simply don't now who
or what triggered what ValueChangeEvent.
Are there frameworks to deal with this or am I overlooking an existing Vaadin pattern?
Handling of valueChange events in Vaadin is a bit of pain, since it always fires, no matter if the user has changed something, or the application has used setValue(....) on the component.
The only solution for this is to remember when you do a setValue(....) in your application and then disable the trigger code in the other components.
For example in this case (endless loop):
field1.addValueChangeListener( field2.setValue('Updated by field1');
field2.addValueChangeListener( field1.setValue('Updated by field2');
Change it that way:
boolean inTrigger= false;
field1.addValueChangeListener(
{
if (!inTrigger)
{
inTrigger= true;
field2.setValue('Updated by field1');
inTrigger= false;
}
});
field2.addValueChangeListener(
{
if (!inTrigger)
{
inTrigger= true;
field1.setValue('Updated by field2');
inTrigger= false;
}
});
That way you can prevent update loops and let execute your code exactly once.
I have almost completed my work with Spring Batch, it's working but then I have problems to handle the errors. I'll make a simple example:
I read one flat file, that I (later) map with 3 variables:
ID CODE NAME
AAA3333333Alex
AAA3333333Mark
BBB4444444Paul
I want the reader to read the flat file with a control break (I don't know if it's the right term in english, in italian it's something like "key break"): I read the elements with the same ID and CODE and only when the key changes return them to the reader:
while ((line = (Person) peek()) != null) { //while there are elements to read
if (line.getId().equals(prevElement.getId()) && line.getCode().equals(prevElement.getCode())
//do something
}
This works fine: when the ID or CODE changes I return the elements to the writer.
To make this work I had to set the commit-interval to 1 from the application-context. The thing is that in the worst case, if the elements are different for each line, I commit every single element and it becomes all very very slow.
So I said: let's put an outer control. Instead of returning the elements to the writer each time the key changes, I put them in a list, and then I return the list every 200 key changes (like a...handmade commit-interval):
while (controlBreakCount < 200 &&) {
while (!exit && (line = (Person) peek()) != null) { //while there are elements to read
if (line.getId().equals(prevElement.getId()) && line.getCode().equals(prevElement.getCode())
//do something
else { //if the key changes
//there is a controlBreakCount++; to increase the count
//add the elements to a list
}
}
}
return the list
and this works too (the real code has more controls, but this was to explain in a simple way).
The problem comes here: how to handle the errors with the listener in this case. With the outer while I have put (the one with the controlBreakCount), if even one of the 200 elements has an error all the elements currently in the list go to the listener and so it's very difficult to recognize the element with the error.
I guess my solution is not the best way to handle the "control break", but I can't find really much about this (and I'm not very pro with Spring Batch)...may I have some help?
Thank you very much :)
The 'do something' in your code is suspect... :/ Which operations do you do on the element just read? In reader you should only aggregate items and pass them to processor or writer.
IMHO you have to set a commit-interval greater than 1 to make process faster and try to group your data with same ID+CODE key in your own custom reader and THEN pass to writer in this way:
class PersonList {
String id;
String code;
List<Person> persons = new ArrayList<Person>();
}
when you find a keybreak (ID+CODE differs from previous) you have to create a new PersonList object and until ID+CODE are the same of previous line add to current PersonList.person list.
If your problem is about manage single line error with my solution you you can manage in your reader and, if you want, skip items with same ID+CODE in your reader as well.
Your reader must change its signature to MyPersonItemReader<PersonList> and your writer will write PersonList objects, but you have under your control the object (with ID+CODE).
I hope I had understood correctly your problem.
I'm using Opa for a school project in which there has to be some synchronization of a textfield between several users. The easy way to solve this, is to transmit the complete field whenever there is a change performed by one of the users. The better way is of course to only transmit the changes.
My idea was to use the caret position in the textfield. As a user types, one can get the last typed character based on the caret position (simply the character before the caret). A DOM element has an easy-to-use field for this called selectionStart. I have this small Javascript for this:
document.getElementById('content').selectionStart
which correctly returns 5 if the caret stands at the fifth character in the field. In Opa, I cannot use selectionStart on either a DOM or a dom_element so I thought I'd write a small plugin. The result is this:
##extern-type dom_element
##register jsGetCaretPosition: dom_element -> int
##args(node)
{
return node.selectionStart;
}
This compiles with the opp-builder without any problem and when I put this small line of code in my Opa script:
#pos = %%caret.jsGetCaretPosition%%(Dom.of_selection(Dom.select_id("content")));
that also compiles without problems. However, when I run the script, it always returns "undefined" and I have no idea what I'm doing wrong. I've looked in the API and Dom.of_selection(Dom.select_id("content")) looked like the correct way to get the corresponding dom_element typed data to give to the plugin. The fact that the plugin returns "undefined" seems to suggest that the selected element does not know the member "selectionStart" eventhough my testcode in Javascript suggest otherwise. Anyone can help?
In Opa dom_element are the results of jQuery selection (i.e. an array of dom nodes). So if I well understood your program you should write something like node[0].selectionStart instead of node.selectionStart.
Moreover you should take care of empty selection and selection which doesn't contains textarea node (without selectionStart property). Perhaps the right code is tmp == undefined ? -1 : tmp = node[0].selectionStart == undefined ? -1 : tmp