I find getVersion API to be a bit hard to grasp. After some manual experiments with workflow changes, I found out that it's perfectly fine to have such a piece of code:
val version = Workflow.getVersion("change#1", 1, 1);
val anotherVersion = Workflow.getVersion("change#2", 2, 2);
Does it mean that the integer version is assigned to a changeId and not workflow instance? Does a single workflow instance/execution keep a set of integer-based versions?
What is the purpose of minSupported and maxSupported parameters? Why simply not to use an API like below?
val version = Workflow.getVersion("change#1")
if (version) {
// code after "change#1" changes
} else {
// code before "#change#1" changes
}
You are correct, the version is assigned to a changeId not a workflow instance. This allow versioning each piece of the workflow code independently. It allows fixing bugs while workflow is already running and didn't reach that part of the code.
The main reason is validation. The getVersion call records in the workflow history maxVersion when the code was executed for the first time. So on replay the correct version is used to guarantee correct replay even if the maxVersion has changed. When a branch is removed the minVersion is incremented. Imagine that such code is deployed by mistake when there is a workflow that needs the removed branch. The getVersion is going to detect that minVersion is larger than the one recorded in the history and is going to fail the decision task essentially blocking the workflow execution instead of breaking it. The same happens if the recorded version is higher than the maxVersion argument.
Update: Answer to the comment
In other words, I'm trying to come up with a situation where using
many different changeIds and not exceeding maxVersion=1 is not enough
They are enough if you don't perform removal of branches. But if you do then having validation of the minimal version is very convenient. For example look at the following code:
val version = Workflow.getVersion("change", 0, 2);
if (version == DEFAULT_VERSION) {
// before change
} else if (version == 1) {
// first change
} else {
// second hange
}
Let's remove the default version:
val version = Workflow.getVersion("change", 1, 2);
if (version == 1) {
// first change
} else {
// second hange
}
Now look at the version without min and max:
var version1 = Workflow.getVersion("change1");
var version2 = Workflow.getVersion("change2");
if (version1 == DEFAULT_VERSION) {
// before change
} else if (version2 == DEFAULT_VERSION) {
// first change
} else {
// second hange
}
Let's remove the default branch:
var version2 = Workflow.getVersion("change2");
if (version2 == DEFAULT_VERSION) {
// first change
} else {
// second hange
}
Note that a workflow that used the last sample code is going to break in unpredictable way if it is routed by mistake to a worker that doesn't know about version2, but only about the original default version. The first example with min max version is going to detect the issue gracefully.
Related
I am trying to learn reactive programming, so forgive me if I ask a silly question. I'm also open to advice on changing my design.
I am working in scala-swing to display the results of a simulator. With one setting, a chart is displayed as a histogram; with the other setting the chart is displayed as the cumulative sum. (I'm probably using the wrong word; in the first setting you might have bin1=2, bin2=5, bin3=3; in the second setting the first height is 2, the second is 2 + 5, the third is 2 + 5 + 3, etc.). The simulator can be slow, so I originally used a Future to compute it, and the set the data into the chart. I decided to try a reactive approach, so my requirements are: 1. I don't want to recreate the data when I change the display mode, and 2. I want to set the Observable once for the chart and have the chart listen to the same Observable permanently.
I got this to work when I started the chain with a PublishSubject and the Future set the data into the start of the chain. When the display mode changed, I created a new PublishSubject().map(newRenderingLogic).subscribe(theChartsObservable). I am now trying to do what looks like the "right way," but it's not working correctly. I've tried to simplify what I have done:
val textObservable: Subject[String] = PublishSubject()
textObservable.subscribe(text => {
println(s"Text: ${text}")
})
var textSubscription: Option[Subscription] = None
val start = Observable.from(Future {
"Base text"
}).cache
var i = 0
val button = new Button() {
text = "Click"
reactions += {
case event => {
i += 1
if (textSubscription.isDefined) {
textSubscription.get.unsubscribe()
}
textSubscription = Some(start.map(((j: Int) => { (base: String) => s"${base} ${j}" })(i)).subscribe(textObservable))
}
}
}
On start, an Observable is created and logic to print some text is added to it. Then, an Observable with the generated data is created and a cache is added so that the result is replayed if the next subscription comes in after its results are generated. Then, a button is created. Then on button clicks a middle observable is chained with unique logic (it's a function that creates a function to append the value of i into the string, run with the current value of i; I tried to make something that couldn't just be reused) that is supposed to change with each click. Then the first Observable is subscribed to it so that the results of the whole chain end up being printed.
In theory, the cache operation takes care of not regenerating the data, and this works once, but onComplete is called on textObservable and then it can't be used again. It works if I subscribe it like this:
textSubscription = Some(start.map(((j: Int) => { (base: String) => s"${base} ${j}" })(i)).subscribe(text => textObservable.onNext(text)))
because the call to onComplete is intercepted, but this looks wrong and I wanted to know if there was a more typical way to do this, or architect it. It makes me think that I don't understand how this is supposed to be done if there isn't an out-of-the-box operation to do this.
Thank you.
I'm not 100% sure if I got the essence of your question right, but: if you have an Observable that may complete and you want to turn it into an Observable that never completes, you can just concatenate it with Observable.never.
For example:
// will complete after emitting those three elements:
val completes = Observable.from(List(1, 2, 3))
// will emit those three elements, but will never complete:
val wontComplete = completes ++ Observable.never
I'm traying to create a trigger that count each time a case is re-opened.
What I need is that changing the Status "Cerrado" to "Asignado" add +1
I've this trigger but it doesn't working at all.
Reperturas__c a type number and in the formula I've a 0.
"Cerrado" close the case.
Any solution guys?. Thanks!
trigger caseReOpen on Case (before update) {
for(Case c:trigger.new){
if(trigger.Oldmap.get(c.Id).Status=='Cerrado'){
c.Reaperturas__c ++;
}
}
}
Should work, weird. How's this?
trigger caseReOpen on Case (before update) {
for(Case c: trigger.new){
Case old = trigger.oldMap.get(c.Id);
if(c.Status == 'Asignado' && old.Status == 'Cerrado'){
++c.Reaperturas__c;
}
}
}
Compiled and saved ok? Can you put field history tracking on it, maybe you have something else that resets the counter? Are the actual picklist values like that or are these just labels? You need to use API values in Apex so if they're in English and you just have labels translated - your code won't work.
What if you base the check on !c.IsClosed && old.IsClosed ? It's a computed checkbox, can't be edited directly but it's bit more portable... If in future you add more statuses counting as closing (closed completed, closed cancelled etc?). https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_objects_case.htm
I am updating sort & filter models via api:
this.gridApi.setFilterModel(filterModels);
this.gridApi.setSortModel(sortModels);
The problem with this is I have a server request bound to the change even of both sort & filter so when user changes then the data is updated. This means when I change model on code like restoring a state or resetting the filters it causes multiple requests.
Is there a way to update the filter/sort model without triggering the event?
I see there is a ColumnEventType parameter but couldn't see how it works. Can I specify some variable that I can look for inside my event handlers to get them to ignore calls that are not generated from user?
I am trying to manage URL state so when url query params change my code sets the models in the grids but this ends up causing the page to reload multiple times because the onFilter and onSort events get called when the model is set and there is no way I can conceive to prevent this.
At the time, you are going to have to manage this yourself, ie, just before you call the setModel, somehow flag this in a shared part of your app (maybe a global variable)
Then when you react to these events, check the estate of this, to guess where it came from.
Note that at the moment, we have added source to the column events, but they are not yet for the model events, we are planning to add them though, but we have no ETA
Hope this helps
I had to solve similar issue. I found solution which working for my kind of situation. Maybe this help someone.
for (let j = 0; j < orders.length; j++) {
const sortModelEntry = orders[j];
if (typeof sortModelEntry.property === 'string') {
const column: Column = this.gridColumnApi.getColumn(sortModelEntry.property);
if (column && ! column.getColDef().suppressSorting) {
column.setSort(sortModelEntry.direction.toLowerCase());
column.setSortedAt(j);
}
}
this.gridApi.refreshHeader();
Where orders is array of key-value object where key is name of column and value is sorting directive (asc/desc).
Set filter without refresh was complicated
for (let j = 0; j < filters.length; j++) {
const filterModelEntry = filters[j];
if (typeof filterModelEntry.property === 'string') {
const column: Column = this.gridColumnApi.getColumn(filterModelEntry.property);
if (column && ! column.getColDef().suppressFilter) {
const filter: any = this.gridApi.getFilterApi(filterModelEntry.property);
filter['filter'] = filterModelEntry.command;
filter['defaultFilter'] = filterModelEntry.command;
filter['eTypeSelector'].value = filterModelEntry.command;
filter['filterValue'] = filterModelEntry.value;
filter['filterText'] = filterModelEntry.value;
filter['eFilterTextField'].value = filterModelEntry.value;
column.setFilterActive(true);
}
}
}
Attributes in filter:
property - name of column
command - filter action (contains, equals, ...)
value - value used in filter
For anyone else looking for a solution to this issue in Nov 2020, tapping into onFilterModified() might help. This gets called before onFilterChanged() so setting a value here (eg. hasUserManuallyChangedTheFilters = false, etc.) and checking the same in the filter changed event is a possible workaround. Although, I haven't found anything similar for onSortChanged() event, one that gets called before the sorting is applied to the grid.
I am not sure any clean way to achieve this but I noticed that FilterChangedEvent has "afterFloatingFilter = false" only if filterModel was updated from ui.
my workaround is as below
onFilterChanged = event:FilterChangedEvent) => {
if(event.afterFloatingFilter === undefined) return;
console.log("SaveFilterModel")
}
I have a recursive function that needs to compare the results of the current call to the previous call to figure out whether it has reached a convergence. My function does not contain any action - it only contains map, flatMap, and reduceByKey. Since Spark does not evaluate transformations (until an action is called), my next iteration does not get the proper values to compare for convergence.
Here is a skeleton of the function -
def func1(sc: SparkContext, nodes:RDD[List[Long]], didConverge: Boolean, changeCount: Int) RDD[(Long] = {
if (didConverge)
nodes
else {
val currChangeCount = sc.accumulator(0, "xyz")
val newNodes = performSomeOps(nodes, currChangeCount) // does a few map/flatMap/reduceByKey operations
if (currChangeCount.value == changeCount) {
func1(sc, newNodes, true, currChangeCount.value)
} else {
func1(sc, newNode, false, currChangeCount.value)
}
}
}
performSomeOps only contains map, flatMap, and reduceByKey transformations. Since it does not have any action, the code in performSomeOps does not execute. So my currChangeCount does not get the actual count. What that implies, the condition to check for the convergence (currChangeCount.value == changeCount) is going to be invalid. One way to overcome is to force an action within each iteration by calling a count but that is an unnecessary overhead.
I am wondering what I can do to force an action w/o much overhead or is there another way to address this problem?
I believe there is a very important thing you're missing here:
For accumulator updates performed inside actions only, Spark guarantees that each task’s update to the accumulator will only be applied once, i.e. restarted tasks will not update the value. In transformations, users should be aware of that each task’s update may be applied more than once if tasks or job stages are re-executed.
Because of that accumulators cannot be reliably used for managing control flow and are better suited for job monitoring.
Moreover executing an action is not an unnecessary overhead. If you want to know what is the result of the computation you have to perform it. Unless of course the result is trivial. The cheapest action possible is:
rdd.foreach { case _ => }
but it won't address the problem you have here.
In general iterative computations in Spark can be structured as follows:
def func1(chcekpoinInterval: Int)(sc: SparkContext, nodes:RDD[List[Long]],
didConverge: Boolean, changeCount: Int, iteration: Int) RDD[(Long] = {
if (didConverge) nodes
else {
// Compute and cache new nodes
val newNodes = performSomeOps(nodes, currChangeCount).cache
// Periodically checkpoint to avoid stack overflow
if (iteration % checkpointInterval == 0) newNodes.checkpoint
/* Call a function which computes values
that determines control flow. This execute an action on newNodes.
*/
val changeCount = computeChangeCount(newNodes)
// Unpersist old nodes
nodes.unpersist
func1(checkpointInterval)(
sc, newNodes, currChangeCount.value == changeCount,
currChangeCount.value, iteration + 1
)
}
}
I see that these map/flatMap/reduceByKey transformations are updating an accumulator. Therefore the only way to perform all updates is to execute all these functions and count is the easiest way to achieve that and gives the lowest overhead compared to other ways (cache + count, first or collect).
Previous answers put me on the right track to solve a similar convergence detection problem.
foreach is presented in the docs as:
foreach(func) : Run a function func on each element of the dataset. This is usually done for side effects such as updating an Accumulator or interacting with external storage systems.
It seems like instead of using rdd.foreach() as a cheap action to trigger accumulator increments placed in various transformations, it should be used to do the incrementing itself.
I'm unable to produce a scala example, but here's a basic java version, if it can still help:
// Convergence is reached when two iterations
// return the same number of results
long previousCount = -1;
long currentCount = 0;
while (previousCount != currentCount){
rdd = doSomethingThatUpdatesRdd(rdd);
// Count entries in new rdd with foreach + accumulator
rdd.foreach(tuple -> accumulator.add(1));
// Update helper values
previousCount = currentCount;
currentCount = accumulator.sum();
accumulator.reset();
}
// Convergence is reached
I've got a piece of code that opens a data reader and for each record (which contains a url) downloads & processes that page.
What's the simplest way to make it multi-threaded so that, let's say, there are 10 slots which can be used to download and process pages in simultaneousy, and as slots become available next rows are being read etc.
I can't use WebClient.DownloadDataAsync
Here's what i have tried to do, but it hasn't worked (i.e. the "worker" is never ran):
using (IDataReader dr = q.ExecuteReader())
{
ThreadPool.SetMaxThreads(10, 10);
int workerThreads = 0;
int completionPortThreads = 0;
while (dr.Read())
{
do
{
ThreadPool.GetAvailableThreads(out workerThreads, out completionPortThreads);
if (workerThreads == 0)
{
Thread.Sleep(100);
}
} while (workerThreads == 0);
Database.Log l = new Database.Log();
l.Load(dr);
ThreadPool.QueueUserWorkItem(delegate(object threadContext)
{
Database.Log log = threadContext as Database.Log;
Scraper scraper = new Scraper();
dc.Product p = scraper.GetProduct(log, log.Url, true);
ManualResetEvent done = new ManualResetEvent(false);
done.Set();
}, l);
}
}
You do not normally need to play with the Max threads (I believe it defaults to something like 25 per proc for worker, 1000 for IO). You might consider setting the Min threads to ensure you have a nice number always available.
You don't need to call GetAvailableThreads either. You can just start calling QueueUserWorkItem and let it do all the work. Can you repro your problem by simply calling QueueUserWorkItem?
You could also look into the Parallel Task Library, which has helper methods to make this kind of stuff more manageable and easier.