Query - Query is to expose this internal state to the external world. A query is exposed as an asynchronous callback that is invoked by external entities.
What do you mean by asynchronous callback?
And Doc says, Query has two limitations 1). Should not mutate the state of a Workflow 2). There won't be any blocking operation.
#Override
public String queryGreeting() {
greeting = "val";
return greeting;
}
But I did mutate the variable in the query method and It is changing the value.
Is it just a conviction that we should not write mutable or blocking code inside query method?
I didn't see any difference between query and signal. A query method will be called even after the completion of a workflow where as Signal won't?
Is my understanding correct?
The query should not mutate workflow variables. This is going to break workflow recovery.
The signal can mutate any workflow data as well as invoke blocking operations like activities.
Related
I have a REST API to calculate something upon a request, and if the same request is made again, return the result from the cache, which consist of documents saved in MongoDB. To know if two request is the same, I am hashing some relevant fields in the request. But when same request is made in a quick succession, duplicate documents occur in MongoDB, which later results in "IncorrectResultSizeDataAccessException" when I try to read them.
To solve it I tried to synchronize on hash value in following controller method (tried to cut out non relevant parts):
#PostMapping(
path = "/{myPath}",
consumes = {MediaType.APPLICATION_JSON_UTF8_VALUE},
produces = {MediaType.APPLICATION_JSON_UTF8_VALUE})
#Async("asyncExecutor")
public CompletableFuture<ResponseEntity<?>> retrieveAndCache( ... a,b,c,d various request parameters) {
//perform some validations on request...
//hash relevant equest parameters
int hash = Objects.hash(a, b, c, d);
synchronized (Integer.toString(hash).intern()) {
Optional<Result> resultOpt = cacheService.findByHash(hash);
if (resultOpt.isPresent()) {
return CompletableFuture.completedFuture(ResponseEntity.status(HttpStatus.OK).body(opt.get().getResult()));
} else {
Result result = ...//perform requests to external services and do some calculations...
cacheService.save(result);
return CompletableFuture.completedFuture(ResponseEntity.status(HttpStatus.OK).body(result));
}
}
}
//cacheService methods
#Transactional
public Optional<Result> findByHash(int hash) {
return repository.findByHash(hash); //this is the part that throws the error
}
I am sure that no hash collision is occuring, its just when the same request is performed in a quick succession duplicate records occur. To my understanding, it shouldn't occur as long as I have only 1 running instance of my spring boot application. Do you see any other reason than there are multiple instances running in production?
You should check the settings of your MongoDB client.
If one thread calls the cacheService.save(result) method, and after that method returns, releases the lock, then another thread calls cacheService.findByHash(hash), it's still possible that it will not find the record that you just saved.
It's possible that e.g. the save method returns as soon as the saved object is in the transaction log, but not fully processed yet. Or the save is processed on the primary node, but the findByHash is executed on the secondary node, where it's not replicated yet.
You could use WriteConcern.MAJORITY, but I'm not 100% sure if it covers everything.
Even better is to let MongoDB do the locking by using findAndModify with FindAndModifyOptions.upsert(true), and forget about the lock in your java code.
In Lagom, what do you do when a command handler must perform some asynchronous operations? For example:
override def behavior = Actions().onCommand[MyCommand, Done] {
case (cmd, ctx, state) =>
// some complex code that performs asynchronous operations
// (for example, querying other persistent entities or the read-side
// by making calls that return Future[...] and composing those),
// as summarized in a placeholder below:
val events: Future[Seq[Event]] = ???
events map {
xs => ctx.thenPersistAll(xs: _*) { () => ctx.reply(Done) }
}
}
The problem with the code like that is that the compiler expects the command handler to return Persist, not Future[Persist].
Is this done on purpose, to make sure that the events are persisted in the correct order (that is, the events generated by a prior command must be saved before the events generated by a later command)? But can't that be handled by proper management of the event offsets, so that the journal always orders them correctly, regardless of when they are actually saved?
And what does one do in situations like this, when the command handling is complex enough to require making asynchronous calls from the command handler?
There is a similar question on the mailing list with an answer from James.
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/lagom-framework/Z6lynjNTqgE
In short, your entity in a CQRS application is a consistency boundary and should only depend on data that it's immediately available inside it, not outside (no call to external services).
What you are probably looking for a what it's called Command Enrichment. You receive an request, collect some data from external services and build a command containing everything you need to send to your Entity.
You certainly should not query the read-side to make business decisions inside your write-side entity. You also should not make business decisions on data coming from other entities.
Your entity should be able to make all the decision because it is the consistency boundary of your model.
What I've been doing in these cases is to pass the PersistentEntityRef to the asynchronous operation, so that it can issue commands to the entity and its those command-handlers (not the one that spawned the async computation) that then persist the events.
Just bear in mind that none of this is atomic, so you have to think about what happens if the asynchronous operation fails midway through issuing the commands or if some commands succeed and some fail, etc. Presumably you'll want some retry mechanism for systemic failures. If you build your command-handlers to be idempotent, it will help you deal with the duplicates.
I have a PUT request that is too long to run. I'd like to make it async, using continuations (await/promise feature).
I create a job (LongJobThatUpdatesThePassedEntity) that modifies my entity
public static void myLongPut(#required Long id, String someData) {
MyJpaModel myJpaModel = MyJpaModel.findById(id);
//straightforward modifications
updateMyJpaModel(someData);
myJpaModel.save();
//long processing modifications to entity, involving WS calls
Promise<String> delayedResult = new LongJobThatUpdatesThePassedEntity(id).now();
await(delayedResult);
render(myJpaModel.refresh());
}
How are the DB transactions managed?
is there a commit before the job's call?
the job has it's own DB transaction?
if there is an issue in the LongJobThatUpdatesThePassedEntity that rollsback, the modifications done in updateMyJpaModel are persisted?
can I do render(myJpaModel.refresh()) at the end?
will it contain the straighforward modifications and the long ones?
thank's
I can answer most of your question for Play 1.4.3, which is the version I'm currently using. I don't expect that much has changed since Play 1.2.
How are the DB transactions managed?
Play! handles the transactions for jobs and controller actions using an "invocation", which is a Play-specific concept. In short, for any invocation, each plugin gets a chance to do some setup and cleanup before and after the invoked method runs. For database access, the JPAPlugin.withinFilter method starts and closes the transaction using the JPA class's helper methods.
is there a commit before the job's call?
When you call await(Future<T>), it has the effect of closing the current transaction and starting a new one. The specific mechanism is that it throws a "Suspend" exception, which bubbles up to PlayHandler$NettyInvocation and causes the afterInvocation callbacks to be called. This causes JPAPlugin.afterInvocation to call
JPA.closeTx() which either commits or rollsback the transaction, as appropriate.
When the Job exits and the await() continuation is resumed. This is also handled as an invocation, so the transaction is started in the same way as before, using JPAPlugin.withinFilter(). However, unlike before, the controller action is not the target of the invocation, but instead ActionInvoker.invoke() calls invokeWithContinuation, which restores the saved continuation state and resumes execution by returning from await().
JPA.withTransaction looks like it has some special logic to retain the same entity manager across the continuation suspend/resume. I think without this, you wouldn't be able to call refresh().
In your code, I think there's a race condition between when await() closes the transaction and the Job starts its transaction. That is, it's possible that the Job's transaction begins before the controller commits the "before await" transaction. To avoid this, you can explicitly call JPA.closeTx() before calling Job.now().
Based on code inspection, it looks like the way Play! is implemented, it so happens that the Job will exit and the Job's transaction will be closed before the "after await()" transaction is opened. I don't know if any
documentation that says this is an intended part of the await() contract, so if this is essential for your appliaction, you can avoid using undocumented behavior by committing the transaction just before your Job.doJobWithResult() method returns.
the job has it's own DB transaction?
Yes, unless its annotated to not have a transaction.
if there is an issue in the LongJobThatUpdatesThePassedEntity that rollsback, the modifications done in updateMyJpaModel are persisted?
Based on the explanation above, each of the three transactions are independent. If one is rolled back, I don't see how it would affect the others.
I'm currently working on something that involves iterating through a Sales Order and Sales Order Products via a trigger on the Sales Order object. I've created an Apex class that is called from the Sales Order after update trigger. The trigger passes a string (Sales Order Id) to the static method of the class. This future call method queries for Sales Order Products that belong to the Sales Order id, and makes a web service call for each item in the collection. This all works great, however I would like for this process to be more robust and handle errors more intelligently. What I would like to be able to do is abort the whole process when the method encounters something it doesn't like, let's say it identifies a product in the order it doesn't like as an example. The only process I've found that can handle aborting is via the Queueable Interface, and calling the class via System.enqueueJob(). This however doesn't help me as I cannot for the life of me figure out a way to pass any parameters to this class when System.enqueueJob() is invoked, since the class methods are static and the interface forces the process to run from the execute() method, which only takes a Context parameter. Am I going down the wrong road with this? The only other possibility I was thinking of was to just create methods for all of the subprocesses in my class and return from those if they encounter any errors and set a bool flag that can be used to skip processes afterward in the class. Sorry if this doesn't make sense, if so let me know and I'll try to provide more information.
You can pass parameters to a Queueable job in the constructor. i.e.:
System.enqueueJob(new myQueueableClass(salesOrderId));
You need to add a constructor in your Queueable class that will accept the Sales Order Id and store it in a private variable also declared inside the Queueable class, which then can be accessed by the execute() method.
function _allUsers(callback){
var db = connect.get();
db.collection("users").find({}).toArray(function(err,data){
if(err){
callback(err);
}else{
callback(null,data);
}
});
}
I am trying to understand this code, I have been looking around the web but I find the explanations kinda defficult to understand ( I am new at Mean stack), so my questions are:
What does the Collection method do? I am not sure but the string "users" is it just the name of our collection with all users?
Why do we have to use a callback in this situation? (I find callbacks very confusing).
And why do we have to give toArray function, an annonymous function?
Instead of toArray could I use pretty method() without any annonymous function as a parameter?
MEAN Stack is a software bundle of software programs supporting applications written in all javascript. This means you can use javascript from your database, to your back-end and front-end.
MEAN actually stands for the first characters of each software program included in the stack. MongoDB, Expressjs, AngularJS and NodeJS.
1
MongoDB is a NoSQL database which uses BSON (similar to JSON) to store so called documents. Look at a document as if it is a single entity or row in a traditional database. These entities (or rows) are stored in collections (a collection of documents) which can be compared to tables.
So the answer to your 1st question is opens up the users collection, which grants access to all the user documents.
2
NodeJS is asynchronous by design. This allows NodeJS to perform a lot of operations while running on a single thread*. Because NodeJS is single-threaded we need a way to write our code non-blocking meaning we can start an operation, proceed with executing other code and come back whenever that operation is finished.
In your case we request access to the users collection, this takes some time. In order to allow other parts of our application to continue processing we use a callback. When we have access to our collection, our callback is executed and we can perform whatever operation we wanted to do when we first requested access.
*NodeJS actually runs on multiple threads but a developer never has to worry about multithreading, NodeJS does that for us.'
3
This is exactly what the previous point is about.
The .toArray() method returns an array that contains all the documents from a cursor. The method iterates completely the cursor, loading all the documents into RAM and exhausting the cursor. Source
.toArray() is a computionally intensive operation. Since we do not want to wait untill .toArray() is finished but proceed processing the rest of our code, we give it a callback so that we can come back to our collection processing whenever it's ready.
4
From what I can read from the docs I guess you could indeed write blocking code and do it this way:
var users = db.collection("users").find({}).toArray();
This however will block your code entirely. There is never a good reason to do this.
Disclaimer: I left out or oversimplified details in this explanation for ease of understanding.
db.collection('users') this will return the users collection instance
we are using callback for asynchronous
the annonymous function in toArray is its callback
this is dependent on the library in use..
without any annonymous function as a parameter
expressjs is an asynchronous programming, we need callback || Promises
You can think of the collection as of table in MySQL. A collection consists of documents (rows/items/records in MySQL). Your example calls the Users collection and finds all documents (records) in it.
About the callbacks - NodeJS/Express are commonly callbacks-oriented. This is the pattern they use and most of the code is using it, because it is asynchronous. If you need to be sure that some snippet is executed right after some other snippet, you have to use callback (or promise).
Calling toArray() depends on what your callback expects. You can skip calling this method if the callback expects the Query object returned by the find() method. All that depends on your callback.
You can use non-anonymous function, too, but you have to have in mind the asynchronous logic and continue using callbacks/promises. You can read more about callbacks and promises in this Quora's article.
Here you can find more about the find() method.