I need assistance.
An issue with one of my endpoints timing out is causing me distress.
I did some performance tweaking with SQL and other REST services I am using but it only helps a little bit.
A nice solution for this problem, I thought, would be to use some of the Async capabilities of Spring Boot and Java 8 and perform some sort of "Fire and forget" action.
I tried something like that, but it is no good, the "Time to rock!" message gets printed out all right but it seems that getLyrics() method is not invoked at all!
//MyController.java
#GET
#Path("na/na/na")
#Produces({"application/json"})
public Response getLyrics() {
final String lyrics = delegate.getLyrics();
return Response.ok().entity(lyrics.isEmpty()).build();
}
//MyDelegate.java
#Async("threadPoolTaskExecutor")
public Future<Boolean> getLyrics() {
LOG.info("Time to rock!");
final boolean result = lyricsService.getLyrics();
return new AsyncResult<Boolean>(result);
}
//MyAsyncConfig.java
#Configuration
#EnableAsync
public class MyAsyncConfig {
#Bean(name = "threadPoolTaskExecutor")
public Executor threadPoolTaskExecutor() {
return new ThreadPoolTaskExecutor();
}
}
So, lyricsService.getLyrics() (that is not being called for some reason) does all the work, calls other services, fetches stuff from the SQL database, and performs calls against some other REST endpoints. All of this takes time and sometimes* causes a timeout. I would like it to process in peace, and if possible, return some sort of response when possible.
I tried several variations of this solution as it seems to be close to what I need, but can't seem to get why it is not working for me.
*often
I think, spring futures have to wait until operation is done. But java completable future is more powerful maybe you can try that.
Future<String> a = ...;
while(!a.isDone()){
}
here is a sample.
https://spring.io/guides/gs/async-method/
You may use Spring's DefferedResult, along with Java8's Computable future to make your controller non-blocking and thereby delegate the long running taks inside the Comupatable Future's whenAsync method.Here is a working example -
https://github.com/kazi-imran/TransactionStatistics
Related
Env: Mac OS 12.1, JDK 17, Vert.x 4.2.4
Question: how to capture command line input from a verticle? Tried so far following in the public void start(Promise<Void> startPromise) throws Exception method:
getVertx().createSharedWorkerExecutor("sys-in").executeBlocking(promise -> {
try (final BufferedReader br = new BufferedReader(new InputStreamReader(System.in))) {
String line;
int count = 0;
do {
System.out.print("message to MC: ");
line = br.readLine();
count++;
//doSth(line); // e.g. send line over multicast
} while (count < 3);
} catch (Throwable t) {
// log.info("<start> ", t);
} finally {
// bye(); // send a final message and close vertx
promise.complete();
}
});
This will start, get 3 nulls from br, and exit. Also tried a separated ExecutorService, in vain. Couldn't find any help in Vert.x doc either. Any hints are appreciated:
aware of the warnings of Vert.x when doing blocking stuff
Vert.x might not meant to be used this way, but would be cool if it (reading from command line) can be done with the same toolkit
I understand what you are trying to accomplish, but the problem is that that goes against fundamentals of verticles concept. Waiting for user input is potentially infinitely blocking operation i.e. there is no guarantee user will ever input the values. In that case, you are left with the verticle that is hung forever, spending resources and stuck in one spot. Multiply this if you are using worker verticles and you might have serious problems with the app. This issue is also emphasized here: https://vertx.io/docs/vertx-core/java/#blocking_code (under Warning).
In the link provided you can also find a suggested solution with a separate thread solution. Non-vertx thread won't mind being blocked and when the user input is provided can inform the vertx part of the application via the event bus that the user input dependent code can now be executed.
This might not be the solution you had in mind since it's not pure vertx, but have in mind that vert.x is just another tool, and that tool is not a good fit for what you are trying to accomplish here. However, it can be paired well with plain Java and it won't mind.
I'm new to Blazor and trying to make a page with several separate components to handle a massive form. Each individual component covers a part of the form.
The problem I'm facing is that each of my components needs access to data from the back-end, and not every component uses the same data. When the page loads, each components makes an attempt to fetch data from the server, which causes a problem with Entity Framework.
A second operation started on this context before a previous operation
completed. This is usually caused by different threads using the same
instance of DbContext.
This is obviously caused by the fact that my components are initialized at the same time, and all make their attempt to load the data simultaneously. I was under the impression that the way DI is set up in Blazor, this wouldn't be a problem, but it is.
Here are the components in my template:
<CascadingValue Value="this">
<!-- BASE DATA -->
<CharacterBaseDataView />
<!-- SPECIAL RULES -->
<CharacterSpecialRulesView />
</CascadingValue>
Here is how my components are initialized:
protected async override Task OnInitializedAsync()
{
CharacterDetailsContext = new EditContext(PlayerCharacter);
await LoadCharacterAsync();
}
private async Task LoadCharacterAsync()
{
PlayerCharacter = await PlayerCharacterService.GetPlayerCharacterAsync(ViewBase.CharacterId.Value);
CharacterDetailsContext = new EditContext(PlayerCharacter);
}
When two components with the above code are in the same view, the mentioned error occurs. I thread using the synchronous version "OnInitialized()" and simply discarding the task, but that didn't fix the error.
Is there some other way to call the data so that this issue doesn't occur? Or am I going about this the wrong way?
You've hit a common problem in using async operations in EF - two or more operations trying to use the same context at once.
Take a look at the MS Docs article about EF DBContexts - there's a section further down specific to Blazor. It explains the use of a DbContextFactory and CreateDbContext to create contexts for units-of-work i.e. one context per operation so two async operations each have a separate context.
Initially to solve the threading issues, I used DbContextFactory to create contexts for each operation - however this resulted in database in-consistency issues across components, and I realised I need change tracking across components.
Therefore instead, I keep my DbContext as scoped, and I don't create a new context before each operation.
I then adapted my OnInitializedAsync() methods to check if the calls to the database have completed, before making these calls through my injected services. This works really well for my app:
#code {
static Semaphore semaphore;
//code ommitted for brevity
protected override async Task OnInitializedAsync()
{
try
{
//First open global semaphore
semaphore = Semaphore.OpenExisting("GlobalSemaphore");
while (!semaphore.WaitOne(TimeSpan.FromTicks(1)))
{
await Task.Delay(TimeSpan.FromSeconds(1));
}
//If while loop is exited or skipped, previous service calls are completed.
ApplicationUsers = await ApplicationUserService.Get();
}
finally
{
try
{
semaphore.Release();
}
catch (Exception ex)
{
Console.WriteLine("ex.Message");
}
}
}
Am new to Microservices and CQRS event handling. I am trying to understand with one simple task. In this task I have three REST external services to handle one transaction/request(Service). The three services are
step1: customer create.
step2: create business for customer
step3: Create Address for business.
I want to implement SAGA for these events with InMemorySagaRepository and saga manager.
Where exactly I have to initiate the SagaManager with repository, Is it in RestController or in CommandHandler ?
Can you please help me in understanding sagas flow ?
Thanks in Advance.
Half a year later, and I'm making an edit as I've now taken a course held by Greg Young called Greg Young's CQRS, Domain Events, Event Sourcing and how to apply DDD
I really recommend it to anyone thinking about CQRS. Help A LOT to understand what things actually are
Original anwser
In our product we use Sagas as something that reacts to events.
This means that our sagas are really just Subscribers to a specific Event. The saga then holds some logic as to whether it should do something or not.
If the saga finds that an action should be taken, it creates a Command which it puts on the CommandBus.
This means that Sagas are just 'reactors' and use the same path in as a user would (skipping the APIs etc).
But what a Saga really is, and what it should do, differs from the one talking about them to the other. (Disclaimer: This is how I read these posts, they might actually all say the same thing, but in a way to fluffy way for me [+my team] to see that)
http://blog.jonathanoliver.com/cqrs-sagas-with-event-sourcing-part-i-of-ii/ for example, raises the point that Sagas should not contain 'business logic' (anything that contains 'if' is business logic according to the post).
https://msdn.microsoft.com/en-us/library/jj591569.aspx Talks about Sagas as 'Process managers' which coordinate things between different Aggregates (remember that Aggregate1 can't talk to Aggregat2 directly, so a 'Process manager' is required to orchestrate the communication). To put it simply: Event -> Saga -> Command -> Event -> Saga... To reach the final destination.
https://lostechies.com/jimmybogard/2013/03/21/saga-implementation-patterns-variations/ Talks about two different patterns of what a Saga is. One is 'Publish-gatherer' which basically coordinates what should happen based on a Command. The other is 'Reporter', which just reports the status of things to where they need to go. It doesn't coordinate things, it just reports whatever it needs to report.
http://kellabyte.com/2012/05/30/clarifying-the-saga-pattern/ Has a write-up of what the Saga-pattern 'is'. The claim is that Sagas should/could compensate for different workflows that break.
http://cqrs.nu/Faq/sagas Has a very short description on what Sagas are and basically says 'They are state machines that lets aggregates react to other aggregates'.
So, given that, what is it you actually want the Saga to do? Should it coordinate everything? Or should it just react and not care what the Aggregates do?
My edited part
So, after taking the course on CQRS and talking with Greg about this, I've come to the conclusion that there is quite a lot of confusion out there on the web.
Lets start with just the concept 'Saga'. A Saga has actually nothing to do with CQRS. It's not a concept of it. 'Saga' a form of a two-phase-commit, only it's optimised for success rather than fail ( https://en.wikipedia.org/wiki/Compensating_transaction )
Now, what most people mean when they talk CQRS and say "Saga" is "Process Manager". And process managers are quite complicated it seems (Greg has a whole other course for just Process Managers).
Basically what they do is the manage the whole process of something (as the name suggests). The link to Microsoft is pretty much what it's all about.
To answer the question:
Where exactly I have to initiate the SagaManager with repository, Is it in RestController or in CommandHandler ?
Outside of them both. A Process Manager is it's own thing. It spans aggregates and repositories. Conceptually it might be better to look at it as a user doing all the things you want the PM do to, just that you program the users interaction and tell it what to listen for.
Disclaimer: I do not work for Greg, or anyone that stands to gain on my promotion for taking his courses. It's just that I learned a lot from it, so I recommend it just like I would recommend reading Eric Evans book on DDD.
In my application i've build Saga process manager using this MSDN documentation, my Saga is implemented in Application Service layer, it listens Events of Sales, Warehouse & Billing bounded contexts and on event occurrence sends Commands via Service Bus.
Simple example, hope it helps you to analyze how to build your saga (I am registering saga as handler in Composition Root) ;):
SAGA:
public class SalesSaga : Saga<SalesSagaData>,
ISagaStartedBy<OrderPlaced>,
IMessageHandler<StockReserved>,
IMessageHandler<PaymentAccepted>
{
private readonly ISagaPersister storage;
private readonly IBus bus;
public SalesSaga(ISagaPersister storage, IBus bus)
{
this.storage = storage;
this.bus = bus;
}
public void Handle(OrderPlaced message)
{
// Send ReserveStock command
// Save SalesSagaData
}
public void Handle(StockReserved message)
{
// Restore & Update SalesSagaData
// Send BillCustomer command
// Save SalesSagaData
}
public void Handle(PaymentAccepted message)
{
// Restore & Update SalesSagaData
// Send AcceptOrder command
// Complete Saga (Dispose SalesSagaData)
}
}
InMemorySagaPersister: (as SalesSagaDataID i am using OrderID its unique across whole process)
public sealed class InMemorySagaPersister : ISagaPersister
{
private static readonly Lazy<InMemorySagaPersister> instance = new Lazy<InMemorySagaPersister>(() => new InMemorySagaPersister());
private InMemorySagaPersister()
{
}
public static InMemorySagaPersister Instance
{
get
{
return instance.Value;
}
}
ConcurrentDictionary<int, ISagaData> data = new ConcurrentDictionary<int, ISagaData>();
public T GetByID<T>(int id) where T : ISagaData
{
T value;
var tData = new ConcurrentDictionary<int, T>(data.Where(c => c.Value.GetType() == typeof(T))
.Select(c => new KeyValuePair<int, T>(c.Key, (T)c.Value))
.ToArray());
tData.TryGetValue(id, out value);
return value;
}
public bool Save(ISagaData sagaData)
{
bool result;
ISagaData existingValue;
data.TryGetValue(sagaData.Id, out existingValue);
if (existingValue == null)
result = data.TryAdd(sagaData.Id, sagaData);
else
result = data.TryUpdate(sagaData.Id, sagaData, existingValue);
return result;
}
public bool Complete(ISagaData sagaData)
{
ISagaData existingValue;
return data.TryRemove(sagaData.Id, out existingValue);
}
}
One approach might be to have some sort of starting command that starts the Saga. In this scenario it would be configured in your composition root to listen to a certain command type. Once a command has been received in your message dispatcher (or whatever middleware messaging stuff you have) it would look for any Sagas that have been registered to be started by the command. You would then create the Saga and pass it the command. It could then react to other commands and events as they happen.
In your scenario I would suggest your Saga is a type of command handler so the initiation of it would be upon receiving a command
We have an enterprise DB that is replicated through many sites throughout the world. We would like our app to attempt to connect to one of the local sites, and if that site is down we want it to fall back to the enterprise DB. We'd like this behavior on each of our DB operations.
We are using Entity Framework, C#, and SQL Server.
At first I hoped I could just specify a "Failover Partner" in the connection string, but that only works in a mirrored DB environment, which this is not. I also looked into writing a custom IDbExecutionStrategy. But these strategies only allow you to specify the pattern for retrying a failed DB operation. It does not allow you to change the operation in any way like directing it to a new connection.
So, do you know of any good pattern for dealing with this type of operation, other than duplicating retry logic around each of our many DB operations?
Update on 2014-05-14:
I'll elaborate in response to some of the suggestions already made.
I have many places where the code looks like this:
try
{
using(var db = new MyDBContext(ConnectionString))
{
// Database operations here.
// var myList = db.MyTable.Select(...), etc.
}
}
catch(Exception ex)
{
// Log exception here, perhaps rethrow.
}
It was suggested that I have a routine that first checks each of the connections strings and returns the first one that successfully connects. This is reasonable as far as it goes. But some of the errors I'm seeing are timeouts on the operations, where the connection works but the DB has issues that keep it from completing the operation.
What I'm looking for is a pattern I can use to encapsulate the unit of work and say, "Try this on the first database. If it fails for any reason, rollback and try it on the second DB. If that fails, try it on the third, etc. until the operation succeeds or you have no more DBs." I'm pretty sure I can roll my own (and I'll post the result if I do), but I was hoping there might be a known way to approach this.
How about using some Dependency Injection system like autofac and registering there a factory for new context objects - it will execute logic that will try to connect first to local and in case of failure it will connect to enterprise db. Then it will return ready DbContext object. This factory will be provided to all objects that require it with Dependency Injection system - they will use it to create contexts and dispose of them when they are not needed any more.
" We would like our app to attempt to connect to one of the local sites, and if that site is down we want it to fall back to the enterprise DB. We'd like this behavior on each of our DB operations."
If your app is strictly read-only on the DB and data consistency is not absolutely vital to your app/users, then it's just a matter of trying to CONNECT until an operational site has been found. As M.Ali suggested in his remark.
Otherwise, I suggest you stop thinking along these lines immediately because you're just running 90 mph down a dead end street. As Viktor Zychla suggested in his remark.
Here is what I ended up implementing, in broad brush-strokes:
Define delegates called UnitOfWorkMethod that will execute a single Unit of Work on the Database, in a single transaction. It takes a connection string and one also returns a value:
delegate T UnitOfWorkMethod<out T>(string connectionString);
delegate void UnitOfWorkMethod(string connectionString);
Define a method called ExecuteUOW, that will take a unit of work and method try to execute it using the preferred connection string. If it fails, it tries to execute it with the next connection string:
protected T ExecuteUOW<T>(UnitOfWorkMethod<T> method)
{
// GET THE LIST OF CONNECTION STRINGS
IEnumerable<string> connectionStringList = ConnectionStringProvider.GetConnectionStringList();
// WHILE THERE ARE STILL DATABASES TO TRY, AND WE HAVEN'T DEFINITIVELY SUCCEDED OR FAILED
var uowState = UOWStateEnum.InProcess;
IEnumerator<string> stringIterator = connectionStringList.GetEnumerator();
T returnVal = default(T);
Exception lastException = null;
string connectionString = null;
while ((uowState == UOWStateEnum.InProcess) && stringIterator.MoveNext())
{
try
{
// TRY TO EXECUTE THE UNIT OF WORK AGAINST THE DB.
connectionString = stringIterator.Current;
returnVal = method(connectionString);
uowState = UOWStateEnum.Success;
}
catch (Exception ex)
{
lastException = ex;
// IF IT FAILED BECAUSE OF A TRANSIENT EXCEPTION,
if (TransientChecker.IsTransient(ex))
{
// LOG THE EXCEPTION AND TRY AGAINST ANOTHER DB.
Log.TransientDBException(ex, connectionString);
}
// ELSE
else
{
// CONSIDER THE UOW FAILED.
uowState = UOWStateEnum.Failed;
}
}
}
// LOG THE FAILURE IF WE HAVE NOT SUCCEEDED.
if (uowState != UOWStateEnum.Success)
{
Log.ExceptionDuringDataAccess(lastException);
returnVal = default(T);
}
return returnVal;
}
Finally, for each operation we define our unit of work delegate method. Here an example
UnitOfWorkMethod uowMethod =
(providerConnectionString =>
{
using (var db = new MyContext(providerConnectionString ))
{
// Do my DB commands here. They will roll back if exception thrown.
}
});
ExecuteUOW(uowMethod);
When ExecuteUOW is called, it tries the delegate on each database until it either succeeds or fails on all of them.
I'm going to accept this answer since it fully addresses all of concerns raised in the original question. However, if anyone provides and answer that is more elegant, understandable, or corrects flaws in this one I'll happily accept it instead.
Thanks to all who have responded.
I'm having a rare issue in my code, I have a method that makes a very simple validation based on a string variable:
private void showNextStep(String psCondition,String poStepName){
int liCurrentStep=Integer.valueOf(poStepName);
String lsNextTrueStep=moSteps[liCurrentStep][4];
String liNextFalseStep=moSteps[liCurrentStep][5];
if ("Yes".equals(psCondition)){
moFrmStepsContainer.getField(liNextFalseStep).hide();
moFrmStepsContainer.getField(lsNextTrueStep).show();
}else{
moFrmStepsContainer.getField(liNextFalseStep).show();
moFrmStepsContainer.getField(lsNextTrueStep).hide();
}
}
Now, here is the ticky part: if I execute the application without debugging mode, it does the validation right all the time, how ever if don't it always goes to the else block (or at least I think) I tried to use JS alerts (I have a class that calls JS methods) to debug manually and check the valors of the variables; the valors where all right and the validation was also good. This means that only debugging or putting alerts before at the beggining of the IF block it does the validation right, otherwise it always goes to the ELSE, what do you think it could be?
It might be worth mentioning this is a web application made in netbeans 6.9, using the framework GWT 2.1. This application runs in firefox 25.0.1
Thank you!
UPDATE
Here is the code of the event that calls my method
final ComboBoxItem loYesNo=new ComboBoxItem("cmbYesNo" + moSteps[liStepIndex][0],"");
loYesNo.setValueMap("Yes","No");
loYesNo.setVisible(false);
loYesNo.setAttribute("parent", liStepIndex);
loYesNo.addChangedHandler(new ChangedHandler() {
public void onChanged(ChangedEvent poEvent){
String lsStepName=loYesNo.getName();
FormItem loItem=moFrmStepsContainer.getField(lsStepName);
String liStepNumber=String.valueOf(loItem.getAttributeAsInt("parent"));
showNextStep((String)poEvent.getItem().getValue(),liStepNumber);
}
});