Using mirth version 3.8.1. I've set up an alert for a channel's errors. When errors come from the destination transformer (which is Javascript), the alert is able to access the {messageId} variable and pull the correct id. However, when an error originates in the Javascript-based database writer, the alert just returns '{messageId}' instead of the value.
I tried a bunch of things...
The global map is accessible from the alert, but putting a message id in there would get overwritten by another processing thread.
Other destination types - http sender, tcp sender, channel writer, and even a non-javascript-based database writer destination all work.
I even stripped the database writer code down to just:
var dbConn;
dbConn = DatabaseConnectionFactory.createDatabaseConnection('com.mysql.cj.jdbc.Driver','jdbc:mysql://host:port/dbname','','');
Do I just have to raise specific exceptions within the db writer code and raise alerts when those exceptions are hit, and send the message id in the error string?
You stumbled across a bug. I opened an issue and a fix.
If not for another bug that also neglects to provide the messageId, you should be able to use alerts.sendAlert('Custom Error Message'). alerts is an instance of AlertSender from the User API that mirth creates for you. I created a fix for that as well.
The only workaround I know of at this time to manually send an alert that includes the messageId is to call the EventController directly. The caveat is that this is technically not supported as part of a public API and usage could break in future versions without notice.
com.mirth.connect.server.controllers.ControllerFactory
.getFactory()
.createEventController()
.dispatchEvent(new com.mirth.connect.donkey.server.event.ErrorEvent(
connectorMessage.getChannelId(),
connectorMessage.getMetaDataId(),
connectorMessage.getMessageId(),
com.mirth.connect.donkey.model.event.ErrorEventType.USER_DEFINED_TRANSFORMER,
connectorMessage.getConnectorName(),
null, /* connectorType */
'A TEST ERROR MESSAGE',
null /* throwable */
)
);
This will work as written from a filter, transformer, Javascript Writer, or Database Writer in javascript mode. In other contexts, connectorMessage won't be defined and you'll have to provide some of those values in a different way. If you don't need the messageId and don't want to throw an exception, just use alerts.sendAlert(errorMessage) since that doesn't require calling unsupported internal classes.
Related
I've configured monolog to send errors via email as described in the symfony docs here: https://symfony.com/doc/4.3/logging/monolog_email.html
Works well with all errors happing during a request, as well as console command errors.
But it does not send emails for errors which occurred during the handling of a messenger message.
Errors are shown when running the consumer bin/console messenger:consume async -vv and they also show up in prod.log like this:
[2020-01-10 12:52:38] messenger.CRITICAL: Error thrown while handling message...
Thanks for any hints on how to set up monolog to get messenger errors emailed too.
In fact monolog swift_mailer type use SwiftMailerHandler
wish also implements reset interface and use memory spool by default wish keep all emails in buffer until it is destructed, so till the end of request :
onKernelTerminate
onCliTerminate
OR till reset method is called, which means that for messenger worker no emails will be send ever because ther's no instant flush - all of them will be kept in in-memory buffer, and probably lost if the process will be killed.
To solve this, you can just disable the default spool memory setting for swiftmailer.
Another solution is to flush your emails after WorkerMessageFailedEvent event gets fired, you can implement an event subscriber to do it for this.
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\Messenger\Event\WorkerMessageFailedEvent;
use Symfony\Component\Messenger\Event\WorkerMessageHandledEvent;
use Symfony\Contracts\Service\ResetInterface;
/**
* Class ServiceResetterSubscriber.
*/
class ServiceResetterSubscriber implements EventSubscriberInterface
{
protected ResetInterface $servicesResetter;
public function __construct(ResetInterface $servicesResetter)
{
$this->servicesResetter = $servicesResetter;
}
public function resetServices(): void
{
$this->servicesResetter->reset();
}
public static function getSubscribedEvents(): array
{
return [
WorkerMessageFailedEvent::class => ['resetServices', 10],
];
}
}
Register your service with the right argument:
App\EventSubscriber\ServiceResetterSubscriber:
arguments: ['#services_resetter']
By the way without this (and without buffer limit) your app will leak and no emails will be sent ever.
Another trick:
Make sure that your message implements \JsonSerializable to get the message content in your logs, because messenger uses his monolog directly and its context serializer wish use json_encode for seriliazation.
That's why we need to customize their JSON representation when encoded is done with json_encode.
Using a saga, given an event EventA, saga starts, it sends a command (or many).
How can we make sure that the command is sent successfully then actual logic in other micro-service did not throw, etc.
Let's have an example of email saga:
When a user register, we create a User Aggregate which publishes UserRegisteredEvent, a saga will be created and this saga is responsible to make sure that registration email is sent to user (email may contain a verification key, welcome message, etc).
Should we use :
commandGateway.sendAndWait with a try/catch -> does it scale?
commandGateway.send and use a deadline and use some kind of "fail event" like SendEmailFailedEvent -> requires to associate a "token" for commands so can associate the "associationProperty" with the correct saga
that sent SendRegistrationEmailCommand
commandGateway.send(...).handle(...) -> in handle can we reference eventGateway/commandGateway that were in MyEmailSaga?
If error we send an event? Or can we modify/call a method from the saga instance we had. If no error then other service have sent an event like "RegistrationEmailSentEvent" so saga will end.
use deadline because we just use "send" and do not handle the eventual error of the command which may have failed to be sent (other service is down, etc)
something else?
Or a combination of all?
How to handle errors below? (use deadline or .handle(...) or other)
Errors could be:
command has no handlers (no service up, etc)
command was handled but exception is raised in other service and no event is sent (no try/catch in other service)
command was handled, exception raised and caught, other service publish an event to notify it failed to send email (saga will receive event and do appropriate action depending on event type and data provided -> maybe email is wrong or does not exist so no need to retry)
other errors I missed?
#Saga
public class MyEmailSaga {
#Autowired
transient CommandGateway commandGateway;
#Autowired
transient EventGateway eventGateway;
#Autowired
transient SomeService someService;
String id;
SomeData state;
/** count retry times we send email so can apply logic on it */
int sendRetryCount;
#StartSaga
#SagaEventHandler(associationProperty = "id")
public void on(UserRegisteredEvent event) {
id = event.getApplicationId();
//state = event........
// what are the possibilities here?
// Can we use sendAndWait but it does not scale very well, right?
commandGateway.send(new SendRegistrationEmailCommand(...));
// Is deadline good since we do not handle the "send" of the command
}
// Use a #DeadlineHandler to retry ?
#DeadlineHandler(deadlineName = "retry_send_registration_email")
fun on() {
// resend command and re-schedule a deadline, etc
}
#EndSaga
#SagaEventHandler(associationProperty = "id")
public void on(RegistrationEmailSentEvent event) {
}
}
EDIT (after accepted answer):
Mainly two options (Sorry but kotlin code below):
First option
commandGateway.send(SendRegistrationEmailCommand(...))
.handle({ t, result ->
if (t != null) {
// send event (could be caught be the same saga eventually) or send command or both
}else{
// send event (could be caught be the same saga eventually) or send command or both
}
})
// If not use handle(...) then you can use thenApply as well
.thenApply { eventGateway.publish(SomeSuccessfulEvent(...)) }
.thenApply { commandGateway.send(SomeSuccessfulSendOnSuccessCommand) }
2nd option:
Use a deadline to make sure that saga do something if SendRegistrationEmailCommand failed and you did not receive any events on the failure (when you do not handle the command sent).
Can of course use deadline for other purposes.
When the SendRegistrationEmailCommand was received successfully, the receiver will publish an event so the saga will be notified and act on it.
Could be an RegistrationEmailSentEvent or RegistrationEmailSendFailedEvent.
Summary:
It seems that it is best to use handle() only if the command failed to be sent or receiver has thrown an unexpected exception, if so then publish an event for the saga to act on it.
In case of success, the receiver should publish the event, saga will listen for it (and eventually register a deadline just in case); Receiver may also send event to notify of error and do not throw, saga will also listen to this event.
ideally, you would use the asynchronous options to deal with errors. This would either be commandGateway.send(command) or commandGateway.send(command).thenApply(). If the failure are businesslogic related, then it may make sense to emit events on these failures. A plain gateway.send(command) then makes sense; the Saga can react on the events returned as a result. Otherwise, you will have to deal with the result of the command.
Whether you need to use sendAndWait or just send().then... depends on the activity you need to do when it fails. Unfortunately, when dealing with results asynchronously, you cannot safely modify the state of the Saga anymore. Axon may have persisted the state of the saga already, causing these changes to go lost. sendAndWait resolves that. Scalability is not often an issue, because different Sagas can be executed in parallel, depending on your processor configuration.
The Axon team is currently looking at possible APIs that would allow for safe asynchronous execution of logic in Sagas, while still keeping guarantees about thread safety and state persistence.
The question assumes the use of Event Sourcing.
When rebuilding current state by replaying events, event handlers should be idempotent. For example, when a user successfully updates their username, a UsernameUpdated event might be emitted, the event containing a newUsername string property. When rebuilding current state, the appropriate event handler receives the UsernameUpdated event and sets the username property on the User object to the newUsername property of the UsernameUpdated event object. In other words, the handling of the same message multiple times always yields the same result.
However, how does such an event handler work when integrating with external services? For example, if the user wants to reset their password, the User object might emit a PasswordResetRequested event, which is handled by a portion of code that issues a 3rd party with a command to send an SMS. Now when the application is rebuilt, we do NOT want to re-send this SMS. How is this situation best avoided?
There are two messages involved in the interaction: commands and events.
I do not regard the system messages in a messaging infrastructure the same as domain events. Command message handling should be idempotent. Event handlers typically would not need to be.
In your scenario I could tell the aggregate root 100 times to update the user name:
public UserNameChanged ChangeUserName(string username, IServiceBus serviceBus)
{
if (_username.Equals(username))
{
return null;
}
serviceBus.Send(new SendEMailCommand(*data*));
return On(new UserNameChanged{ Username = userName});
}
public UserNameChanged On(UserNameChanged #event)
{
_username = #event.UserName;
return #event;
}
The above code would result in a single event so reconstituting it would not produce any duplicate processing. Even if we had 100 UserNameChanged events the result would still be the same as the On method does not perform any processing. I guess the point to remember is that the command side does all the real work and the event side is used only to change the state of the object.
The above isn't necessarily how I would implement the messaging but it does demonstrate the concept.
I think you are mixing two separate concepts here. The first is reconstructing an object where the handlers are all internal methods of the entity itself. Sample code from Axon framework
public class MyAggregateRoot extends AbstractAnnotatedAggregateRoot {
#AggregateIdentifier
private String aggregateIdentifier;
private String someProperty;
public MyAggregateRoot(String id) {
apply(new MyAggregateCreatedEvent(id));
}
// constructor needed for reconstruction
protected MyAggregateRoot() {
}
#EventSourcingHandler
private void handleMyAggregateCreatedEvent(MyAggregateCreatedEvent event) {
// make sure identifier is always initialized properly
this.aggregateIdentifier = event.getMyAggregateIdentifier();
// do something with someProperty
}
}
Surely you wouldn't put code that talks to an external API inside an aggregate's method.
The second is replaying events on a bounded context which could cause the problem you are talking about and depending on your case you may need to divide your event handlers into clusters.
See Axon frameworks documentation for this point to get a better understanding of the problem and the solution they went with.
Replaying Events on a Cluster
TLDR; store the SMS identifier within the event itself.
A core principle of event sourcing is "idempotency". Events are idempotent, meaning that processing them multiple times will have the same result as if they were processed once. Commands are "non-idempotent", meaning that the re-execution of a command may have a different result for each execution.
The fact that aggregates are identified by UUID (with a very low percentage of duplication) means that the client can generate the UUIDs of newly created aggregates. Process managers (a.k.a., "Sagas") coordinate actions across multiple aggregates by listening to events in order to issue commands, so in this sense, the process manager is also a "client". Cecause the process manager issues commands, it cannot be considered "idempotent".
One solution I came up with is to include the UUID of the soon-to-be-created SMS in the PasswordResetRequested event. This allows the process manager to only create the SMS if it does not yet already exist, hence achieving idempotency.
Sample code below (C++ pseudo-code):
// The event indicating a password reset was successfully requested.
class PasswordResetRequested : public Event {
public:
PasswordResetRequested(const Uuid& userUuid, const Uuid& smsUuid, const std::string& passwordResetCode);
const Uuid userUuid;
const Uuid smsUuid;
const std::string passwordResetCode;
};
// The user aggregate root.
class User {
public:
PasswordResetRequested requestPasswordReset() {
// Realistically, the password reset functionality would have it's own class
// with functionality like checking request timestamps, generationg of the random
// code, etc.
Uuid smsUuid = Uuid::random();
passwordResetCode_ = generateRandomString();
return PasswordResetRequested(userUuid_, smsUuid, passwordResetCode_);
}
private:
Uuid userUuid_;
string passwordResetCode_;
};
// The process manager (aka, "saga") for handling password resets.
class PasswordResetProcessManager {
public:
void on(const PasswordResetRequested& event) {
if (!smsRepository_.hasSms(event.smsUuid)) {
smsRepository_.queueSms(event.smsUuid, "Your password reset code is: " + event.passwordResetCode);
}
}
};
There are a few things to note about the above solution:
Firstly, while there is a (very) low possibility that the SMS UUIDs can conflict, it can actually happen, which could cause several issues.
Communication with the external service is prevented. For example, if user "bob" requests a password reset that generates an SMS UUID of "1234", then (perhaps 2 years later) user "frank" requests a password reset that generates the same SMS UUID of "1234", the process manager will not queue the SMS because it thinks it already exists, so frank will never see it.
Incorrect reporting in the read model. Because there is a duplicate UUID, the read side may display the SMS sent to "bob" when "frank" is viewing the list of SMSes the system sent him. If the duplicate UUIDs were generated in quick succession, it is possible that "frank" would be able to reset "bob"s password.
Secondly, moving the SMS UUID generation into the event means you must make the User aggregate aware of the PasswordResetProcessManager's functionality (but not the PasswordResetManager itself), which increases coupling. However, the coupling here is loose, in that the User is unaware of how to queue an SMS, only that an SMS should be queued. If the User class were to send the SMS itself, you could run into the situation in which the SmsQueued event is stored while the PasswordResetRequested event is not, meaning that the user will receive an SMS but the generated password reset code was not saved on the user, and so entering the code will not reset the password.
Thirdly, if a PasswordResetRequested event is generated but the system crashes before the PasswordResetProcessManager can create the SMS, then the SMS will eventually be sent, but only when the PasswordResetRequested event is re-played (which might be a long time in the future). E.g., the "eventual" part of eventual consistency could be a long time away.
The above approach works (and I can see that it should also work in more complicated scenarious, like the OrderProcessManager described here: https://msdn.microsoft.com/en-us/library/jj591569.aspx). However, I am very keen to hear what other people think about this approach.
I have a ServiceWorker registered on my page and want to pass some data to it so it can be stored in an IndexedDB and used later for network requests (it's an access token).
Is the correct thing just to use network requests and catch them on the SW side using fetch, or is there something more clever?
Note for future readers wondering similar things to me:
Setting properties on the SW registration object, e.g. setting self.registration.foo to a function within the service worker and doing the following in the page:
navigator.serviceWorker.getRegistration().then(function(reg) { reg.foo; })
Results in TypeError: reg.foo is not a function. I presume this is something to do with the lifecycle of a ServiceWorker meaning you can't modify it and expect those modification to be accessible in the future, so any interface with a SW likely has to be postMessage style, so perhaps just using fetch is the best way to go...?
So it turns out that you can't actually call a method within a SW from your app (due to lifecycle issues), so you have to use a postMessage API to pass serialized JSON messages around (so no passing callbacks etc).
You can send a message to the controlling SW with the following app code:
navigator.serviceWorker.controller.postMessage({'hello': 'world'})
Combined with the following in the SW code:
self.addEventListener('message', function (evt) {
console.log('postMessage received', evt.data);
})
Which results in the following in my SW's console:
postMessage received Object {hello: "world"}
So by passing in a message (JS object) which indicates the function and arguments I want to call my event listener can receive it and call the right function in the SW. To return a result to the app code you will need to also pass a port of a MessageChannel in to the SW and then respond via postMessage, for example in the app you'd create and send over a MessageChannel with the data:
var messageChannel = new MessageChannel();
messageChannel.port1.onmessage = function(event) {
console.log(event.data);
};
// This sends the message data as well as transferring messageChannel.port2 to the service worker.
// The service worker can then use the transferred port to reply via postMessage(), which
// will in turn trigger the onmessage handler on messageChannel.port1.
// See https://html.spec.whatwg.org/multipage/workers.html#dom-worker-postmessage
navigator.serviceWorker.controller.postMessage(message, [messageChannel.port2]);
and then you can respond via it in your Service Worker within the message handler:
evt.ports[0].postMessage({'hello': 'world'});
To pass data to your service worker, the above mentioned is a good way. But in case, if someone is still having a hard time implementing that, there is an other hack around for that,
1 - append your data to get parameter while you load service-worker (for eg., from sw.js -> sw.js?a=x&b=y&c=z)
2- Now in service worker, fetch those data using self.self.location.search.
Note, this will be beneficial only if the data you pass do not change for a particular client very often, other wise it will keep changing the loading url of service worker for that particular client and every time the client reloads or revisits, new service worker is installed.
I've started using the AspProviders code to store my session data in my table storage.
I'm sporadically getting the following error:
Description: Exception of type 'System.Web.HttpException' was thrown. INNER_EXCEPTION:Error accessing the data store! INNER_EXCEPTION:An error occurred while processing this request. INNER_EXCEPTION: ConditionNotMet The condition specified using HTTP conditional header(s) is not met. RequestId:0c4239cc-41fb-42c5-98c5-7e9cc22096af Time:2010-10-15T04:28:07.0726801Z
StackTrace:
System.Web.SessionState.SessionStateModule.EndAcquireState(IAsyncResult ar)
System.Web.HttpApplication.AsyncEventExecutionStep.OnAsyncEventCompletion(IAsyncResult ar) INNER_EXCEPTION:
Microsoft.Samples.ServiceHosting.AspProviders.TableStorageSessionStateProvider.ReleaseItemExclusive(HttpContext context, String id, Object lockId) in \Azure\AspProviders\TableStorageSessionStateProvider.cs:line 484
System.Web.SessionState.SessionStateModule.GetSessionStateItem()
System.Web.SessionState.SessionStateModule.PollLockedSessionCallback(Object state) INNER_EXCEPTION:
Microsoft.WindowsAzure.StorageClient.Tasks.Task1.get_Result()
Microsoft.WindowsAzure.StorageClient.Tasks.Task1.ExecuteAndWait()
Microsoft.WindowsAzure.StorageClient.TaskImplHelper.ExecuteImplWithRetry[T](Func`2 impl, RetryPolicy policy)
Microsoft.Samples.ServiceHosting.AspProviders.TableStorageSessionStateProvider.ReleaseItemExclusive(TableServiceContext svc, SessionRow session, Object lockId) in \Azure\AspProviders\TableStorageSessionStateProvider.cs:line 603
Microsoft.Samples.ServiceHosting.AspProviders.TableStorageSessionStateProvider.ReleaseItemExclusive(HttpContext context, String id, Object lockId) in \Azure\AspProviders\TableStorageSessionStateProvider.cs:line 480 INNER_EXCEPTION:
System.Data.Services.Client.DataServiceContext.SaveResult.d__1e.MoveNext()
Anyone run into this? The only useful information I've found is this, which I'm hesitant to do:
If you want to bypass the validation, you can open TableStorageSessionStateProvider.cs, find ReleaseItemExclusive, and modify the code from:
svc.UpdateObject(session);
to:
svc.Detach(session);
svc.AttachTo("Sessions", session, "*");
svc.UpdateObject(session);
from here
Thanks!
So I decided to change this:
svc.UpdateObject(session);
svc.SaveChangesWithRetries();
to this:
try
{
svc.UpdateObject(session);
svc.SaveChangesWithRetries();
}
catch
{
svc.Detach(session);
svc.AttachTo("Sessions", session, "*");
svc.UpdateObject(session);
svc.SaveChangesWithRetries();
}
So, I'll see how that works...
I've encountered this problem as well and after some investigation it seems to happen more often when you have more than one instance and you try to make calls in rapid succession in the same session. (e.g. if you had an auto complete box and making ajax calls on each key press)
This occurs because when you try to access the session data, first of all the web server takes out a lock on that session. When the request is complete, it releases the lock. With the table service provider, it updates this lock status by updating a field in the table. What I think is happening is that Instance1 loads the session row, then Instance2 loads the session row, Instance1 saves down the updated lock status and when Instance2 attempts to save the lock status it gets an error because the object isn't in the same state as when it loaded it (the ETag doesn't match any more).
This is why the fix that you found will stop the error from occurring, because by specifying the "*" in the AttachTo, when Instance2 attempts to save the lock it will turn off ETag checking (and over write the changes made by Instance1).
In our situation we have altered the provider so that we can turn off session for certain paths (the ajax call that was giving us our problems didn't need access to session data, neither did the loading of images) which may be an option for you depending on what is causing your problem.
Unfortunately the TableStorageSessionStateProvider is part of the sample projects and so isn't (as far as I'm aware, but I'll happily be told otherwise) officially supported by Microsoft. It does have other issues, like the fact that it doesn't clean up it's session data once a session expires, so you will end up with lots of junk in the session table and blob container that you'll have to clean up some other way.