On AEM CaaS, we are trying to send email notification If replication queue is stuck via custom ReplicationEventHandler. We used the agent manager to get the replication queue and trying to add send email logic when queue is blocked.
We have applied 2 approaches based upon the API Docs which doesn't seems working.
Approach 1 : This sends the emails multiple times, even queue is not blocked
for (Agent agent : agentsMap.values()) {
if (agent.isEnabled() && agent.getId().equals("publish")) {
ReplicationQueue replicationQueue = agent.getQueue();
if(replicationQueue.getStatus().getNextRetryTime() != 0) {
Map<String, String> emailParams = new HashMap<>();
emailParams.put("agentId",agent.getId());
emailParams.put("agentName",agent.getConfiguration().getConfigPath());
sendEmail(emailParams);
log.info("::: Replication Queue Blocked :::");
}
}
}
}
Approach 2 : This doesn't trigger email, even queue is blocked.
if(agent.isValid() && agent.isEnabled()) {
ReplicationQueue replicationQueue = agent.getQueue();
if(!replicationQueue.entries().isEmpty()) {
ReplicationQueue.Entry firstEntry = replicationQueue.entries().get(0);
if(firstEntry.getNumProcessed() > 3) {
// Send Email That Queue Is Blocked
}
} else {
// Queue is Not Empty
}
}
Looking for solution..
Thanks
Related
My script at G AppsScript ran more often than the trigger is set for.
The purpose of the script is to check Gmail inbox every hour and if an automated email was not delivered - alert a slack channel.
There is an automation that delivers email to the Gmail address every hour, Gmail rules add a label to the emails. The script checks for the label, if found - the label is removed, the email is marked as read. When there is no label - webhook url is triggered to send an alert.
However, now the code was executed 3 times within an hour instead of 1 time as trigger is set. This resulted in 2 notifications to slack.
Could someone help to understand what is wrong?
trigger
executions
function parseEmailByLabel() {
var gmailLabelName = "ParseThis",
externalHandlerScript = "https://hooks.slack.com/workflows/T1234",
gmailLabelObject = GmailApp.getUserLabelByName(gmailLabelName),
threads = gmailLabelObject.getThreads(),
messages,
message,
params,
response;
if (threads != "") {
for (var i = 0; i < threads.length; i++) {
messages = threads[i].getMessages();
for (var j = 0; j < messages.length; j++) {
message = messages[j];
message.markRead();
}
threads[i].removeLabel(gmailLabelObject);
}
} else if (threads == "") {
params = {
'method': 'post',
};
response = UrlFetchApp.fetch(externalHandlerScript, params).getContentText();
Logger.log(response);
}
}
You may have created more than one trigger. You may check here:
https://script.google.com/home/triggers
We are using gcloud pub-sub 1.102.0 release.
Problem :- We have around 8K messages to process. We have 3 subscribers running in 3 different k8s pods and we are doing stream pulling. We are receiving the ACK for few messages (around 100) from there on wards we are not receiving the ACKs (Verified in GCP console). But the message process is going on in background. And we saw couple of duplicate messages as well. In our use case to process a single message it takes around 40 secs to 1 min. Ack deadline has been configured to 10 mins while creating the subscription.
Flow control configured with 20L.
ExecutorProvider configured with 3.
public void runSubsciber() throws Exception {
Subscriber subscriber = null;
MessageReceiver receiver = new MessageReceiver() {
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
messageProcessor.processMessage(message.getData().toStringUtf8());
consumer.ack();
}
};
try {
FlowControlSettings flowControlSettings =
FlowControlSettings.newBuilder()
.setMaxOutstandingElementCount(20L)
.build();
ExecutorProvider executorProvider =
InstantiatingExecutorProvider.newBuilder().setExecutorThreadCount(3).build();
projectId=ServiceOptions.getDefaultProjectId();
ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(projectId, subId);
subscriber = Subscriber.newBuilder(subscriptionName, receiver)
.setExecutorProvider(executorProvider)
.setFlowControlSettings(flowControlSettings)
.build();
subscriber.addListener(new SubscriberListener(), MoreExecutors.directExecutor());
subscriber.startAsync().awaitRunning();
} catch (Exception e) {
throw new Exception(e);
} finally {
if (subscriber != null) {
//subscriber.stopAsync();
}
}
}
Please help us out, Thanks in advance.
I am building a OPC UA Client using OPC Foundation SDK. I am able to create a subscription containing some Monitoreditems.
On the OPC UA server these monitored items change value constantly (every second or so).
I want to disconnect the client (simulate a connection broken ), keep the subcription alive and wait for a while. Then I reconnect having my subscriptions back, but I also want all the monitored Item values queued up during the disconnect. Right now I only get the last server value on reconnect.
I am setting a queuesize:
monitoredItem.QueueSize = 100;
To kind of simulate a connection error I have set the "delete subscription" to false on ClosesSession:
m_session.CloseSession(new RequestHeader(), false);
My question is how to capture the content of the queue after a disconnect/connection error???
Should the ‘lost values’ be “new MonitoredItem_Notification” automatically when the client reconnect?
Should the SubscriptionId be the same as before the connection was broken?
Should the sessionId be the same or will a new SessionId let med keep the existing subscriptions? What is the best way to simulate a connection error?
Many questions :-)
A sample from the code where I create the subscription containing some MonitoredItems and the MonitoredItem_Notification event method.
Any OPC UA Guru out there??
if (node.Displayname == "node to monitor")
{
MonitoredItem mon = CreateMonitoredItem((NodeId)node.reference.NodeId, node.Displayname);
m_subscription.AddItem(mon);
m_subscription.ApplyChanges();
}
private MonitoredItem CreateMonitoredItem(NodeId nodeId, string displayName)
{
if (m_subscription == null)
{
m_subscription = new Subscription(m_session.DefaultSubscription);
m_subscription.PublishingEnabled = true;
m_subscription.PublishingInterval = 3000;//1000;
m_subscription.KeepAliveCount = 10;
m_subscription.LifetimeCount = 10;
m_subscription.MaxNotificationsPerPublish = 1000;
m_subscription.Priority = 100;
bool cache = m_subscription.DisableMonitoredItemCache;
m_session.AddSubscription(m_subscription);
m_subscription.Create();
}
// add the new monitored item.
MonitoredItem monitoredItem = new MonitoredItem(m_subscription.DefaultItem);
//Each time a monitored item is sampled, the server evaluates the sample using a filter defined for each monitoreditem.
//The server uses the filter to determine if the sample should be reported. The type of filter is dependent on the type of item.
//DataChangeFilter for Variable, Eventfilter when monitoring Events. etc
//MonitoringFilter f = new MonitoringFilter();
//DataChangeFilter f = new DataChangeFilter();
//f.DeadbandValue
monitoredItem.StartNodeId = nodeId;
monitoredItem.AttributeId = Attributes.Value;
monitoredItem.DisplayName = displayName;
//Disabled, Sampling, (Report (includes sampling))
monitoredItem.MonitoringMode = MonitoringMode.Reporting;
//How often the Client wish to check for new values on the server. Must be 0 if item is an event.
//If a negative number the SamplingInterval is set equal to the PublishingInterval (inherited)
//The Subscriptions KeepAliveCount should always be longer than the SamplingInterval/PublishingInterval
monitoredItem.SamplingInterval = 500;
//Number of samples stored on the server between each reporting
monitoredItem.QueueSize = 100;
monitoredItem.DiscardOldest = true;//Discard oldest values when full
monitoredItem.CacheQueueSize = 100;
monitoredItem.Notification += m_MonitoredItem_Notification;
if (ServiceResult.IsBad(monitoredItem.Status.Error))
{
return null;
}
return monitoredItem;
}
private void MonitoredItem_Notification(MonitoredItem monitoredItem, MonitoredItemNotificationEventArgs e)
{
if (this.InvokeRequired)
{
this.BeginInvoke(new MonitoredItemNotificationEventHandler(MonitoredItem_Notification), monitoredItem, e);
return;
}
try
{
if (m_session == null)
{
return;
}
MonitoredItemNotification notification = e.NotificationValue as MonitoredItemNotification;
if (notification == null)
{
return;
}
string sess = m_session.SessionId.Identifier.ToString();
string s = string.Format(" MonitoredItem: {0}\t Value: {1}\t Status: {2}\t SourceTimeStamp: {3}", monitoredItem.DisplayName, (notification.Value.WrappedValue.ToString().Length == 1) ? notification.Value.WrappedValue.ToString() : notification.Value.WrappedValue.ToString(), notification.Value.StatusCode.ToString(), notification.Value.SourceTimestamp.ToLocalTime().ToString("HH:mm:ss.fff"));
richTextBox1.AppendText(s + "SessionId: " + sess);
}
catch (Exception exception)
{
ClientUtils.HandleException(this.Text, exception);
}
}e here
I don't know how much of this, if any, the SDK you're using does for you, but the approach when reconnecting is generally:
try to resume (re-activate) your old session. If this is successful your subscriptions will already exist and all you need to do is send more PublishRequests. Since you're trying to test by closing the session this probably won't work.
create a new session and then call the TransferSubscription service to transfer the previous subscriptions to your new session. You can then start sending PublishRequests and you'll get the queued notifications.
Again, depending on the stack/SDK/toolkit you're using some or none of this may be handled for you.
I'm using MassTransit with Reactive Extensions to stream messages from the queue in batches. Since the behaviour isn't the same as a normal consumer I need to be able to send a message to the error queue if it fails an x number of times.
I've looked through the MassTransit source code and posted on the google groups and can't find an anwser.
Is this available on the ConsumeContext interface? Or is this even possible?
Here is my code. I've removed some of it to make it simpler.
_busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://localhost/"), h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.UseInMemoryScheduler();
cfg.ReceiveEndpoint(host, "customer_update_queue", e =>
{
var _observer = new ObservableObserver<ConsumeContext<Customer>>();
_observer.Buffer(TimeSpan.FromMilliseconds(1000)).Subscribe(OnNext);
e.Observer(_observer);
});
});
private void OnNext(IList<ConsumeContext<Customer>> messages)
{
foreach (var consumeContext in messages)
{
Console.WriteLine("Content: " + consumeContext.Message.Content);
if (consumeContext.Message.RetryCount > 3)
{
// I want to be able to send to the error queue
consumeContext.SendToErrorQueue()
}
}
}
I've found a work around by using the RabbitMQ client mixed with MassTransit. Since I can't throw an exception when using an Observable and therefore no error queue is created. I create it manually using the RabbitMQ client like below.
ConnectionFactory factory = new ConnectionFactory();
factory.HostName = "localhost";
factory.UserName = "guest";
factory.Password = "guest";
using (IConnection connection = factory.CreateConnection())
{
using (IModel model = connection.CreateModel())
{
string exchangeName = "customer_update_queue_error";
string queueName = "customer_update_queue_error";
string routingKey = "";
model.ExchangeDeclare(exchangeName, ExchangeType.Fanout);
model.QueueDeclare(queueName, false, false, false, null);
model.QueueBind(queueName, exchangeName, routingKey);
}
}
The send part is to send it directly to the message queue if it fails an x amount of times like so.
consumeContext.Send(new Uri("rabbitmq://localhost/customer_update_queue_error"), consumeContext.Message);
Hopefully the batch feature will be implemented soon and I can use that instead.
https://github.com/MassTransit/MassTransit/issues/800
I am having trouble subscribing to a socketcluster (http://socketcluster.io/) channel when using a redux-saga generator in my chat app. The socketcluster backend is setup in a way where any messages are saved in the database then published into the receiving user's personal channel, which is named after the user's id. For example, User A has an id '123abc' and would subscribe to the channel named '123abc' for their realtime messages.
The code below does receive new messages that are published to a channel but it throws a "TypeError: Converting circular structure to JSON" onload and breaks all of my other redux-saga generators in the app. I've done digging in Chrome Devtools and my theory is that it has something to do with queue created in the createChannel function. Also, I've tried returning a deferred promise in the subscribeToChannel function but that also caused a Circular Conversion Error, I can post that code on request.
I referred to this answer at first: https://stackoverflow.com/a/35288877/5068616 and it helped me get the below code in place but I cannot find any similar issues on the internet. Also something to note, I am utilizing redux-socket-cluster (https://github.com/mattkrick/redux-socket-cluster) to sync up the socket and state, but I don't think it is the root of the problem
sagas.js
export default function* root() {
yield [
fork(startSubscription),
]
}
function* startSubscription(getState) {
while (true) {
const {
userId
} = yield take(actions.SUBSCRIBE_TO_MY_CHANNEL);
yield call(monitorChangeEvents, subscribeToChannel(userId))
}
}
function* monitorChangeEvents(channel) {
while (true) {
const info = yield call(channel.take) // Blocks until the promise resolves
console.log(info)
}
}
function subscribeToChannel(channelName) {
const channel = createChannel();
const socket = socketCluster.connect(socketConfig);
const c = socket.subscribe(channelName);
c.watch(event => {
channel.put(event)
})
return channel;
}
function createChannel() {
const messageQueue = []
const resolveQueue = []
function put(msg) {
// anyone waiting for a message ?
if (resolveQueue.length) {
// deliver the message to the oldest one waiting (First In First Out)
const nextResolve = resolveQueue.shift()
nextResolve(msg)
} else {
// no one is waiting ? queue the event
messageQueue.push(msg)
}
}
// returns a Promise resolved with the next message
function take() {
// do we have queued messages ?
if (messageQueue.length) {
// deliver the oldest queued message
return Promise.resolve(messageQueue.shift())
} else {
// no queued messages ? queue the taker until a message arrives
return new Promise((resolve) => resolveQueue.push(resolve))
}
}
return {
take,
put
}
}
Thanks for the help!