Distinguish between create and update event of Managed Objects - apama

I'm using Apama in Cumulocity. Whenever a managed object (device) is created in Cumulocity, I would like to provide it with some initial parameters, here the required interval the device needs to report to Cumulocity before it's considered to be unavailable.
My problem is that in Apama I don't seem to have any way to distinguish between create and update events. So if I receive a managed object, add some parameters to it and send it back to the managed object channel, I end up in a loop.
I can of course do some check after the event has been received, but I would prefer to filter on only the create events of managed objects and don't perform any IF checks.
Is there any way I can filter on only create events? What is the difference between the CHANNEL and the UPDATE_CHANNEL? It doesn't seem to make a difference which one I use.
My current code looks as follows. What I want to achieve is avoid the IF statement and filter directly on create events in the listener.
monitor InitializeDevice {
action onload() {
monitor.subscribe(ManagedObject.CHANNEL);
on all ManagedObject(type = "c8y_MQTTDevice") as mo {
log "###Received managed object. Content is: " + mo.toString() at INFO;
if (mo.params.hasKey("c8y_RequiredAvailability")) {
//Assuming an interval has already been set, do nothing.
log "###Received managed object with required availability fragment. Doing nothing." at INFO;
}
else {
//Set the response interval on the managed object
dictionary<string,any> params := mo.params;
dictionary<string,any> paramssub := new dictionary<string,any>;
paramssub.add("responseInterval",3);
params.add("c8y_RequiredAvailability",paramssub);
mo.params := params;
log "###Added required interval to managed object. Content is: " + mo.toString() at INFO;
send mo to ManagedObject.UPDATE_CHANNEL;
}
}
}
}
When I execute this monitor and create a new managed object, this is what is printed to the logs:
2019-05-27 16:15:07.310 INFO [12648] - InitializeDevice [6] ###Received managed object. Content is: com.apama.cumulocity.ManagedObject("5708279","c8y_MQTTDevice","some-device",[],[],[],[],[],[],{},{"c8y_IsDevice":any(dictionary<any,any>,{}),"owner":any(string,"some-owner")})
2019-05-27 16:15:07.310 INFO [12648] - InitializeDevice [6] ###Added required interval to managed object. Content is: com.apama.cumulocity.ManagedObject("5708279","c8y_MQTTDevice","some-device",[],[],[],[],[],[],{},{"c8y_IsDevice":any(dictionary<any,any>,{}),"c8y_RequiredAvailability":any(dictionary<string,any>,{"responseInterval":any(integer,3)}),"owner":any(string,"some-owner")})
2019-05-27 16:15:07.310 INFO [12648] - InitializeDevice [6] ###Received managed object. Content is: com.apama.cumulocity.ManagedObject("5708279","c8y_MQTTDevice","some-device",[],[],[],[],[],[],{},{"c8y_IsDevice":any(dictionary<any,any>,{}),"c8y_RequiredAvailability":any(dictionary<string,any>,{"responseInterval":any(integer,3)}),"owner":any(string,"some-owner")})
2019-05-27 16:15:07.310 INFO [12648] - InitializeDevice [6] ###Received managed object with required availability fragment. Doing nothing.
2019-05-27 16:15:08.244 INFO [7868] - InitializeDevice [6] ###Received managed object. Content is: com.apama.cumulocity.ManagedObject("5708279","c8y_MQTTDevice","some-device",[],[],[],[],[],[],{},{"c8y_Availability":any(dictionary<any,any>,{any(string,"lastMessage"):any(dictionary<any,any>,{any(string,"date"):any(integer,27),any(string,"day"):any(integer,1),any(string,"hours"):any(integer,16),any(string,"minutes"):any(integer,15),any(string,"month"):any(integer,4),any(string,"seconds"):any(integer,7),any(string,"time"):any(integer,1558966507220),any(string,"timezoneOffset"):any(integer,-120),any(string,"year"):any(integer,119)}),any(string,"status"):any(string,"AVAILABLE")}),"c8y_Connection":any(dictionary<any,any>,{any(string,"status"):any(string,"DISCONNECTED")}),"c8y_IsDevice":any(dictionary<any,any>,{}),"c8y_RequiredAvailability":any(dictionary<any,any>,{any(string,"responseInterval"):any(integer,3)}),"owner":any(string,"some-owner")})
2019-05-27 16:15:08.244 INFO [7868] - InitializeDevice [6] ###Received managed object with required availability fragment. Doing nothing.
Is there any way to filter directly on create events? Why do I received two print statements after the update?
Thanks
Mathias

After some investigation it seems that there isn't any way to distinguish the Creation and Update messages. So the code you are using at the moment is probably the only way to do this.
Edited:
BUT The second part of the question:
Why do I received two print statements after the update?
c8y sends managed object to MO.CHANNEL -> Apama monitor
monitor adds c8y_RequiredAvailability
monitor sends managed object with update to MO.UPDATE_CHANNEL -> c8y
c8y sends updated managed object containing c8y_RequiredAvailability -> Apama monitor
c8y sends managed object + c8y_Availability-> Apama monitor
so 3 is the confirmation of your update, and 4 is c8y asynchronously sending the final update with availability on the MO.CHANNEL
To be explicit - the MO.CHANNEL is where created and updated objects arrive into Apama. Sending on that channel shouldn't have an effect. The MO.UPDATE_CHANNEL is the request channel where you send updates to which may then trigger further messages on the MO.CHANNEL as c8y processes.

Related

Sharing objects with all verticles instances

My application, an API server, is thought to be organized as follows:
MainVerticle is called on startup and should create all necessary objects for the application to work. Mainly a mongoDB pool of connections (MongoClient.createShared(...)) and a global configuration object available instance-wide. It also starts the HTTP Listener, several instances of a HttpVerticle.
HttpVerticle is in charge of receiving requests and, based the command xxx in the payload, execute the XxxHandler.handle(...) method.
Most of the XxxHandler.handle(...) methods will need to access the database. In addition, some others will also deploy additional verticles with parameters from the global conf. For example LoginHandler.handle(...) will deploy a verticle to keep user state while he's connected and this verticle will be undeployed when the user logs out.
I can't figure out how to get the global configuration object while being in XxxHandler.handle(...) or in a "sub"-verticle. Same for the mongo client.
Q1: For configuration data, I tried to use SharedData. In `MainVerticle.start() I have:
LocalMap<String, String> lm = vertx.sharedData().getLocalMap("conf");
lm.put("var", "val");
and in `HttpVerticle.start() I have:
LocalMap<String, String> lm = vertx.sharedData().getLocalMap("conf");
log.debug("var={}", lm.get("var"));
but the log output is var=null.... What am I doing wrong ?
Q2: Besides this basic example with a <String, String> map type, what if the value is a mutable Object like JsonObject which actually is what I would need ?
Q3: Finally how to make the instance of the mongo client available to all verticles?
Instead of getLocalMap() you should be using getClusterWideMap(). Then you should be able to operate on shared data accross the whole cluster and not just in one verticle.
Be aware that the shared operations are async and the code might look like (code in Groovy):
vertx.sharedData().getClusterWideMap( 'your-name' ){ AsyncResult<AsyncMap<String,String>> res ->
if( res.succeeded() )
res.result().put( 'var', 'val', { log.info "put succeeded: ${it.succeeded()}" } )
}
You should be able to use any Serializable objects in your map.

Event MyEvent has ID 2 which is already in use

I am implementing event tracing using EWT in a Service Fabric application and are faced with these errors
ERROR: Exception in Command Processing for EventSource MyCompany-ServiceFabricApplication-LiveDataReader: Event OnCommandMessageReceived has ID 2 which is already in use
The "OnCommandMessageReceived" is my custom event
[Event(2, Level = EventLevel.Verbose, Message = "Queue client created '{0}'")]
public void OnQueueClientCreated(string queueClientName)
{
if (IsEnabled())
{
WriteEvent(2, queueClientName);
}
}
I have multiple/many of these errors and I have tried to messing around with numbers but ...
Is there some Powershell command or else that can tell what IDs are in use or is there a safe range or something?
PS: When that event is fired i can see it in visual studio diagnostic events viewer but the Message is empty. It would be cool if it displayed the message from the payload. Is that possible?
ETW Events must have an unique ID per provider. So look if you have other events with ID 2 and change tie ID to a different value.

How do I call a method on my ServiceWorker from within my page?

I have a ServiceWorker registered on my page and want to pass some data to it so it can be stored in an IndexedDB and used later for network requests (it's an access token).
Is the correct thing just to use network requests and catch them on the SW side using fetch, or is there something more clever?
Note for future readers wondering similar things to me:
Setting properties on the SW registration object, e.g. setting self.registration.foo to a function within the service worker and doing the following in the page:
navigator.serviceWorker.getRegistration().then(function(reg) { reg.foo; })
Results in TypeError: reg.foo is not a function. I presume this is something to do with the lifecycle of a ServiceWorker meaning you can't modify it and expect those modification to be accessible in the future, so any interface with a SW likely has to be postMessage style, so perhaps just using fetch is the best way to go...?
So it turns out that you can't actually call a method within a SW from your app (due to lifecycle issues), so you have to use a postMessage API to pass serialized JSON messages around (so no passing callbacks etc).
You can send a message to the controlling SW with the following app code:
navigator.serviceWorker.controller.postMessage({'hello': 'world'})
Combined with the following in the SW code:
self.addEventListener('message', function (evt) {
console.log('postMessage received', evt.data);
})
Which results in the following in my SW's console:
postMessage received Object {hello: "world"}
So by passing in a message (JS object) which indicates the function and arguments I want to call my event listener can receive it and call the right function in the SW. To return a result to the app code you will need to also pass a port of a MessageChannel in to the SW and then respond via postMessage, for example in the app you'd create and send over a MessageChannel with the data:
var messageChannel = new MessageChannel();
messageChannel.port1.onmessage = function(event) {
console.log(event.data);
};
// This sends the message data as well as transferring messageChannel.port2 to the service worker.
// The service worker can then use the transferred port to reply via postMessage(), which
// will in turn trigger the onmessage handler on messageChannel.port1.
// See https://html.spec.whatwg.org/multipage/workers.html#dom-worker-postmessage
navigator.serviceWorker.controller.postMessage(message, [messageChannel.port2]);
and then you can respond via it in your Service Worker within the message handler:
evt.ports[0].postMessage({'hello': 'world'});
To pass data to your service worker, the above mentioned is a good way. But in case, if someone is still having a hard time implementing that, there is an other hack around for that,
1 - append your data to get parameter while you load service-worker (for eg., from sw.js -> sw.js?a=x&b=y&c=z)
2- Now in service worker, fetch those data using self.self.location.search.
Note, this will be beneficial only if the data you pass do not change for a particular client very often, other wise it will keep changing the loading url of service worker for that particular client and every time the client reloads or revisits, new service worker is installed.

store and setRequest

I have a jobque mechanism in ZF.
The jobque simlpy stores the the function call (Class, Method and params) and later executes it as CLI daemon. The daemon works, however at places the application looks for information from the request object, and when called from the CLI these places fail, or get no info.
I would like to store the original request object together with the job and when the job is processed set the request object back as if the job was done by the originall request, somethin along the line of the following pseudo code:
$ser_request = serialize(Zend_Controller_Front::getInstance ()->getRequest ());
-->save to db
-->retrive from db
$ZCF= new Zend_Controller_Front;
$ZCF::getInstance ()->setRequest (unserialize($ser_request))
The aim is to store and replay the jobs later withouth having to change the rest of the application.
Any suggestions how to do that?
I am not sure if this works, but here's an idea. Try to implement _sleep and _wakeup magic methods for the request object. Haven't tried it out, but maybe it's at least a starting solution.

asp.net mvc azure "Error accessing the data store!"

I've started using the AspProviders code to store my session data in my table storage.
I'm sporadically getting the following error:
Description: Exception of type 'System.Web.HttpException' was thrown. INNER_EXCEPTION:Error accessing the data store! INNER_EXCEPTION:An error occurred while processing this request. INNER_EXCEPTION: ConditionNotMet The condition specified using HTTP conditional header(s) is not met. RequestId:0c4239cc-41fb-42c5-98c5-7e9cc22096af Time:2010-10-15T04:28:07.0726801Z
StackTrace:
System.Web.SessionState.SessionStateModule.EndAcquireState(IAsyncResult ar)
System.Web.HttpApplication.AsyncEventExecutionStep.OnAsyncEventCompletion(IAsyncResult ar) INNER_EXCEPTION:
Microsoft.Samples.ServiceHosting.AspProviders.TableStorageSessionStateProvider.ReleaseItemExclusive(HttpContext context, String id, Object lockId) in \Azure\AspProviders\TableStorageSessionStateProvider.cs:line 484
System.Web.SessionState.SessionStateModule.GetSessionStateItem()
System.Web.SessionState.SessionStateModule.PollLockedSessionCallback(Object state) INNER_EXCEPTION:
Microsoft.WindowsAzure.StorageClient.Tasks.Task1.get_Result()
Microsoft.WindowsAzure.StorageClient.Tasks.Task1.ExecuteAndWait()
Microsoft.WindowsAzure.StorageClient.TaskImplHelper.ExecuteImplWithRetry[T](Func`2 impl, RetryPolicy policy)
Microsoft.Samples.ServiceHosting.AspProviders.TableStorageSessionStateProvider.ReleaseItemExclusive(TableServiceContext svc, SessionRow session, Object lockId) in \Azure\AspProviders\TableStorageSessionStateProvider.cs:line 603
Microsoft.Samples.ServiceHosting.AspProviders.TableStorageSessionStateProvider.ReleaseItemExclusive(HttpContext context, String id, Object lockId) in \Azure\AspProviders\TableStorageSessionStateProvider.cs:line 480 INNER_EXCEPTION:
System.Data.Services.Client.DataServiceContext.SaveResult.d__1e.MoveNext()
Anyone run into this? The only useful information I've found is this, which I'm hesitant to do:
If you want to bypass the validation, you can open TableStorageSessionStateProvider.cs, find ReleaseItemExclusive, and modify the code from:
svc.UpdateObject(session);
to:
svc.Detach(session);
svc.AttachTo("Sessions", session, "*");
svc.UpdateObject(session);
from here
Thanks!
So I decided to change this:
svc.UpdateObject(session);
svc.SaveChangesWithRetries();
to this:
try
{
svc.UpdateObject(session);
svc.SaveChangesWithRetries();
}
catch
{
svc.Detach(session);
svc.AttachTo("Sessions", session, "*");
svc.UpdateObject(session);
svc.SaveChangesWithRetries();
}
So, I'll see how that works...
I've encountered this problem as well and after some investigation it seems to happen more often when you have more than one instance and you try to make calls in rapid succession in the same session. (e.g. if you had an auto complete box and making ajax calls on each key press)
This occurs because when you try to access the session data, first of all the web server takes out a lock on that session. When the request is complete, it releases the lock. With the table service provider, it updates this lock status by updating a field in the table. What I think is happening is that Instance1 loads the session row, then Instance2 loads the session row, Instance1 saves down the updated lock status and when Instance2 attempts to save the lock status it gets an error because the object isn't in the same state as when it loaded it (the ETag doesn't match any more).
This is why the fix that you found will stop the error from occurring, because by specifying the "*" in the AttachTo, when Instance2 attempts to save the lock it will turn off ETag checking (and over write the changes made by Instance1).
In our situation we have altered the provider so that we can turn off session for certain paths (the ajax call that was giving us our problems didn't need access to session data, neither did the loading of images) which may be an option for you depending on what is causing your problem.
Unfortunately the TableStorageSessionStateProvider is part of the sample projects and so isn't (as far as I'm aware, but I'll happily be told otherwise) officially supported by Microsoft. It does have other issues, like the fact that it doesn't clean up it's session data once a session expires, so you will end up with lots of junk in the session table and blob container that you'll have to clean up some other way.