MQ - Peaking Queues in .NET - msmq

I am relatively new to MS-MQ Development and I was wondering if any of you guru's out there know an answer to the following:
In the concept of queues, there is something called "peaking", whereby; you can look at the message on the queue BUT NOT take it off the queue. This is different to performing a GET call.
Does anyone know how to do this?
Thanks in Advance.

Well, it's not much different than recieving a message from a queue.
var queue = new MessageQueue([path to queue]);
var msg = queue.Peek();
It is also possible to iterate over the messages in the queue by using
var msg = queue.Peek(TimeSpan.FromSeconds(10), PeekAction.Next);
and you can also peek by different Id's
PeekByCorrelationId
PeekByLookupId
PeekById
Check out the examples in the documentation on MSDN for each of these methods as they can get you started quite easily.
http://msdn.microsoft.com/en-us/library/829zyck7.aspx

Related

Syncing of buffer-transmission with ESP32, I2S MEMS-mic and SD-card (FreeRTOS, PlatformIO, ESP-PROG)

i know this forum dislikes "open" questions like this, nevertheless i'd like somebody to help untie the knot in my head, much appreciated.
The goal is simple:
read a stereo 32bit 44100 S/s I2S signal from 2 adafruit sph0645 mics
create a wav-header and store the data onto an SD-card
I've been at this for a few days now and i know that this will be much more complicated than i originally thought. Main reason: signal quality. Like most tutorials on this subject the simplest "hello world" for these mics is a looped polling for I2S-samples. Poll, fill buffer, output via serial or write to SD-card. This returns a choppy, noisy, sped up version of RL-audio. The filling of the internal DMA-buffers can be seen as constant, but the rest is mostly chaos, so
how to i sync these DMA-buffers with the rest of my code?
From experience with the STM32 HAL i'd imagine some register which can be set to throw an interrupt whenever a buffer is full, or an event which can be sent between tasks via queues. Examples on this subject either poll in a main loop with mono an abysmal sample-rate and bit depth or use pages of overkill code and never adress what it does, "just copy and it works", not good. Does the ESP32-Arduino framework provide some way to to this properly? The espressif-documentation isn't something to look forward to, since some of their I2S interface functions don't even work (if you are researching this topic as well, you too might have noticed that i2s_read only returns zeros). Just a hint into the right direction would help, i'm writing my own code anyway. Interrupts? Events? Timers? Polling for full buffers? Only you might know.
have a good one, thx
Thanks to https://github.com/atomic14/ i now have an answer for a syncing-method which works very well. This method has been tried by https://esp32.com/viewtopic.php?t=12546 who also didn't fully understand what was going on: the espressif i2s-interface offers a flag stored in an event which is triggererd every time one of the specified dma-buffers has received a full set of data, ergo, is full. It looks like this:
while(<your condition>){
i2s_event_t evt;
if (xQueueReceive(<your queue>, &evt, portMAX_DELAY) == pdPASS){
if (evt.type == I2S_EVENT_RX_DONE){
size_t bytesRead = 0;
do{
//read data via i2s_read or i2s_read_bytes
} while (bytesRead > 0);
No data is stored in this queue, but rather a flag which can then be used to synchronize dma-filling and further buffering/calculating/sending the read data.
HOWEVER this only works if you install the i2s driver in a specific setup. Instead of using
i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL);
in your setup, you can activate the "affinity" for events by passing a queue-handle and a lenght:
i2s_driver_install(I2S_NUM_0, &i2s_config, 4, &<your queue>);
hope this helps getting started, it sure did help me.

Proper way to set or update JBPM6 Process Variables

I am trying to set the process variable but getting below error
java.lang.IllegalStateException: Process instance 10[SampleBusinessProcess] is disconnected.
at org.jbpm.process.instance.impl.ProcessInstanceImpl.getProcess(ProcessInstanceImpl.java:100)
at org.jbpm.workflow.instance.impl.WorkflowProcessInstanceImpl.setVariable(WorkflowProcessInstanceImpl.java:343)
Here is my code -
ProcessInstance processInstance = runtimeEngine.getKieSession().getProcessInstance(processInstanceId);
((WorkflowProcessInstanceImpl) processInstance).setVariable("myvariable", "myvalue");
Please help me with the best way to set the process variable.
Thanks.
I would recommend to use the jbpm-services for interacting with the engine, this offers an operation to set a variable: https://github.com/kiegroup/jbpm/blob/7.7.0.Final/jbpm-services/jbpm-services-api/src/main/java/org/jbpm/services/api/ProcessService.java#L223
Alternatively, if you make sure that your code is executed in the same transaction (by for example starting and committing the transaction yourself, around the code you currently have), you will be able to set the variable this way, as the process instance will only disconnect as soon as the transaction commits.
I can't comment on the previous answer so I'm posting this answer.
The quick and dirty way I found (inspired from previous answer) is:
public void setProcessInstanceVariables(long processInstanceId, Map<String, Object> variables) {
kieSession.execute(new SetProcessInstanceVariablesCommand(processInstanceId, variables));
}
Thanks a lot Kris Verlaenen!

How to get application process to wait until the socket has data to read using libevent bufferevents?

I'm working with libevent for the first time and have been having an issue trying to get my application to not run until the read callback is called. I am using bufferevents as well. Essentially I am doing is trying to avoid the sleep in my main application loop and instead have the OS wake up the process (via libevent) when there is data to be read off the socket. Anyone know how to do this? I found in an alpha build of libevent that you can set a base event loop to be EVLOOP_NO_EXIT_ON_EMPTY, but from looking at the libevent code that will just use up my whole proc I believe. I also read on this question that it is a bad idea to set a socket to blocking on windows which is why I haven't done that as a solution either. I will mark this with libuv and libev too since they are similar and might contribute to my solution.
you have to use the following api, some of the API may be oudated you can search google for new one.
struct event_base *base ;
struct event g_eve
base = event_init();
//after binding the socket register your socket for read event using below api
event_set(&g_eve, SockFd, EV_READ | EV_PERSIST, CallbackFunctin, &g_eve);
event_add(&g_eve, NULL);
event_base_dispatch(base);

Debug missing messages in akka

I have the following architecture at the moment:
Load(Play app with basic interface for load tests) -> Gateway(Spray application with REST interface for incoming messages) -> Processor(akka app that works with MongoDB) -> MongoDB
Everything works fine as long as number of messages I am pushing through is low. However when I try to push 10000 events, that will eventully end up at MongoDB as documents, it stops at random places, for example on 742 message or 982 message and does nothing after.
What would be the best way to debug such situations? On the load side I am just pushing hard into the REST service:
for (i ← 0 until users) workerRouter ! Load(server, i)
and then in the workerRouter
WS.url(server + "/track/message").post(Json.toJson(newUser)).map { response =>
println(response.body)
true
}
On the spray side:
pathPrefix("track") {
path("message") {
post {
entity(as[TrackObj]) { msg =>
processors ! msg
complete("")
}
}
}
}
On the processor side it's just basically an insert into a collection. Any suggestions on where to start from?
Update:
I tried to move the logic of creating messages to the Gatewat, did a cycle of 1 to 10000 and it works just fine. However if spray and play are involed in a pipeline it interrupts and random places. Any suggestions on how to debug in this case?
In a distributed and parallel environment it is next to impossible to create a system that work reliably. Whatever debugging method you use it will only allow you to find a few bugs that will happen during the debug session.
Once our team spent 3 months(!) while tuning an application for a robust 24/7 working. And still there were bugs. Then we applied a method of Model checking (Spin). Within a couple of weeks we implemented a model that allowed us to get a robust application. However, model checking requires a bit different way of thinking and it can be difficult to start.
I moved the load test app to Spray framework and now it works like a charm. So I suppose the problem was somewhere in the way that I used WS API in Play framework:
WS.url(server + "/track/message").post(Json.toJson(newUser)).map { response =>
println(response.body)
true
}
The problem is resovled but not solved, won't work on a solution based on Play.

MessageReadPropertyFilter getting reset when using MSMQ

Strange one. We have a multi-threaded app which pulls messages off a MSMQ Queue and then subsequently performs actions based on the messages. All of this is done using DTC.
Sometimes, for some reason I can't describe, we get message read errors when pulling Messages off the queue.
The code that is being used in the app:
Message[] allMessagesOnQueue = this.messageQueue.GetAllMessages();
foreach (Message currentMessage in allMessagesOnQueue)
{
if ((currentMessage.Body is IAMessageIDealWith))
{
// do something;
}
}
When the currentMessage.Body is accessed, at times it throws an exception:
System.InvalidOperationException: Property Body was not retrieved when receiving the message. Ensure that the PropertyFilter is set correctly.
Now - this only happens some of the time - and it appears as though the MessageReadPropertyFilter on the queue has the Body property set to false.
As to how it gets like this is a bit of a mystery. The Body property is one of the defaults and we absolutley never explicitly set it to false.
Has anyone else seen this kind of behaivour or has some idea why this value is getting set to be false?
As alluded to earlier, you could explicitly set the boolean values on the System.Messaging.MessagePropertyFilter object that is accessible on your messageQueue object via the MessageReadPropertyFilter property.
If you want all data to be extracted from a message when received or peaked, use:
this.messageQueue.MessageReadPropertyFilter.SetAll(); // add this line
Message[] allMessagesOnQueue = this.messageQueue.GetAllMessages();
// ...
That may hurt performance of reading many messages, so if you want just a few additional properties, create a new MessagePropertyFilter with custom flags:
// Specify to retrieve selected properties.
MessagePropertyFilter filter= new MessagePropertyFilter();
filter.ClearAll();
filter.Body = true;
filter.Priority = true;
this.messageQueue.MessageReadPropertyFilter = filter;
Message[] allMessagesOnQueue = this.messageQueue.GetAllMessages();
// ...
You can also set it back to default using:
this.messageQueue.MessageReadPropertyFilter.SetDefaults();
More info here: http://msdn.microsoft.com/en-us/library/system.messaging.messagequeue.messagereadpropertyfilter.aspx
I have seen it as well, and have tried initializing it with the properties I'm accessing explicitly set, and not setting them anywhere else. I periodically get the same error you are getting, my app is multi-threaded as well, what I ended up doing is trapping that error and reconnecting to MSMQ when I get it.
Sometimes, for some reason I can't describe, we get message read errors when pulling Messages off the queue.
Are you using the same MessageQueue instance from more than one thread, without locking? In that case, you will encounter spurious changes in MessageReadPropertyFilter - at least I did, when I tried.
Why? Because
Only the GetAllMessages method is thread safe.
What can you do? Either
wrap a lock (_messageQueue) around all access to your messageQueue OR
create multiple MessageQueue instances, one per thread