It seems that MSMQ doesn't use any Database management system to manage messages.
How does MSMQ manage messages?
Does it store the messages in flat file?
I'm trying to implement a messages management system.
MSMQ uses flat files located in %windir%\system32\msmq.
If you want to implement your own queueing, I suggest you take a look at Ayende's blog post on queueing
it stores them as files on the disk.
If you wanna manage them use the System.Messaging API
Related
I'm interested in publishing a file's contents over a kafka channel in realtime (I can do this in python) but I'm wondering what strategy might be effective to prevent sending duplicate data in case my publisher crashes and I need to restart it? Is there anything in kafka that can help with this directly or must I explicitly track the file offset I've published so far?
I suppose another way might be for the publisher to bootstrap the data already published and count the bytes received then file seek and recover?
Are there any existing scripts or apps that handle this already I can perhaps leverage instead?
Instead of publishing it yourself, I strongly recommend using Kafka Connect. In addition of not having to write a custom made code, the connectors could also support the "exactly-once" feature for you.
More details about connectors can be found here: https://www.confluent.io/product/connectors/
You might want to check kafka's Log compaction feature. It does the deduplication for u provided u have unique key for all the duplicate messages.
https://kafka.apache.org/documentation/#compaction
How to get history messages from ActiveMQ like Kafka's consumer group, from beginning?
No. ActiveMQ does not store messages longer than needed for delivery to be complete. If you need to keep track of history - you should implement something for that purpose. ActiveMQ offers mirrored queues so that all queues can be wire-taped at once. But actual storage has to be made by hand.
From own experience - it's easy to implement archiving to a database, but if you want to make some decent search queries, handle replay of older messages etc - it takes some thought
I'm looking at writing a collector to read log messages off an MQ queue. I could have someone write Java or C# code, or I can write it in Powershell. Big question is can I actually? I haven't seen anything that talks about using MQ as an input source to Powershell. It seems like it would fit the model from a "pipeline" perspective...
SupportPac MO74 is a Powershell interface and it's been updated to at least v8.0.0.2.
So, if you want to get the message from MS queue you can use msmq cmdlets. Check this
In the other hand if you want to read it from a websphere queue try MQ Modules
Similarly if you want to try with some other queues I assume they should have an API. You can load the dll thru relection and use it in powershell.
Does anybody have experience with configuring distribution lists (sending a message to one queue, and having that message be forwarded to several other queues) for Websphere MQ v7? I want to configure it on my queue manager, rather than the client having to know all the queues to send the messages to. Also, I would prefer not to use a topic, because I want to be able to manage each queue separately. Is there some configuration file, or some way to use WebSphere MQ Explorer to do this?
Thanks
A program that uses a distribution list doesn't have to "know" the queues it sends to in the sense of hard-coding the names. But it does have to supply the list of queue names. Typically you can place these into a namelist and have the sending program retrieve them there. When the program calls PUT it must also be prepared to parse a structure of return codes rather than a single MQRC.
However, you really should reconsider using a topic. You can create administrative subscriptions for each destination queue. This allows you to send the publications to any local or remote queue that you like. It also have the advantage of being able to add or delete destinations without having to restart - or worse, recompile - the sending application.
You can use WMQ Explorer either to manage a namelist or to manage the topic and administrative subscriptions. The topic/subscriptions method is the only way to do this purely through configuration. To use distribution lists requires a program specifically designed for the purpose.
We are using EntLib Logging Application Block. And also it turned out that we should use msmq for logging because of performance.
Now we are trying to use Msmqdistributor service to log those messages in the queue.
What are pros and cons of Msmqdistributor service of Enterprise Library?
Please share your experience.
The main drawback is going to be the Microsoft Message Queue (MSMQ) itself. MSMQ has been around for awhile and it is a pretty cool tool. It does however lack utilities. Because of the way that data is stored in the queue, most people end up needing to write some helper utilities for debugging and manually manipulating the queue. Some other things to consider:
Queue size - if too many items get put in the queue, and aren't removed in a timely manner the server can stall.
Purpose - MSMQ is designed for multi-step transactions (such as billing), you mention you are going to use it for logging. If the log is just for debugging, Then a DB table or a flat file or sending errors to a bug tracker will serve you better. If you need complicated logging and are using MSMQ to send the information to a different copmuter, then you will find MSMQ more useful.