Can someone point me to some good resources about setting up MSMQ for queue clustering? I'm interested in help with:
Actually setting up the clustered MSMQ nodes, and making it function
Setting up an application to send messages to a clustered queue.
Setting up applications (at least 2) to read from the same clustered queue.
For item 1, here is a nice starting point: "Building MSMQ cluster".
You will find several other links in that article that will guide you further.
Microsoft also has a detailed document: "Deploying Message Queuing (MSMQ) 3.0 in a Server Cluster".
Items 2 and 3 seem to a little bit too much for just one thread.
You might want to use some framework like NServiceBus to work with the queues. NServiceBus comes with example projects that will help you to find a start.
Related
Background
I inherited a Kafka/Zookeeper installation. I have a passing knowledge of those - I know the general architecture, how clients work, about topics, etc., have been involved in programming Java clients etc.
But the installation is somewhat dubious. They are three instances of Kafka and Zookeeper each (in their separate docker containers). Supposedly they should work, but what I am seeing is all processes spout immense amount of log output with loads and loads of (diverse) warnings and errors. I have the impression that some of these seem to be quite normal (or are being self-healed all the time), and am having a very hard time figuring if everything works as intended or not, and set up correctly.
Some of these are - according to Google - related to unclean shutdowns of the brokers; corrupted individual topics and such. As this is a test environment, I can easily delete such files.
I know about some commands which help me check topics etc. (basic stuff, like listing them, displaying their individual configuration etc.).
However...
Question
Is there an online ressource/documentation which can be used as a systematic walkthrough to check whether everything is basically setup OK; for example to clear up these questions:
Do the three Zookeepers and the three Kafka instances correctly talk to each other for high-availability purposes? Do they have a correct "leader" etc.?
Are the servers generally "healthy", i.e., easily able to accept connections etc.?
How are the topics working (what's in there, how many messages, etc.)?
I am aware that one may very quickly dismiss this question as too generic; I am not asking you to solve my problems. I am looking for a ressource to systematically walk through such an installation - it may or may not cover the examples I have given, but it definitely should give a systematic way to find out if things are fundamentally wrong.
Rather than looking solely at logs, you might want to familiarize yourself with JMX metrics and how you can gather them across the cluster.
If you want to actually collect and analyze logs, you'll likely need to separately use something like Elasticsearch.
You won't see "how many messages" in a topic, and you'll need even more monitoring to know if a port is actually open and the Kafka process is running, the disks are filling up, etc.
My point here is that, Kafka needs fed and watered, if you plan to productionalize it, you can't just set up a small cluster and forget about it. Even if you think it's setup correctly at the beginning, increasing the load on it will cause it to fall in a bad state eventually.
For a limited trial for your dev environment to get a full look at your cluster health, Confluent Control Center can assist with that.
To solve the "what's in there" problem, I suggest you setup a Schema Registry, and convince Kafka producers to use it.
This packtpub tutorial/training by Stéphane Maarek is wonderful resource for setting kafka in cluster mode. However he did that in AWS cloud in ubuntu VM.
I have followed the same steps and installed in Vagrant VMs in cent OS. You can find the code here.
The VM has yahoo kafka manager to monitor the kafka internal details. list of broker available, healthy , partitions, leaders etc.,
kafka manager can help you with high level monitoring.
Please provide your comments.
From what I understand, in order to achieve MSMQ load-balancing, one must use a technology such as NLB.
And in order to achieve MSMQ high-availability, one must cluster the related Biztalk Host (and hence the underlying servers have to be in a cluster themselves).
Yet, according to Microsoft Documentation, NLB and FailOver Clustering technologies are not compatible. See this link for reference: http://support.microsoft.com/kb/235305
Can anyone PLEASE explain to me how MSMQ load-balancing and high-availability can be achieved?
thank you in advance,
M
I've edited my original answer because on reflection, I think I was talking nonsense.
I don't believe that it is possible to achieve both load balancing and high availability in a BizTalk transactional scenario. Have a look at the section "Migration considerations for moving from MSMQ/T to MSMQ adapter in BizTalk 2006" on the following site http://blogs.msdn.com/eldarm/
To summarise that post, there are a couple of scenarios:
High Availability (Non-transactional)
You simply have MSMQ on more than one BizTalk server behind NLB
High Availability (Transactional)
For this you need to have a clustered MSMQ host, which means that you can't do any sort of load balancing upon a single queue.
One possible halfway solution is to create two MSMQ adapters, on different clustered hosts, each handling different queues. Doesn't sound too nice to me though.
A key point is understanding the reasons why you would want transactional, clustered behaviour - you need this for ordered delivery and to ensure no duplicates.
In general I wouldn't go to the trouble of load balancing MSMQ - BizTalk itself is load balanced once messages have reached the MessageBox database. While it is true that you will see asymmetric load due to the queue processing happening on one machine, in the overall context of your BizTalk environment this should not be significant.
Again, it is worth remembering that you are clustering MSMQ for reasons beyond simple high availability:
MSMQ adapter receive handler - MSMQ does not support remote
transactional reads; only local
transactional reads are supported. The
MSMQ adapter receive handler must run
in a host instance that is local to
the clustered MSMQ service in order to
complete local transactional reads
with the MSMQ adapter.
That was from the following MSDN page.
I hope this edited answer helps - I don't think it was what you were after, maybe I'm wrong and you'll find a workable solution for NLB and transactional MSMQ, but the more I think about it the more it seems that the two scenarios are not compatible.
A final thought is that you could try posting a similar question on Server Fault - you get a few BizTalk devs on Stack Overflow, including at least two MVPs, but at least where I work this is that sort of question I'd be passing on to my networking team.
I'm trying to decide if MSMQ is the right tool for communication between our application and a third party web service we are currently communicating with directly. We're looking to uncouple this such that if the service goes down, life could still go on as normal.
I can't find anything outside the usual MS fluff about it being the greatest thing and solving all your problems etc etc. It would be really useful if I could find some information that was somewhere between marketing fluff and API - like an architecture diagram of the components and how they integrate with each other, and more importantly how I integrate with them.
It's probably that I'm just looking for the information in the wrong places, so if someone could point me in the right direction, I'd appreciate it.
A typical MSMQ architecture would be composed of 3 parts...
Message Queue - This would be on one of your servers. You would have to install the MSMQ bits and create your queue.
Client - Your client would insert messages into the queue. I'm assuming you're using .NET. If so, most of what you want is going to be located in the System.Messaging namespace.
Windows Service - This would also run on a server, probably the same server as your queue. Its job would be to watch the queue, process messages as they come in, handle making sure the external service is available, and probably do some logging.
Here's an article that should go into a little more detail and give you some code samples.
MSMQ is a implementation of a message queue as are websphere mq and a bunch of other systems. When looking into the concepts and high level architecture I would suggest reading up on message queue's and how they are applied in disconnected scenario's. I can highly recommend Patterns of Enterprise Application Architecture. For specific examples on msmq check out Pro MSMQ: Microsoft Message Queue Programming it doesn't contain allot of special information but it does group it allot better then most resources available on the internet. This Hello World with MSMQ article would give you a nice overview of what it entails and it's easily executed on a development system.
If you are calling a remote web service from your application, it makes sense to use a queue to decouple your application processing from the remote system. By abstracting the communicating through messaging and having a gateway service that is responsible for communication to the web service, you isolate your application from the latency of the web service and build fault tolerance into the design by reducing the request/response usage inside your application (since messaging is by default asynchronous - you deal with it up front).
There are frameworks for .NET that can make this much easier (such as MassTransit or NServiceBus).
You can also check out SOA Patterns (by Arnon Rotem-Gal-Oz, Manning Press, in MEAP) and Enterprise Integration Patterns (Hohpe,Woolf), the latter of which is an essential read for anyone building a message-based system.
I'm working on a messaging/notification system for our products. Basic requirements are:
Fire and forget
Persistent set of messages, possibly updating, to stay there until the sender says to remove them
The libraries will be written in C#. Spring.NET just released a milestone build with lots of nice messaging abstraction, which is great - I plan on using it extensively. My basic question comes down to the question of message brokers. My architecture will look something like app -> message broker queue -> server app that listens, dispatches all messages to where they need to go, and handles the life cycle of those long-lived messages -> message broker queue or topic -> listening apps.
Finally, the question: Which message broker should I use? I am biased towards ActiveMQ - We used it on our last project and loved it. I can't really think of a single strike against it, except that it's Java, and will require java to be installed on a server somewhere, and that might be a hard sell to some of the people that will be using this service. The other option I've been looking at is MSMQ. I am biased against it for some unknown reason, and it also doesn't seem to have great multicast support.
Has anyone used MSMQ for something like this? Any pros or cons, stuff that might sway the vote one way or the other?
One last thing, we are using .NET 2.0.
I'm kinda biased as I work on ActiveMQ but pretty much all of benefits listed for MSMQ above also apply to ActiveMQ really.
Some more benefits of ActiveMQ include
great support for cross language client access and multi protocol support
excellent support for enterprise integration patterns
a ton of advanced features like exclusive queues and message groups
The main downside you mention is that the ActiveMQ broker is written in Java; but you can run it on IKVM as a .net assembly if you really want - or run it as a windows service, or compile it to a DLL/EXE via GCJ. MSMQ may or may not be written in .NET - but it doesn't really matter much how its implemented right?
Irrespective of whether you choose MSMQ or ActiveMQ I'd recommend at least considering using the NMS API which as you say is integrated great into Spring.NET. There is an MSMQ implementation of this API as well as implementations for TibCo, ActiveMQ and STOMP which will support any other JMS provider via StompConnect.
So by choosing NMS as your API you will avoid lockin to any proprietary technology - and you can then easily switch messaging providers at any point in time; rather than locking your code all into a proprietary API
Pros for MSMQ.
It is built into Windows
It supports transactions, it also supports queues with no transactions
It is really easy to setup
AD Integration
It is fast, but you would need to compare ActiveMQ and MSMQ for your traffic to know which is faster.
.NET supports it nativity
Supports fire and forget
You can peek at the queue, if you have readers that just look. not sure if you can edit a message in the queue.
Cons:
4MB message size limit
2GB Queue size limit
Queue items are held on disk
Not a mainstream MS product, docs are a bit iffy, or were it has been a few years since I used it.
Here is a good blog for MSMQ
Take a look at zeromq. It's one of the fastest message queues around.
I suggest you have a look at TIBCO Enterprise Messaging Service - EMS, which is a high performance messaging product that supports multicasting, routing, supports JMS specification and provides enterprise wide features including your requirements suchas fire-forget and message persistence using file/database using shared state.
As a reference, FEDEX runs on TIBCO EMS
as its messaging infrastructure.
http://www.tibco.com/software/messaging/enterprise_messaging_service/default.jsp
There are lot other references if i provide, you'd really be surprised.
There are so many options in that arena...
Free: MantaRay a peer to peer fully JMS compliant system. The interesting part of Mantaray is that you only need to define where the message goes and MantaRay routes it anyways that will get your message to it's detination - so it is more resistant to failures of individual nodes in your messaging fabric.
Paid: At my day job I administer an IBM WebSphere MQ messaging system with several hundred nodes and have found it to be very good. We also recently purchased Tibco EMS and it seems that it will be pretty nice to use as well.
We are using EntLib Logging Application Block. And also it turned out that we should use msmq for logging because of performance.
Now we are trying to use Msmqdistributor service to log those messages in the queue.
What are pros and cons of Msmqdistributor service of Enterprise Library?
Please share your experience.
The main drawback is going to be the Microsoft Message Queue (MSMQ) itself. MSMQ has been around for awhile and it is a pretty cool tool. It does however lack utilities. Because of the way that data is stored in the queue, most people end up needing to write some helper utilities for debugging and manually manipulating the queue. Some other things to consider:
Queue size - if too many items get put in the queue, and aren't removed in a timely manner the server can stall.
Purpose - MSMQ is designed for multi-step transactions (such as billing), you mention you are going to use it for logging. If the log is just for debugging, Then a DB table or a flat file or sending errors to a bug tracker will serve you better. If you need complicated logging and are using MSMQ to send the information to a different copmuter, then you will find MSMQ more useful.