I'm wondering if there are any guidelines for which machine MSMQ should be installed on, in a situation in which one machine (A) will be sending messages, and another machine (B) will be receiving them. Are there any factors that would indicate either that it's better to install MSMQ on A (in which case the sender writes messages locally, but the receiver reads them remotely), or to install on B (in which case it's the other way round?)
(If it makes any difference, in our case the sender is a website, and the receiver is an app server machine that sits behind the website, and to which the website sends messages. In the absence of any other advice, I'd have been inclined to install MSMQ on the app server, on the vague principle of keeping as little software as possible on the web server on security grounds, since that server is in the first line of any attack. To complicate matters, the website sits behind a load balancer - so 'machine A' is actually two identical machines, likewise for machine 'B').
MSMQ has to run on all machines that are sending or receiving messages.
It doesn't matter if you "send local/read remote" or "send remote/read local".
Related
I have a basic question about mail servers. Usually any general server (not necessarily mail servers) handle all kinds of related requests i.e. both direction interaction with client (like sending and receiving msg to and from client) but in case of mail servers, there are 2 different servers -- one for outgoing messages called outgoing server following SMTP protocol and another for incoming messages called incoming server following POP3/IMAP protocol. Why so? . For that matter couldn't the 2 protocols be accommodated in one single protocol to handle both direction message flow. Also, typically are these 2 servers hosted on same machine in a general business?
Because the protocols are old. In the old Unix tradition, and ignoring UUCP for now:
SMTP came first, to transfer mail between sites, and mail was read locally on the server/network using a text mode client when logged in. There was no need to fetch mail, you used the 'mail' command, and it accessed your local mail spool (a file containing your mail, sitting on the file system, that the SMTP server appended to).
Later on, came the development that people wanted to read mail on their own hosts so a protocol was invented to serve that. The POP server would read that same spool file, and allow you to download all your messages to your intermittently connected client computer. SMTP was reused for sending mail because it already existed and was easily adapted for that purpose.
These days, there are usually three servers of note: SMTP submission server, SMTP transmission server, and IMAP. The submission server is where end users of the service submit their e-mail to be forwarded onto the final host, for example the Google server a Gmail user submits their email to (on port 465 or port 587, usually, with authentication). The transmission server is responsible for delivering and receiving email between sites (eg. when Yahoo and Gmail exchange mail for their customers with each other, on port 25). And IMAP is used by an end user to fetch their email.
These three services, on large sites, are generally served by separate servers on separate hosts. On very large services, like Gmail, they are separate pools of servers.
On a small business host, they were often just one machine.
There are newer and more integrated protocols. For example, both EAS and EWS have mail fetch and submission contained in the same protocol.
Personally, I regret that almost nobody uses UUCP anymore :)
The separation of shipping, management and collection of mail has a historical, philosophical and design basis.
This is in line with the spirit of the Unix world: "do one thing but do it right" and "make everything as simple as possible".
How complex the issue of e-mail is and how much technology it covers, read here:
https://en.wikipedia.org/wiki/Email
and then see the relevant RFCs in the footnotes for details.
It is difficult to implement a monolithic program that takes into account all the nuances of e-mail and at the same time
keeping it simple (Simple in SMTP). Such a monolith would be a nightmare to develop, maintain and administer,
and the post office has changed many times since the 1970s.
As for the second question, there is no reason to use the same physical machine, but there are also no contraindications other than the amount of resources available.
I'm trying to build instant messaging app. Clients will not only send messages but also often send audios. And I've decided to use websocket connection to communicate with clients. It is fast and allows to send binary data.
The main idea is to receive from client1 message and notify about it client2. But here's the thing. My app will be running on GAE. And what if client1's socket is opened on server1 and client2's is opened on server2. This servers don't know about each others clients.
I have one idea how to solve it, but I am sure it is shitty way. I am going to use some sort of communication between servers(for example JMS or open another websocket connection between servers, doesn't matter right now).
But it surely will lead to a disaster. I can't even imagine how often those servers will speak to each other. For each message server1 should notify server2, server2 should notify client2. But things become even worse when serverN comes into play.
Another way I see this to work is Firebase. But it restricts message size to 4KB. So I can't send audios via it. As a solution I can notify client about new audio and he goes to my server for it.
Hope I clearly explained the problem. Does anyone know how to solve it? Or maybe there are another ways to build such apps?
If you are building a messaging cluster and expect communicating clients to connect to different instances of the server then server-server communication is inevitable. Usually it's not a problem though.
First, if you don't use any load balancing your clients will connect to the same server 50% of time on average (in case of 2 servers).
Second, intra-datacenter links are fast and free in all known public clouds.
Third, you can often do something smart on the frontend to make sure two likely to communicate clients connect to the same server. For instance direct all clients from the same country to the same server using DNS load balancing.
The second part of the question is about passing large media files. It's a common best practice to send it out of band - store on the server and only pass the reference to it. Like someone suggested in the comment, save the audio on the server and just send a message like "audio is available, fetch it from here ...". You don't need to poll the server for that. Just fetch it once when the receiving client requests it.
In general, it seems like you are trying to reinvent the wheel. Just use something off the shelf.
Let all client get connected to multiple servers and each server keeps this metadata
A centralized system like zookeeper stores active servers details
When a client c1 sends a message to client c2:
the message is received by a server (say s1, we can add a load balancer to distribute incoming requests)
s1 will broadcast this information to all other servers to get which server the client c2 is connected to OR a better approach to use consistent hashing which decides which server the client can connect to & in this approach message broadcast is not required
the corresponding server responses to server s1 (say s2)
now s1 sends the message m to s2 and server s2 to client c2
Cons of the above approach:
Each server will have a connection with the n-1 servers, creating a mesh topology
Centralized system (zookeeper) becomes a single point of failures (which is solvable)
Apps like Whatsapp, G-Talk uses XMPP and TCP/IP.
I'm having some trouble with this, and have a hard time explaining the problem in words, but I will do the best I can.
I would like to send a simple message to a computer from a different computer. Nothing fancy. I have done some research, and a lot of sources say to create a script that accesses the built-in MSG.EXE function in Windows (I am currently on Windows 10) however, this only seems to work on machines connected to LAN.
From what I have gathered, I would need to create a website that handles the requests: get message from Computer A and send it to Computer B. Could I do this (In Powershell or VBS) without having to make a site specifically for it?
What I want:
I want Computer A to send a plain-text message to Computer B. Preferably, when the message is successfully sent a message box will appear that says "From: (whomever) and the message. Using MSG.EXE did what I wanted it to do, however it only worked with LAN-connected devices. Could I make this work with static IP? If so, how?
Additional Information:
The two computers are not on a LAN, but are connected to the Internet.
Computer A (the sender) and Computer B (the receiver) each have a static IP address, however that can obviously change.
Ports are available in each computer.
I have a Raspberry Pi that could act as a server "middle man" should that be required.
To be very clear: There are a lot of posts that talk about sending messages over LAN. I am NOT trying to send a message over LAN. Any help would be appreciated.
You're being quite vague in why you want it and what you want it to do; the main problem is that home internet connections are built for things to connect out, and nothing to connect in.
Fixes:
Port forwarding where you say "incoming connections go to this or that computer.
Building a service where both sides "connect out", i.e. you have a website hosted somewhere out on the internet. One computer connects out to send the message, the other connects out to check for messages.
Outside that, the next problem is knowing where on the internet to connect to.
Fixes:
Hard code the end points. Static IPs are good for this because the definition of a static IP is that it's an address which doesn't change. Mysteriously, you have static IPs which can change and you consider that 'obvious'. (??)
DNS where you register a domain, and the computers look at a domain name for where to connect. You update DNS with the current IP.
DNS where you use a free DNS service on someone else's name and your service listener updates that as its IP changes.
And after that, most people's computers/laptops/phones are switched off or in low power sleep mode most of the time, and their internet connections are comparatively unreliable, so any random message probably won't be answered.
Fixes:
Use a serious website service
Run an always-on computer to be that service
Deal with it being offline and messages being flakey
Use a store-and-forward system like email, which queues and retries, like e-mail and instant messengers do
get message from Computer A and send it to Computer B. Could I do this (In Powershell or VBS) without having to make a site specifically for it?
Yes. Quick answer: pick a port, configure site B's modem/router to forward the port to computer B. Open the port on computer B's firewall. Write some kind of TCP socket listener on Computer B, and the sender on Computer A, connecting to the public IP address of site B.
Long answer: then enjoy writing your own messaging protocol, error handling, message authentication, etc.
Alternative: do the same but going to your rPi. Have it listening. Not in powershell (sadly) or vbscript (luckily), and have it serve the messages up by HTTP/JSON, both computers invoke-webrequest or invoke-restmethod and post/get them with regular polling.
Windows Server 2012
MSMQ 6 Workgroup Mode
We've had issues trying to recover MSMQ messages that were sent to the transaction dead letter queue. We've tried moving them to the outbound queue, the message seems to send fine (even the Event Log says so) however it never gets to the destination queue.
After trial and error we've figured out how to get them to another queue on the same server but not to the destination queue on a remote server. We don't want to lose anymore messages. Does anyone have any suggestion on how we can deliver these messages?
Thank you,
David
As I understood your question, it's a one time problem with some number of messages you already have in MSMQ, and not general connectivity issue between machines? If so, you should be able to solve it with some MSMQ management tool. Disclaimer: I'm the author of one such tool - QueueExplorer. I don't know what other tools can do, but with QueueExplorer you can copy/paste or drag/drop messages to another machine opened in separate tab/window. In order to do that QueueExplorer has to perform MSMQ Send operation, so messages will have to pass through MSMQ between these two machines.
So if there's still that issue that prevented original delivery you'll still be stuck. In that case you can save all messages to a file, transfer it to another machine through file system and load it there to whichever queue they should go. This is obviusly just a manual workaround for one time situation. Btw. this could be done in QueueExplore's trial mode.
If however problem is with connectivity and messages always end up in dead letter queue, it's better to check them from Computer Management. It's one area where it's better than our tool - you can turn on "Class" column and see reason why messages couldn't be delivered. For instance if you see "The time-to-be-received has elapsed" you'll know what's the problem.
I need to work with MSMQ (Microsoft Message Queuing). What is it, what is it for, how does it work? How is it different from web services?
With all due respect to #Juan's answer, both are ways of exchanging data between two disconnected processes, i.e. interprocess communication channels (IPC). Message queues are asynchronous, while webservices are synchronous. They use different protocols and back-end services to do this so they are completely different in implementation, but similar in purpose.
You would want to use message queues when there is a possibility that the other communicating process may not be available, yet you still want to have the message sent at the time of the client's choosing. Delivery will occur the when process on the other end wakes up and receives notification of the message's arrival.
As its name states, it's just a queue manager.
You can Send objects (serialized) to the queue where they will stay until you Receive them.
It's normally used to send messages or objects between applications in a decoupled way
It has nothing to do with webservices, they are two different things
Info on MSMQ:
https://msdn.microsoft.com/en-us/library/ms711472(v=vs.85).aspx
Info on WebServices:
http://msdn.microsoft.com/en-us/library/ms972326.aspx
Transactional Queue Management 101
A transactional queue is a middleware system that asynchronously routes messages of one sort of another between hosts that may or may not be connected at any given time. This means that it must also be capable of persisting the message somewhere. Examples of such systems are MSMQ and IBM MQ
A Transactional Queue can also participate in a distributed transaction, and a rollback can trigger the disposal of messages. This means that a message is guaranteed to be delivered with at-most-once semantics or guaranteed delivery if not rolled back. The message won't be delivered if:
Host A posts the message but Host B
is not connected
Something (possibly but not
necessarily initiated from Host A)
rolls back the transaction
B connects after the transaction is
rolled back
In this case B will never be aware the message even existed unless informed through some other medium. If the transaction was rolled back, this probably doesn't matter. If B connects and collects the message before the transaction is rolled back, the rollback will also reverse the effects of the message on B.
Note that A can post the message to the queue with the guarantee of at-most-once delivery. If the transaction is committed Host A can assume that the message has been delivered by the reliable transport medium. If the transaction is rolled back, Host A can assume that any effects of the message have been reversed.
Web Services
A web service is remote procedure call or other service (e.g. RESTFul API's) published by a (typically) HTTP Server. It is a synchronous request/response protocol and has no guarantee of delivery built into the protocol. It is up to the client to validate that the service has been correctly run. Typically this will be through a reply to the request or timeout of the call.
In the latter case, web services do not guarantee at-most-once semantics. The server can complete the service and fail to deliver a response (possibly through something outside the server going wrong). The application must be able to deal with this situation.
IIRC, RESTFul services should be idempotent (the same state is achieved after any number of invocations of the same service), which is a strategy for dealing with this lack of guaranteed notification of success/failure in web service architectures. The idea is that conceptually one writes state rather than invoking a service, so one can write any number of times. This means that a lack of feedback about success can be tolerated by the application as it can re-try the posting until it gets a 'success' message from the server.
Note that you can use Windows Communication Foundation (WCF) as an abstraction layer above MSMQ. This gives you the feel of working with a service - with only one-way operations.
For more information, see:
http://msdn.microsoft.com/en-us/library/ms789048.aspx
Actually there is no relation between MSMQ and WebService.
Using MSMQ for interprocess communication (you can use also sockets, windows messaging, mapped memory).
it is a windows service that responsible for keeping messages till someone dequeue them.
you can say it is more reliable than sockets as messages are stored on a harddisk but it is slower than other IPC techniques.
You can use MSMQ in dotnet with small lines of code, Just Declare your MessageQueue object and call Receive and Send methods.
The Message itself can be normal string or binary data.
As everyone has explained MSMQ is used as a queue for messages. Messages can be wrapper for actual data, object and anything that you can serialize and send across the wire. MSMQ has it's own limitations. MSMQ 1.0 and MSMQ 2.0 had a 4MB message limit. This restriction was lifted off with MSMQ 3.0. Message oriented Middleware (MOM) is a concept that heavily depends on Messaging. Enterprise Service Bus foundation is built on Messaging. All these new technologies, depend on Messaging for asynchronous data delivery with reliability.
MSMQ stands for Microsoft Messaging Queue.
It is simply a queue that stores messages formatted so that it can pass to DB (may on same machine or on Server). There are different types of queues over there which categorizes the messages among themselves.
If there is some problem/error inside message or invalid message is passed, it automatically goes to Dead queue which denotes that it is not to be processed further. But before passing a message to dead queue it will retry until a max count and till it is not processed. Then it will be sent to the Dead queue.
It is generally used for sending log message from client machine to server or DB so that if there is any issue happens on client machine then developer or support team can go through log to solve problem.
MSMQ is also a service provided by Microsoft to Get records of Log files.
You get Better Idea from this blog http://msdn.microsoft.com/en-us/library/ms711472(v=vs.85).aspx.