I'm using Rebus (https://github.com/rebus-org (v.0.83)) and until now it's been all local to a single machine. Now I need to use a remote queue from my website to an app server. It's not abundantly clear to me how to set this up with Rebus. A few questions
I guess I need MSMQ on both machines (web & app) correct? I've configured the web site to UseMsmqInOneWayClientMode;
or is there a way to specify to send it over http?
My configuration looks like this more or less. I'm guessing that inputQUeue needs to point to the local machine not the remote one right?
<rebus inputQueue="mywebqueue" errorQueue="MyErrorQueue#mymachine" workers="1" maxRetries="5">
<endpoints>
<add messages="MyLibrary.CreateMessage, MyLibrary" endpoint="MyQueue#mymachine"/>
</endpoints>
</rebus>
Help would really be appreciated.
Yes. All machines that need to use MSMQ somehow, need to have MSMQ installed. Even as a one-way client, like your webserver, because MSMQ achieves its high availability by providing outgoing queues when you send to remote machines.
I think so. I've never used MSMQ beyond its basic reliable messaging capabilities. Google around, I bet you can find something ;)
Your configuration looks right. And yes - input queues are always local, whereas queues you send to (error queue and all endpoints specified in the endpoint mappings) can be remote too.
Related
I have an Ubuntu VM installed on a client's VMware system. Recently, the client's IT informed us that his firewall has been detecting consistent potential port scans to our VM's internal IP address (coming from 87.238.57.227). He asked if this was part of a known package update process on our VM.
He sent us a firewall output where we can see several instances of the port scan, but there are also instances of our Ubuntu VM trying to communicate back to the external server on port 37258 (this is dropped by the firewall).
Based on a google lookup, the hostname of the external IP address is "feris.postgresql.org", with the ASN pointing to a European company called Redpill-Linpro. As far as I can tell, they offer IT consulting services, specializing in open source software (like PostgreSQL, which is installed on our VM). I have never heard of them before though and have no idea why our VM would be communicating with them or vice-versa. I'm also not sure if I'm interpreting the IP lookup information correctly: https://ipinfo.io/87.238.57.227
I'm looking for a way to confirm or disprove that this is just our VM pinging for a standard postgres update. If that's the case I'd like to restrict this behaviour. We would prefer to do these types of updates manually and limit the communication outside of the VM to what is strictly necessary for the functionality of our application.
Update
I sent an email to Redpill's abuse account. They responded quickly saying that the server should not be port scanning anyone and if it appears that way, something is wrong.
The server is part of a cluster of machines that serves apt.postgresql.org among other postgres download sites. I don't think we have anything like ansible or puppet installed that would automatically check for updates but I will look into that to make sure. I'm wondering if Ubuntu reaching out to update the MOTD with the number of available packages would explain why our VM is trying to reach out to the external postgres server?
The abuse rep said in any case there should only be outgoing connections from the VM, not incoming. He asked for some additional info so I will keep communicating with him and try to update this post accordingly
My communication with the client's IT dropped off so I did not get a definitive answer on this, but I'll provide some new details:
I reached out to the abuse email for Redpill-Linpro. He got back to me and confirmed the server corresponding to the detected IP address is part of a cluster that hosts postgres download sites, including apt.postgresql.org. He was surprised to learn we had detected a port scan from their server and seems eager to figure out why that is happening.
He asked if the client IT could pass along some necessary info for them to set up tracking on that server. But the client IT never got back to me. I think he was satisfied that it wasn't malicious and stopped pursuing it.
Here's one of the messages the abuse rep sent me that may be relevant:
That does look a lot like the tcp to the apt download server yes. It's
strange that your firewall reports that many incoming connections, but
they could be fallout from some connection tracking that's not
operating as intended. The timing appears to be matching up more or
less perfectly. And there should definitely not be any ping-back
connections from it.
Since you appear to be using the http version of the server (and not https) bringing the data in cleartext, they should be able to just
dump the TCP connection contents and verify exactly what it does. But
I bet they are going to see a number of http requests initiated by the
apt client that is checking for updates.
We've developed a MassTransit based demo which is working well as long as all processes run on the same server.
However, as sonn as my receiver wants to subscribe himself at another machine it hangs for a while and afterwards we receive the following exception:
"System.InvalidOperationException: Timeout waiting for subscription service to respond."
Checked already: Firewall rules for MSMQ (inbound and outbound), network, etc.
What could have gone wrong?
Subscription queues on the other machine are private? Is this a problem?
Do we have to change the address format in some special way? msmq://computerName/queueName not ok for remote connections?
Looks like we have forgotten some tiny thing, as nobody else had this problem before...
The most likely thing is MT assumes somethings about remote queues - that they are transactional. Local queues can be queried to discover that, but remote queues you cannot. I would add ?tx=false to the end of your remote queue URI if you aren't using transactional queues for the subscription service.
Next, double check the outgoing queues on the local machine. Are the message stuck there or did they disappear? If you are using transactional queues, can the machines enroll in a DTC transaction together?
Answering your question, all queues are private. This is not a problem, they are still remotely addressable.
I hope this helps get you further. After that, I would consider joining the mailing list and posting your questions there: https://groups.google.com/forum/?fromgroups#!forum/masstransit-discuss
For reference: the problem has been a wrong URL in the receiver queue, the receiver queue always resides on the local system of course. Sorry for any inconvenience.
It seems that both service and server refer to some web based application. But is there any precise definition of the two terms?
A server offers one or more services. Server is also a more technical term, whereas service is more a term off the problem domain.
You also need to distinguish between:
Server as hardware (see post from Dan D)
Server as software (eg. Apache HTTP server)
You can find more elaborate definiton on Wikipedia:
Service
Server
This is regardless of client-server or P2P models.
A server provides services to one or more clients, and a server(hardware) is a computer. A server(hardware) can be anything from a home computer to a big server-rack with a lot of processor power.
From the view of a computer, a server(software) is just a set of services which is available to clients on the network.
Some well known services are web-server, mail-server. ftp-server. notice they are called xxx-server because such programs consist of a client and server part. The postfix is mainly to distinguish whether we are talking about the client or the server.
So at what moment do we call something a server? We do it when a computer shares some service/content on the network, which is accessible by clients. In other words, when we make a server as defined for software.
Regarding the P2P model: every one is both a client and a server, hence called servent. The above apply to the server part of a P2P network, just remember that it also can be a client.
Futher reading:
Client-Server model
P2P
a server is a piece of hardware or on a virtual machine
a service is a process that provides services normally over the network and runs on a server
but a server can also refer to a web server which is actual a service but it's sort of like one as it hosts services
i think those are reasonable working definitions
I think a simpler way to define both besides the definition of the server being a piece of hardware, a server in the software sense is a service that serves data. In other words you interact with a server with a request and you should get a response back. It "serves" data.
A service does not need interaction and is pretty much just a random process that keeps running doing the same thing, but a server is a service because it is basically a process that keep waiting for a request to come in so that it can return a response.
"A service is a component that performs operations in the background without a user interface."
~ Android Developers
Services don't just run on servers
Shell services
Services can run from the shell. Unix refers to services as Daemons (pronounced "demons"), and Windows refers to them as services.
Client-side services
Services can run client-side. Mozilla (and other browsers) support Web Workers which run in a background thread. Client-side frameworks, like Angular, support services as well.
I need to develop a server side application that opens sockets and manages communication with multiple clients. Previous answers have told me this is possible using a single script file, which loops forever.
Is this possible using only a PHP/Perl/Python hosting service? or would I need a VPS or shell access?
Any help is appreciated since I've never worked with sockets before. Thanks for your time.
Cheap Perl/PHP hosting services don't want you running your own long-running processes.
This means you will need a VPS (which obviously includes shell account since you can do anything you want on your private server). A few VPS providers might block outgoing IRC port but I think that is rare.
Linode and Slicehost/Rackspace are just two examples very very well run VPS service providers and I guarantee you can run your own socket application on them.
It would make your host very unhappy since their CPU time is valuable! If you use shared hosting, your host might just kick you out for such a solution! (Read your contract for the fine details.)
I think it could be possible but it depends on the setup of your host, plus the permissions your host are granting you. And most will be unhappy about anything that runs forever. (They prefer to see just short, simple applications.)
Usually the service firewall will block any unexpected ports, or if they are not doing it now they will start doing it after they figure out what you are doing and decide they don't like it.
I would say no because it involve too much security problems
Does anyone have experience deploying GWT apps to EC2?
If I were to install tomcat or apache on a ec2 instance, could I have users connect directly to a url pointing there?
Would that be cost effective, or would java hosting services be best?
Is there any downside to hosting the edge HTTP server on a regular hosting service and have that direct requests to EC2? Performance ever an issue here?
Other answers are correct but I just wanted to share the fact that we are are developing a product that is 100% EC2/S3 based and also have a pure GWT front end.
We use maven2 for builds and the excellent gwt-maven plugin. This makes it easy to produce a WAR package of our web application as output. We use Jetty but Tomcat would work just as well.
We have pound (a http accelerator/load balancer) running on the VM listening for http & https, which then forwards to requests to lighttpd (static) or jetty (app). This also simplifies SSL certificates because pound handles SSL. I've found Java servers have always been a pain to configure with SSL certs.
Yes, you can host pretty much whatever you want, as you effectively have a dedicated Linux machine at your command.
As I last recall, the basic rate for an EC2 instance, on their "low end box" worked out to around $75/month, so you can use that as a benchmark against other vendors. That also assumed that the machine is up 24x7 (since you pay for it by the hour).
The major downside of an EC2 instance is simply that it can "go away" at any time, and when it does, any data written to your instance will "go away" as well.
That means you need to set it up so that you can readily restart the server, but also you need to offline any data that you generate and wish to keep (either to one of Amazons other services, like S3, or to some other external service). That will incur some extra costs depending on volume.
Finally, you will also be billed for any traffic to the service.
The thing to compare it against is another "Virtual Server" from some other vendor. There is a lot of interesting things that can be done with EC2, but it may well be easier to go with a dedicated Virtual hosting service if you're just using a single machine.
Others have given good answers. I would have to add that you need to spend programmer time getting to know EC2's quirks and addressing them (e.g. with EBS). It's not completely trivial, and though it is useful knowledge to have and may be worth it for that reason alone, if you want to get up and running quickly with just a few servers, you should probably look at other hosted options.
On the other hand, if you plan to scale up massively enough (eventually hosting many servers on EC2) then I would highly recommend it. You have to architect a few things, but you need to do that anyways. The flexibility of on-demand computing, and the generally low price, makes this a killer platform once you reach a certain scale of operation.
You definitely can host an http server in EC2, but you need to take into consideration the following:
As mentioned before the cost can be much higher than alternative hosting solutions
Your instance (the machine you've started in EC2) can go off unexpectedly. There is no guarantee from Amazon for 24x7 availability. This mean that the data you've stored in local storage will be lost and when you've start a new instance, it will get a new IP.
To successfully host a server in EC2, you therefore need to employ some other services from Amazon. You need Elastic IP, so that you can circumvent the new IP address problem. You can also use Elastic Block Storage. This is a service that will allow you to mount in your machine a disk, that will not go away when your instance is lost.