I have server A which runs under Windows (C#), I have server B which runs under Linux (Java). I need to setup a fast and reliable way of interaction between these two servers.
Requirements:
The size of data is small (many small messages of kbyte or something)
The rate is 1 message per second at least; good to have 1 per ms
The order should be preserved
Persistency not required
Delivery confirmation - yes
Some options out of my head:
Kafka / other messaging systems. The downsize is that I have to use a
middleware in a form of server.
Use Cloud queues from AWS or other cloud service. Can be costly?
Own messaging via TCP messages (very time consuming, I guess there
are open-source alternatives). No middleware.
Somehow via shared filesystem?
Setup Linux Server on Windows and.. what? Any special options if
applications are sharing same host?
Related
We need to make a full stack load testing. And the data journey is starting from mqtt client. Are there any good services to make a load testing for such case?
Depending on the number of virtual users you would like to simulate and your budget the options are in:
CloudMQTT service which is free for 5 connections and 10 Kbit/s bandwidth
Apache JMeter is a free and open source multi-protocol load testing tool. Out of the box JMeter does not support MQTT protocol however there are plugins like:
mqtt-jmeter
MQTT plugin
Check out Testing the MQTT Messaging Broker for IoT - A Guide
MQTT-Stresser
MQTT Toolbox
So feel free to choose one of the above (or other solution), the final choice mostly depends on your project infrastructure and possibility of i of test tool ntegration into your product delivery lifecycle
I'm building a software agent that run on a server, this software agent act as a server manager i.e. starting/stoping Docker container, monitoring etc.
This server will host/serve many services, these services are programs running in Docker container, 1 program/service per container.
There may be so many servers and these servers aren't necessary be a high performance server, they ranges from a small VM to high performance computer. Right now, I assume that every service uses HTTP to serve request.
The function that I want to implement in this software agent is tracking the number of clients that are currently connecting (requesting) to server for every service (e.x. server A is processing 500 requests) or specific program is ok (e.x. program A is processing 100 requests, program B is processing 200 request).
I want to know this number because I want to do workload balancing across servers that host the same service.
The following is ideas that I have.
Implementing load balancer/reverse proxy inside this agent (I would use this load balancer https://github.com/nwoodthorpe/Load-Balancer-Golang). This may be the last choice because I think it will use pretty much resources for load balancing.
Letting service programs that are running on server tell agent whenever they start and finish processing request. I simply implement UDP socket server in agent to listen for a datagram that tell unique ID of request (actually, can be anything that help me distinguish specific request that being processed) and status whether is being processed or finish processing.
So, I would like to ask for a suggestion for above approaches, which one is better or how should I implement it? Is there any better approach to do this?
I have a requirement to transfer/multicast a large file about >40g of file from a server to multiple clients at the same time and this will be done for only once. Is there any good protocol to do that in Linux? I tried using UFTP, but it didn't work.
UFTP should be a good tool for this situation. If the server and clients are on the same LAN, there shouldn't be any issue with them communicating. If there are one or more routers separating them, then you would either have to configure routers to allow multicast traffic to pass or you could use UFTP's proxy servers to create a bridge between different network segments.
You could use the excellent bittorrent protocol and make it private by using Bittorent Sync.
Go to Bittorrent Sync Web Site for details.
The main advantages I see are :
It's design to transport large files (if you have a network disruption it's not a problem)
It's free
It's cross plateform : Windows, Linux (i386, x64, ARM, PowerPC), FreeBSD, Mac, Android, IOS, and more ...
It's secure (you provide the encryption keys)
It's quite simple to configure
It seems that both service and server refer to some web based application. But is there any precise definition of the two terms?
A server offers one or more services. Server is also a more technical term, whereas service is more a term off the problem domain.
You also need to distinguish between:
Server as hardware (see post from Dan D)
Server as software (eg. Apache HTTP server)
You can find more elaborate definiton on Wikipedia:
Service
Server
This is regardless of client-server or P2P models.
A server provides services to one or more clients, and a server(hardware) is a computer. A server(hardware) can be anything from a home computer to a big server-rack with a lot of processor power.
From the view of a computer, a server(software) is just a set of services which is available to clients on the network.
Some well known services are web-server, mail-server. ftp-server. notice they are called xxx-server because such programs consist of a client and server part. The postfix is mainly to distinguish whether we are talking about the client or the server.
So at what moment do we call something a server? We do it when a computer shares some service/content on the network, which is accessible by clients. In other words, when we make a server as defined for software.
Regarding the P2P model: every one is both a client and a server, hence called servent. The above apply to the server part of a P2P network, just remember that it also can be a client.
Futher reading:
Client-Server model
P2P
a server is a piece of hardware or on a virtual machine
a service is a process that provides services normally over the network and runs on a server
but a server can also refer to a web server which is actual a service but it's sort of like one as it hosts services
i think those are reasonable working definitions
I think a simpler way to define both besides the definition of the server being a piece of hardware, a server in the software sense is a service that serves data. In other words you interact with a server with a request and you should get a response back. It "serves" data.
A service does not need interaction and is pretty much just a random process that keeps running doing the same thing, but a server is a service because it is basically a process that keep waiting for a request to come in so that it can return a response.
"A service is a component that performs operations in the background without a user interface."
~ Android Developers
Services don't just run on servers
Shell services
Services can run from the shell. Unix refers to services as Daemons (pronounced "demons"), and Windows refers to them as services.
Client-side services
Services can run client-side. Mozilla (and other browsers) support Web Workers which run in a background thread. Client-side frameworks, like Angular, support services as well.
Does anyone have experience deploying GWT apps to EC2?
If I were to install tomcat or apache on a ec2 instance, could I have users connect directly to a url pointing there?
Would that be cost effective, or would java hosting services be best?
Is there any downside to hosting the edge HTTP server on a regular hosting service and have that direct requests to EC2? Performance ever an issue here?
Other answers are correct but I just wanted to share the fact that we are are developing a product that is 100% EC2/S3 based and also have a pure GWT front end.
We use maven2 for builds and the excellent gwt-maven plugin. This makes it easy to produce a WAR package of our web application as output. We use Jetty but Tomcat would work just as well.
We have pound (a http accelerator/load balancer) running on the VM listening for http & https, which then forwards to requests to lighttpd (static) or jetty (app). This also simplifies SSL certificates because pound handles SSL. I've found Java servers have always been a pain to configure with SSL certs.
Yes, you can host pretty much whatever you want, as you effectively have a dedicated Linux machine at your command.
As I last recall, the basic rate for an EC2 instance, on their "low end box" worked out to around $75/month, so you can use that as a benchmark against other vendors. That also assumed that the machine is up 24x7 (since you pay for it by the hour).
The major downside of an EC2 instance is simply that it can "go away" at any time, and when it does, any data written to your instance will "go away" as well.
That means you need to set it up so that you can readily restart the server, but also you need to offline any data that you generate and wish to keep (either to one of Amazons other services, like S3, or to some other external service). That will incur some extra costs depending on volume.
Finally, you will also be billed for any traffic to the service.
The thing to compare it against is another "Virtual Server" from some other vendor. There is a lot of interesting things that can be done with EC2, but it may well be easier to go with a dedicated Virtual hosting service if you're just using a single machine.
Others have given good answers. I would have to add that you need to spend programmer time getting to know EC2's quirks and addressing them (e.g. with EBS). It's not completely trivial, and though it is useful knowledge to have and may be worth it for that reason alone, if you want to get up and running quickly with just a few servers, you should probably look at other hosted options.
On the other hand, if you plan to scale up massively enough (eventually hosting many servers on EC2) then I would highly recommend it. You have to architect a few things, but you need to do that anyways. The flexibility of on-demand computing, and the generally low price, makes this a killer platform once you reach a certain scale of operation.
You definitely can host an http server in EC2, but you need to take into consideration the following:
As mentioned before the cost can be much higher than alternative hosting solutions
Your instance (the machine you've started in EC2) can go off unexpectedly. There is no guarantee from Amazon for 24x7 availability. This mean that the data you've stored in local storage will be lost and when you've start a new instance, it will get a new IP.
To successfully host a server in EC2, you therefore need to employ some other services from Amazon. You need Elastic IP, so that you can circumvent the new IP address problem. You can also use Elastic Block Storage. This is a service that will allow you to mount in your machine a disk, that will not go away when your instance is lost.