I have a Java-program which communicates with a C++ program using a socket on localhost. Can I expect to gain any performance (either latency, bandwidth, or both) by moving to use a native OS pipe? I'm primarily interested in Windows at the moment, but any insight related to Unix/Linux/OSX is welcome as well.
EDIT: Clarification: both programs run on the same host, currently communicating via a socket, i.e. by making a TCP/IP connection to localhost:. My question was what are the potential performance benefits of switching to using (local) named pipes (Windows), or their Unix equivalent (AF_UNIX domain socket?).
Ken is right. Named pipes are definitely faster on Windows. On UNIX & Linux, you'd want a UDS or local pipe. Same thing, different name.
Anything other than sockets will be faster for local communication. This includes memory mapped files, local pipes, shared memory, COM, etc.
The first google hit turned up this, which clocked NT4 and XP and found named pipes (that's what you meant, right?) to be faster on Windows.
For local processes communication pipes are definitely faster than sockets. There is a benchmark.
I think even though socket is flexible but it can also lead to bad code design. While using pipe it enforces you to design the architecture of your project like which process should be the parent which should be the children and how they cooperate(this will determine how pipes are established) and assign different functionality to processes. Your project design this way will have hierarchical structure and easy to maintain.
Related
May I ask this question for experienced programmers in Perl and knows a thing or two about network programming. Here's the thing, We have a program to develop in Perl about network analyzers, our scope is that we only scan traffic passing through the switch where the host is connected to, will it be difficult if we scan the whole network instead of only one switch? Me and my team are wondering if this will take a lot of time in coding it, or maybe it will only take up one line of code, something like that. I hope my question is clear. Thanks in advance.
That's not a problem of Perl, but how you get the data to the machine where your analyzer runs. Usually you will only get data for the local machine when using a switch (except for ARP requests), so you need get access to a specifically configured port of the switch (mirror port). To get access to all the data in the network you either need to be connected to all the mirror ports of all switches in the network, have special devices to collect all traffic (network tap) or create some kind of sensor network to accumulate data from various places.
Apart from that I would not use Perl for high-speed network analysis (which is usually needed if you want to analyze data from the whole network). Even with C it is not simple to do a fast capture and Perl introduces too much overhead. But you might use Perl for the analysis of a reduced dataset, e.g. after doing lots of filtering on the original data.
Is there a mechanism in Perl to share a socket between two separate processes-- without forking or threading in Linux?
I would assume no, but this answer leaves me to believe it is possible: https://stackoverflow.com/a/1139425/1170839
I would like to create a listening socket on one process, and allow another process to accept/read/write on it.
On many UNIXy systems, as the link you posted indicates, file descriptors may be passed over local domain sockets. For example, a privileged process can open/prepare an fd and then send it to an unprivileged process for use.
Socket::MsgHdr exposes this functionality for perl, and includes examples of file descriptor passing.
The way to go is to use POE. POE makes multithreading in perl ridiculously easy and is designed for just this. POE is a CPAN framework for event driven multithreaded applications. Hands down, the easiest and best way to do this in Perl is POE. There's no reason to reinvent this when it's all been done before and is so well tested.
See:
http://poe.perl.org/?Evolution_of_a_POE_Server and
http://poe.perl.org/?POE_Cookbook/TCP_Servers
I understand that 0MQ is supposed to be faster than TCP Sockets in a clustered environment and I can see where that would be the case (I think that's what they're referring to when they say "Faster than TCP, for clustered products and supercomputing" on the 0MQ website). However, will I see any kind of speedup using 0MQ instead of TCP sockets to communicate between two processes running on the same machine?
Well, the short version is give it a try.
The slightly longer version is that writing TCP sockets can be hard, there's a lot of things that are easy to have problems with, but 0MQ guarantees the message will be delivered in its entirety. It is also written by experts in network sockets, which, with the best will in the world, you probably aren't, and they use a few advanced tricks to speed things along.
You are not actually running on one machine because the VM is treated as a separate machine. This means that TCP sockets have to run through the whole network stack and cannot take shortcuts like they do when you communicate between processes on one machine.
However, you could try UDP multicast under ZeroMQ to see if that speeds up your application. UDP is less reliable on a wide area network, but in a closed environment of a VM talking to its host, you can safely skip all the TCP reliability stuff.
I guess IPC should be faster than TCP. If you are willing to move to a single process, INPROC is definitely going to be much faster.
I think (have not tested) that the answer is false as ZMQ likely uses the same standard C lib and adds some message headers.
Same thing applies for UDP.
Same thing applies for IPC pipes.
ZMQ could be just as fast but since it adds headers it is not likely.
Now it could be a different story if you really need some sort of header and ZMQ has implemented it better than you. Like for message size or type, but I digress.
I'm writing some Perl code. I want it to run on Windows and Linux/UNIX/OSX. So far it works on *NIX and uses fifos.
I am considering switching to sockets to avoid the problem that POSIX::mkfifo() doesn't work on Windows, so I need to write some separate code to use Win32::Pipe.
I'm feeling ambivalent about the whole thing. It seems to me both fixes require about the same amount of work. Is it a good idea to switch to sockets?
Short answer: IO::Socket::INET works on both Windows and *NIX.
Named Pipes
Slightly easier to code up quickly. You don't need write connect code.
Slightly faster. Sockets have the overhead of TCP and setting up the initial connection.
Works on all platforms.
Works even when network card doesn't exist. Some laptops shut down the network card to save power which can prevent even local sockets from working.
Sockets
Works on all platforms. However, some laptops shut down the network card to save power and even local sockets won't work if there is no network interface.
More portable in Perl. IO::Socket::INET works on both *NIX and Windows.
Allows you to have a separate conversation with each client.
Firewalls are not a problem. Ports over 1024 should work.
Personally, I've decided to switch to sockets. In my application it doesn't matter much. But I think it makes the code a bit simpler, gives me the flexibility to move to > 1 client in the future, and I want to learn IO::Socket anyway.
Answering more generically (ie, it's not perl specific):
Doing this sort of thing in windows vs the rest of the world almost always requires separate code for windows vs everything-else. Pretty much everything-else has good solutions for things like this, like unix file sockets or fifo's or ... Then on windows you have to fall back to sockets.
The right thing to do, IMHO, is to use the right solution on windows that isn't network sockets because that opens the application up to security issues. So on everything else "do it correctly" but then on windows, fall back to something like network sockets instead. But, make sure if you take the network-socket route you should at least use local sockets only (ie, bound to 127.0.0.1).
For perl, I'd be tempted to look in CPAN for a class that's already made this generic. But... I wouldn't be surprised if nothing exists.
LWP::socket works fine in Windows and *NIX. If you opt for sockets over fifos, then you eventually would be able to communicate Windows and *NIX processes. May be you don't need it today, but who knows.
IIRC, later versions of Perl have a working socketpair on Windows.
I want to write a program from scratch to see the sockets activity, what they send, what they receive, etc. I don't want to use a Library because it's more for academic purposes than anything else.
Where should I start?
Just to be clear: my program won't be connecting to anything or creating any socket, it just wants to listen to the activity in a computer.
Any thoughts (in any OS) about where to start will be appreciated.
Thanks in advance.
Take a look at libpcap/WinPcap.
You are out of luck as far as portability goes.
Although the application APIs are the same or similar for UNIX and Windows, and, both implementations are historically based on the same Berkeley Sockets BSD code, the operating system architecture and hence the ways you access low level OS functions such as network IO are completely different.
Linux has a number of 'network sniffing' tools(tcpdump, ethereal etc.) easily available. Not so sure about windows, MS provides a Network Monitor and there are some tools available -- Google 'Sysinternals TDImon' for the MS tools.