I'd like to communicate with mpv using Unix sockets, but Perl 6 doesn't offer any high level interface for doing that. So I thought I could write a little module for that, started digging deeper, found the implementation of IO::Socket::INET, and learned about the NQP ops nqp::socket and nqp::connect.
However, I couldn't find any mention of those in the NQP operations list, and they seem to be quite distinct from the traditional BSD socket API. Hence I'd like to ask: are those two able to create a Unix socket and connect to it? If so, how? Or is there perhaps another way?
I think the best way right now is to use the NativeCall interface to call the C level functions. There are some old design documentation that suggest an IO::Socket::UNIX was thought about. But it's not made it into contemporary releases of the language specification (yet).
Related
I would like to add a REST interface to an existing TCL codebase (so that the programms in other language can use the existing TCL code).
I found a list of Webserver with TCL support but I have no idea which one would be a good solution to quickly map our TCL functions to HTTP/REST calls without tons of boilerplate code.
Has anyone here already done something like this and can tell me which of these servers would be a good (or bad/difficult) solution?
Is there maybe another server/framework that is even better for this use case?
Consider Naviserver. Tcl is its embedded interpreter language. It has a low profile memory overhead, and is regularly maintained and tested for performance and low latency.
For what you’re describing, you might consider Wapp. It’ll do exactly the boilerplate elimination you want, and it’s easy to dive into. You’d probably want to use it as a library, rather than an app, given that you’ve got an existing codebase, but its operation past the initial setup is the same for that use case.
looking into connecting to a secure ftp site (using perl), and downloading all the .log files, saving in new directories named after the day I downloaded the files. I want to do this without modules, as a learning experience, but before I start I wanted to know if you guys thought it was doing, or is way too much for a relatively new programmer and I should just learn the modules?
If it's production work, no, use the modules. Your implementation will be buggy, missing features and unknown to the next person maintaining that code.
Otherwise, yes. It's good to learn the principles of a network protocol. I do have a reservation about FTP as it is a bit baroque, insecure, inefficient and on its way out. scp, HTTP or rsync would be more useful to put your energy into.
I'd start with reading the RFC and putting together your own FTP module using just network sockets. Document and test it as if you were going to release to CPAN as a full learning exercise in making a network module. Run it against some various FTP server implementations as they often interpret the spec differently (or not at all). Don't be afraid to cheat and look at what the existing modules do. Who knows, you might write something better than what's already there.
Learning the principals, just like we did at school for long multiplication and division, means we know how things work when we use a short hand.
However, when new to the world,just like when you learn to speak, you did "A is for Apple" etc, you didnt get explained about the finesse of grammar and all that, you learnt to express yourself enough to be understood.
Programming is a little like the same. While in an ideal world you can easily argue a prewritten generic library is often way less efficient than a specifically targeted set of routines. If the wheel you are using was already invented, it seems a lot of work to make a new one.
So, use the wheels and cogs afailable, once you really have the hang of it, NOW look at inventing your own more efficient ones.
Ad cpan modules:
Modules are an great learning source. Here is zilion modules and you can really learn much studying some of them.
And when/while you mastering your perl, you will start writing you own modules. When your program will use modules anyway (yours one), you can ask - why don't use modules already developed and debugged?
So, learn perl basics, study some modules (for example Net::SFTP) and if you still want write your own solution - it is up to you. :)
'
I am looking at linking a few applications together (all written in different languages like C#, C++, Python) and I am not sure how to go about it.
What I mean by linking? The system I am working on consists of small programs each responsible for a particular processing task. I need to be able to transfer a data set from one application to another easily (the data set in question is not huge, probably a few megabytes) and I also need some form of way to control the current state of the operation (This is where a client-server model rings a bell)
It seems like sockets or maybe SOAP would be a universal solution but just wanted to get some opinions as to what people think about this subject.
Comments/suggestions will be appreciated, thanks!
I personally take a liking towards ØMQ. It's a library that has a familiar BSD-sockets-like interface for passing messages, but you'll find it implements interesting patterns for distributing tasks.
It sounds like you want to arrange several processes in a pipeline. ØMQ allows you to do that using push and poll sockets. (And afterwards, you'll find it's even possible to scale up across multiple processes and machines with little effort.) Take a look at the guide to get started, and the zmq_socket(3) manpage specifically for how push and pull works.
Bindings are available for all the languages you mention.
As for the contents of the message, ØMQ doesn't concern itself with that, they are just blocks of raw data. You can use any format that suits you, such as JSON, or perhaps Protocol Buffers.
What I'm not sure about is the ‘controlling state’ you mention. Are you interested in, for example, cancelling a job halfway through?
For C# to C# you can use Windows Communication Foundation. You may be able to use it with Python and C++ as well.
You may also want to checkout named pipes.
I would think about moving to a model where you eliminate the issue by having centralized data that all of the applications look at. Keep "one source of the truth" so to speak.
Most outside software has trouble linking against C++ code, due to the name-mangling algorithm it uses for its symbols. For that reason, when interfacing with programs written in other languages, it is often best to declare wrappers to things as extern "C" or inside an extern "C" { block.
I need to be able to transfer a data set from one application to another easily (the data set in question is not huge, probably a few megabytes)
Use the file system.
and I also need some form of way to control the current state of the operation
Again, use the file system. A "current_state.json" file with a JSON serialized object is perfect for multiple languages to work with.
It seems like sockets or maybe SOAP would be a universal solution.
Perhaps. But it's overkill for this kind of thing. Your OS already has all the facilities you need. Just use the file system. It's very simple and very reliable.
There are many ways to do interprocess communication. As you said, sockets may be a universal solution. SOAP, i think, is somewhat an overkill. You may also use mailslots. I wrote C++ application using it a couple of years ago. Named pipes could be also a solution, but if you are coding on Windows, it may be difficult.
In my opinion:
Sockets
Mailslots
Are the best candidates.
Are there links, materials, or books about network programming with Perl that are relevant to 2010? I saw some books on programming the network with Perl are very old (about a decade) and the tutorials on the internet are also old and using old modules.
Could someone provide a good reference about programming sockets and network layer/application layer protocols with Perl that are updated to 2010?
This task really demands an asynchronous framework: pretty much everyone uses POE these days. I hate it because for me it doesn't look like Perl. It looks like its own special silly premature-optimized language. Anyway, you can find more info on cpan's docs about poe.
The internals of POE are weird, and POE does rather silly stuff like statically index the contents of #_ for "speed". A POE-component looks like a little brick of voodoo-dung right in the middle of your code. You might like it, give it a shot. If you have to debug it or expand POE::Kernel you might want to look for employment elsewhere.
There is a growing alternative too: AnyEvent. I'd look at it first. It does things massively different and is a much thinner layer that doesn't want to infest your whole application.
Links:
AnyEvent::Socket
POE SocketFactory
Although Lincoln Stein's Network Programming with Perl is a bit old, the content is still mostly good. Network programming at that level hasn't changed that much in the last couple of decades. If you want to learn to play with sockets, that's the book to get to get you started.
However, Perl at the socket level does that same thing the C libraries do. There might be some nice interfaces around it, but look under the interface and you'll see the same things.
I need to move some data from one machine to another. Is it a good idea to write a client server app using sockets in Perl to do the transfer? Will I have problems if one side is written in Java?
I mean, should I be aware of any issues I might face when I try to attempt the above?
Short answer: Using a Perl program as the client or server is just fine. Your only problem might be your personal skill and experience level, but after you do it you know how to do it. :) Most of the problem is choosing how you need to do it, not the technology involved. Perl isn't going to be the problem, but it doesn't have an advantage over other languages either.
As some have already noted, the socket portion of the problem is going to be the same in most languages since almost everything uses the BSD stuff. Perl doesn't have any roadblocks or special gotchas for that. To move data around you create one side to listen on a socket and the other to open a connection and send the data. Easy peasy. You might want to check out Lincoln Stein's Network Programming with Perl for that bit. That can get you the low-level bits.
For higher-level networking, POE is very useful and easy to work with once you get started. It's a framework for dealing with event-driven programming and has many plugins to easily communicate between processes. You might spend a little time learning it, but it gives a lot back too.
If you aren't inventing your own protocol, there's most likely already a Perl module that can format and parse the messages.
If you just want to transfer data, there are several things you can do. The easiest in concept might be just to write lines to the socket and read them as lines from the other end. A bit more complicated than that is using something like Data::Dumper, YAML, or JSON to serialize data to text and send that. For more complex things, such as sharing Perl objects, you might want to use Storable. You freeze your objects, send them as data over the network, then thaw them on the other side.
If you want to implement your client and server in different languages you have a bit more work to figure out how they'll talk to each other. The socket stuff is mostly the same, but a Java server won't understand the output of Perl's Storable (it's possible, but you'll have to parse it yourself and that's no good :). If you do everything right, neither side should care what you used on the other side.
I can only think of one gotcha off the top of my head: most text based network protocols use CRLF for line endings, but Perl on UNIX type machines assumes LF endings by default, this means you will need to change the input and output record separators if you want to use readline (aka <>) and print (also beware of printf, since it doesn't use the output record separator). Of course, if you are going to use a pre-existing protocol, there is probably already a Net::<PROTOCOL NAME> module on CPAN, so you won't need to worry about that. If you are designing your own protocol, I would keep the CRLF convention because it makes it easy to debug the server with telnet (which is really the last valid use for that program).
You don't say whether you need to implement your application to support any particular protocol or whether you need to implement a home grown protocol. The networking support in Perl is flexible enough to support either (or most places in between).
At the low level socket end, your code is going to be fairly similar whatever language your are using - BSD socket APIs are pretty well the same everywhere they are supported. The support you need for this is built into Perl but low level socket programming can be frustrating - it's very low level.
However, Perl's standard library contains the Socket module which is rather easier to use (and well documented).
If you need to implement an existing protocol you may well find that it has already been implemented. For example Net::Telnet implements command/response protocols (like Telnet) making a client app trivial.
Searching CPAN may save you a lot of pain. Look at modules in the Net::* hierarchy
I don't think you're gonna have any major issues that you won't have by not using Perl. Even performance will be comparable to other solutions due to network latencies.
You might want to look at POE framework. It makes writing such components a breeze.
It probably depend on a few factors. Does speed or responsiveness matter? Are you moving data between they same type of machines (Unix to Unix, Windows to Windows)? What type of data are you trying to move (Text or Binary)? What is knowledge about sockets and what languages do you have experience?
I have sent and received binary data over PERL sockets from differing applications, but I don't have much experience with the text processing over sockets from differing machines. If you are moving data between machine you need to keep in mind the way the data is marshalled and if it is packed or aligned on some byte boundry. I have not exchanged data with a Java programs, but is should be similiar.
It probably would help to have some experience with PERL, and I would recommend looking at the examples in the "camel" book. I have used the ones in the book as a starting point and made modification for what I needed to achieve. You may have to consult some other areas of the book if you are dealing with binary data, or to help in doing translations for sending data.
Write socket communication in Perl is relatively easy. Do it right and reliable is big pain even CPAN modules are examples of error prone code. It depends of your expectations.
You are basically asking two questions:
Is Perl a proper language for socket communication?
Is Perl a proper language for UI?
Referring to e5's answer, Perl is indeed a string-centric language with a focus on readable strings, less well equipped to handle binary data. Thus the answer probably lies in the questions: Is your communication string based? Is your UI string based?
If doing binary interaction through a socket, well, you probably could be doing better than Perl (not talking about C, but maybe C-ish languages). If you want to do graphical user-interaction you probably reach faster results by choosing one of the higher languages that focus more on gui interaction. (Java-ish might be the thing here.)