Perl socket parse packets from a network stream - perl

I am trying to figure out a proper way to parse a stream of data using perl.
I have read through many of the examples, documentations and questions, but could not find how I could basically cut a "package" from the stream of data and process it.
This is the situation:
- stream of data coming from a certain IP to an IP and port
- stream contains some gibberish and then something between and with the data in there being semicolon seperated
My attempts so far is to have a Socket listening on the port and process the $data var:
#!/usr/bin/perl
use IO::Socket::INET;
# auto-flush on socket
$| = 1;
# creating a listening socket
my $socket = new IO::Socket::INET (
LocalHost => '127.0.0.1',
LocalPort => '7070',
Proto => 'tcp',
Listen => 5,
Reuse => 1
);
die "cannot create socket $!\n" unless $socket;
print "server waiting for client connection on port 7070 \n";
while(1)
{
# waiting for a new client connection
my $client_socket = $socket->accept();
# get information about a newly connected client
my $client_address = $client_socket->peerhost();
my $client_port = $client_socket->peerport();
print "connection from $client_address:$client_port\n";
# read up to 1024 characters from the connected client
my $data = "";
$client_socket->recv($data, 1024);
print "received data: $data\n";
#data_array = split(/;/,$data);
foreach (#data_array) {
print "$_\n";
}
# write response data to the connected client
$data = "ok";
$client_socket->send($data);
# notify client that response has been sent
shutdown($client_socket, 1);
}
$socket->close();
This works but as far as I understand this will put the whole stream in up to the size and then process that.
My question:
How can I identify the part I need (start-end), process that and then go on to the next?

I've never understood why people use recv to read from a stream socket.
Normally, the reading loop looks something like the following:
my $buf = '';
while (1) {
my $rv = sysread($socket, $buf, 64*1024, length($buf));
if (!defined($rv)) {
die("Can't read from socket: $!\n");
}
if (!$rv) {
die("Can't read from socket: Premature EOF\n") if length($buf);
last;
}
while (my $msg = defined(check_for_full_message_and_extract_it_from_buf($buf))) {
process_msg($msg);
}
}
(Keep in mind that sysread returns as soon as there is some data, even if there's less data than requested.)
For example, the inner loop for sentinel-terminated data would look like the following:
while ($buf =~ s/^(.*)\n//) {
process_msg("$1");
}
For example, the inner loop for length-prefixed blocks would look like the following:
while (1) {
last if length($buf) < 4;
my $len = unpack('N', $buf);
last if length($buf) < 4+$len;
substr($buf, 0, 4, '');
my $msg = substr($buf, 0, $len, '');
process_msg($msg);
}
If you're particular case, you'd remove any data from the start $buf that you want to ignore until you get to the part in which you're interested, then you'd start extracting the items in which you are interested. This is vague, but I only have a vague description of the protocol with which to work.

I solved this by using the original code and adding :
if ( $data=~/<START>>/) {
print "\nFound start\n";
$message.=$data;
while ($message !~/END/){
$client_socket->recv($data, $message_length);
$message.=$data;
print "\nStill reading\n";
};
print "\nFound end\n"; # but may contain (part of) next START
}
I still need to implement the part where I check if the chunk read has part of the next message, but I'll figure that out.
Thank you for the help!

Related

In perl socket programming how to send a data from client and receive it from server

I am using Socket module to perform socket programming in Perl.And now I want to send one data from the client and receive it from the Server side. How I will achive this. Please help.
Given below in the code i used
server
#!/usr/bin/perl -w
# Filename : server.pl
use strict;
use IO::Socket;
use Socket;
use Sys::Hostname;
use constant BUFSIZE => 1024;
# use port 7890 as default
my $port = shift || 7890;
my $proto = getprotobyname('tcp');
my $server = "localhost"; # Host IP running the server
# create a socket, make it reusable
socket(SOCKET, PF_INET, SOCK_STREAM, $proto)
or die "Can't open socket $!\n";
setsockopt(SOCKET, SOL_SOCKET, SO_REUSEADDR, 1)
or die "Can't set socket option to SO_REUSEADDR $!\n";
# bind to a port, then listen
bind( SOCKET, pack_sockaddr_in($port, inet_aton($server)))
or die "Can't bind to port $port! \n";
listen(SOCKET, 5) or die "listen: $!";
print "SERVER started on port $port\n";
# accepting a connection
my $client_addr;
my $val = 100;
while ($client_addr = accept(NEW_SOCKET, SOCKET)) {
# send them a message, close connection
my $name = gethostbyaddr($client_addr, AF_INET );
print NEW_SOCKET "Smile from the server";
print NEW_SOCKET $val;
print "Connection recieved from $name\n";
close NEW_SOCKET;
}
client
#!/usr/bin/perl -w
# Filename : client.pl
use strict;
use IO::Socket;
use Socket;
use Sys::Hostname;
use constant BUFSIZE => 1024;
# initialize host and port
my $host = shift || 'localhost';
my $port = shift || 7890;
my $server = "localhost"; # Host IP running the server
# create the socket, connect to the port
socket(SOCKET,PF_INET,SOCK_STREAM,(getprotobyname('tcp'))[2])
or die "Can't create a socket $!\n";
connect( SOCKET, pack_sockaddr_in($port, inet_aton($server)))
or die "Can't connect to port $port! \n";
my $line;
my $req = 1000;
while ($line = <SOCKET>) {
print "$line\n";
}
close SOCKET or die "close: $!";
Here is a basic example. The code below adds to what you have, but please note that modules IO::Socket::IP or core IO::Socket::INET make it easier than the lower level calls you use.
The only changes to your code (other than shown below) are from SOCKET to lexical my $socket, and an existing declaration is moved inside the while condition.
Every server-client system needs a protocol, an arrangement of how the messages are exchanged. Here, once the client connects the server sends a message and then they exchange single prints.
server.pl
# ... code from the question, with $socket instead of SOCKET
use IO::Handle; # for autoflush
while (my $client_addr = accept(my $new_socket, $socket))
{
$new_socket->autoflush;
my $name = gethostbyaddr($client_addr, AF_INET );
print "Connection received from $name\n";
print $new_socket "Hello from the server\n";
while (my $recd = <$new_socket>) {
chomp $recd;
print "Got from client: $recd\n";
print $new_socket "Response from server to |$recd|\n";
}
close $new_socket;
}
Instead of loading IO::Handle you can make a handle hot (autoflush) using select.
client.pl
I add a counter $cnt to simulate some processing that leads to a condition to break out.
# ... same as in question, except for $socket instead of SOCKET
use IO::Handle;
$socket->autoflush;
my $cnt = 0;
while (my $line = <$socket>) {
chomp $line;
print "Got from server: $line\n";
last if ++$cnt > 3; # made up condition to quit
print $socket "Hello from client ($cnt)\n";
}
close $socket or die "close: $!";
This behaves as expected. The client exits after three messages, the server stays waiting. If you wish to indeed write just once the simple print and read replace the while loops.
The exchanges can be far more sophisticated, see the example in perlipc linked at the end.
A few comments
Using the mentioned modules makes this much easier
Any glitch in flushing can lead to deadlocks, where one party wrote and is waiting to read, while the other did not get the message still sitting in the pipe and is thus, also, waiting to read
Check everything. All checking is left out for brevity
use warnings; is better than the -w switch. See the discussion on warnings page
This is only meant to answer the question of how to enable communication between them. One good resource for study is perlipc, which also has a full example. The docs for involved modules provide a lot of information as well.

Better way to handle perl sockets to read/write to active proccess

First of all I would thank you guys not offering a work around as a solution (although it would be cool to know other ways to do it). I was setting up tg-master project (telegram for cli) to be used by check_mk alert plugin. I found out that telegram runs on a stdin/stdout proccess so I tought it would be cool to "glue" it, so i wrote with a lot of building blocks from blogs and cpan the next 2 pieces of code. They already work (i need to handle broken pipes sometimes) but I was wondering if sharing this could come from some experts new ideas.
As you could see my code relies on a eval with a die reading from spawned process, and I know is not the best way to do it. Any suggestions? :D
Thank you guys
Server
use strict;
use IO::Socket::INET;
use IPC::Open2;
use POSIX;
our $pid;
use sigtrap qw/handler signal_handler normal-signals/;
sub signal_handler {
print "what a signal $!\nlets kill $pid\n";
kill 'SIGKILL', $pid;
#die "Caught a signal $!";
}
# auto-flush on socket
$| = 1;
# creating a listening socket
my $socket = new IO::Socket::INET(
LocalHost => '0.0.0.0',
LocalPort => '7777',
Proto => 'tcp',
Listen => 5,
Reuse => 1
);
die "cannot create socket $!\n" unless $socket;
print "server waiting for client connection on port 7777\n";
my ( $read_proc, $write_proc );
my ( $uid, $gid ) = ( getpwnam "nagios" )[ 2, 3 ];
POSIX::setgid($gid); # GID must be set before UID!
POSIX::setuid($uid);
$pid = open2( $read_proc, $write_proc, '/usr/bin/telegram' );
#flush first messages;
eval {
local $SIG{ALRM} = sub { die "Timeout" }; # alarm handler
alarm(1);
while (<$read_proc>) { }
};
while (1) {
my $client_socket = $socket->accept();
my $client_address = $client_socket->peerhost();
my $client_port = $client_socket->peerport();
print "connection from $client_address:$client_port\n";
# read until \n
my $data = "";
$data = $client_socket->getline();
# write to spawned process stdin the line we got on $data
print $write_proc $data;
$data = "";
eval {
local $SIG{ALRM} = sub { die "Timeout" }; # alarm handler
alarm(1);
while (<$read_proc>) {
$client_socket->send($_);
}
};
# notify client that response has been sent
shutdown( $client_socket, 1 );
}
$socket->close();
Client
echo "contact_list" | nc localhost 7777
or
echo "msg user#12345 NAGIOS ALERT ... etc" | nc localhost 7777
or
some other perl script =)
If you are going to implement a script that performs both reads and writes from/to different handles, consider using select (the one defined as select RBITS,WBITS,EBITS,TIMEOUT in the documentation). In this case you will totally avoid using alarm with a signal handler in eval to handle a timeout, and will only have one loop with all of the work happening inside it.
Here is an example of a program that reads from both a process opened with open2 and a network socket, not using alarm at all:
use strict;
use warnings;
use IO::Socket;
use IPC::Open2;
use constant MAXLENGTH => 1024;
my $socket = IO::Socket::INET->new(
Listen => SOMAXCONN,
LocalHost => '0.0.0.0',
LocalPort => 7777,
Reuse => 1,
);
# accepting just one connection
print "waiting for connection...\n";
my $remote = $socket->accept();
print "remote client connected\n";
# simple example of the program writing something
my $pid = open2(my $localread, my $localwrite, "sh -c 'while : ; do echo boom; sleep 1 ; done'");
for ( ; ; ) {
# cleanup vectors for select
my $rin = '';
my $win = '';
my $ein = '';
# will wait for a possibility to read from these two descriptors
vec($rin, fileno($localread), 1) = 1;
vec($rin, fileno($remote), 1) = 1;
# now wait
select($rin, $win, $ein, undef);
# check which one is ready. read with sysread, not <>, as select doc warns
if (vec($rin, fileno($localread), 1)) {
print "read from local process: ";
sysread($localread, my $data, MAXLENGTH);
print $data;
}
if (vec($rin, fileno($remote), 1)) {
print "read from remote client: ";
sysread($remote, my $data, MAXLENGTH);
print $data;
}
}
In the real production code you will need to carefully check for errors returned by various function (socket creation, open2, accept, and select).

Programming a chat room in perl, I'm having issues with the client?

I'm following this guide explaining how to do a server using IO::Async but I'm having issues with my client code. I have it where I send first then receive. This makes me press enter on each client before receiving any data. I figured I'd have to listen till I wanted to type something but I'm not really sure how. Below is my current client code.
use IO::Socket::INET;
# auto-flush on socket
$| = 1;
# create a connecting socket
my $socket = new IO::Socket::INET (
PeerHost => 'localhost',
PeerPort => '12345',
Proto => 'tcp',
);
die "cannot connect to the server $!\n" unless $socket;
print "My chat room client. Version One.\n";
while (1) {
my $data = <STDIN>;
$socket->send($data);
my $response = "";
$socket->recv($response, 1024);
print ">$response";
last if (index($data, "logout") == 0);
}
$socket->close();
I actually had this problem myself a few weeks ago when trying to make a client/server chat for fun.
Put it off until now.
The answer to your problem of having to hit enter to receive data, is that you need to use threads. But even if you use threads, if you do $socket->recv(my $data, 1024) you won't be able to write anything on the command line.
This isn't using your code, but here is my solution after banging my head against a wall for the last 24hrs. I wanted to add this as an answer, because though the question is out there on stackoverflow, none of the answers seemed to show how to use IO::Select.
Here is the server.pl script, it does not use threading:
#!/usr/bin/perl
use strict;
use warnings;
use IO::Socket::INET;
use IO::Select;
$| = 1;
my $serv = IO::Socket::INET->new(
LocalAddr => '0.0.0.0',
LocalPort => '5000',
Reuse => 1,
Listen => 1,
);
$serv or die "$!";
print 'server up...';
my $sel = IO::Select->new($serv); #initializing IO::Select with an IO::Handle / Socket
print "\nAwaiting Connections\n";
#can_read ( [ TIMEOUT ] )
#can_write ( [ TIMEOUT ] )
#add ( HANDLES )
#http://perldoc.perl.org/IO/Select.html
while(1){
if(my #ready = $sel->can_read(0)){ #polls the IO::Select object for IO::Handles / Sockets that can be read from
while(my $sock = shift(#ready)){
if($sock == $serv){
my $client = $sock->accept();
my $paddr = $client->peeraddr();
my $pport = $client->peerport();
print "New connection from $paddr on $pport";
$sel->add($client); #Adds new IO::Handle /Socket to IO::Select, so that it can be polled
#for read/writability with can_read and can_write
}
else{
$sock->recv(my $data, 1024) or die "$!";
if($data){
for my $clients ($sel->can_write(0)){
if($clients == $serv){next}
print $clients $data;
}
}
}
}
}
}
And the client.pl, which uses threads:
#!/usr/bin/perl
use strict;
use warnings;
use IO::Socket::INET;
use threads;
use IO::Select;
$| = 1;
my $sock = IO::Socket::INET->new("localhost:5000");
$sock or die "$!";
my $sel = IO::Select->new($sock);
print "Connected to Socket ". $sock->peeraddr().":" . $sock->peerport() . "\n";
#This creates a thread that will be used to take info from STDIN and send it out
#through the socket.
threads->create(
sub {
while(1){
my $line = <>;
chomp($line);
for my $out (my #ready = $sel->can_write(0)){
print $out $line;
}
}
}
);
while(1){
if(my #ready = $sel->can_read(0)){
for my $sock(#ready){
$sock->recv(my $data, 1024) or die $!;
print "$data\n" if $data;
}
}
}
There is one other problem that arises though, when the client receives data and prints it to the console, your cursor goes to a new line, leaving behind any characters you had typed.
Hope this helps and answers your question.
For a simple "just send from STDIN, receive to STDOUT" client, you could use any of telnet, nc or socat. These will be simple enough to use for testing.
$ telnet localhost 12345
$ nc localhost 12345
$ socat stdio tcp:localhost:12345
If you actually want to write something in Perl, because you want to use it as an initial base to start a better client from, you probably want to base that on IO::Async. You could then use the netcat-like example here. That will give you a client that looks-and-feels a lot like a simple netcat.
I am guessing you need to set the MSG_DONTWAIT flag on your recv call, and print the response only if it is non-null.
$socket->recv($response, 1024, MSG_DONTWAIT);
print ">$response" if ($response ne "");

Perl HTTP server not reading requests from clients

I have created an HTTP server in Perl to accept requests from clients.
At the moment only one client is sending the request.
This is how my set-up is:
Client --> Server (this is proxy server as well connecting to the internet), Apache 2 running on Ubuntu.
This is the Perl code for my server:
#!/usr/bin/perl
use IO::Socket::INET;
use strict;
use warnings;
use LWP::Simple;
# auto-flush on socket
$| = 1;
my $port = 7890;
# Create a listening port
my $socket = new IO::Socket::INET(
LocalHost => '127.0.0.1',
LocalPort => shift || $port,
Proto => 'tcp',
Listen => SOMAXCONN,
Reuse => 1
) or die "cannot create socket $!\n";
# open a file and write client requests to the file
$| = 1;
open(FH, '>>', '/home/suresh/clientrequest.txt')
or die "could not open the /home/suresh/clientrequest : $!\n";
print FH "server waiting for client on port\n"
or die "could not write to file : $!\n";
while (my $client_socket = $socket->accept()) {
$client_socket->autoflush(1);
#print FH "Welcome to $0 \n";
my $client_address = $socket->peerhost();
my $client_port = $client_socket->peerport();
print FH "connection from $client_address:$client_port\n";
# read from connected client
my $data = "";
$client_socket->recv($data, 1024);
print FH "Data received from $client_address:$client_port: $data\n";
# write response data to the client
$data = "Sucessfully processed your request";
$client_socket->send($data);
shutdown($client_socket, 1);
}
close(FH);
$socket->close();
When I bring this server up and try sending a request from a client, the request is written to the file, so it looks like the requests are captured by the server.
Can anyone please let me know what other configurations I need to do at server side and at client?
If you write
$| = 1;
then flushing is only activated for the default output filehandle. This is STDOUT unless changed with the select() builtin. So FH is not flushed here — I guess this was your intention. Instead, you have to write
FH->autoflush(1);

Using select to poll connections - TCP server

use strict; use warnings;
use IO::Socket;
use IO::Select;
my $read_select = IO::Select->new();
my $write_select = IO::Select->new();
my $socket = IO::Socket::INET->new(
LocalHost => '127.0.0.1',
LocalPort => '5556',
Proto => 'tcp',
Listen => 50,
Reuse => 1,
) or die "Could not create socket: $!";
print "Socket Created . Waiting for connection ...\n";
## poll to accept new connection or to receive data from a connection
$read_select->add($socket);
print "Added socket to read list ";
my $reade;
my $newconn;
my #read;
my #write;
while(1) {
#read = $read_select->can_read();
foreach my $reade(#read) {
if($reade == $socket) {
print "New conn received";
my $newconn = $reade->accept();
$write_select->add($newconn);
}
else {
print "data received";
}
}
}
#write = $write_select->can_write();
foreach my $write(#write) {
$write->send("got ur data");
}
I am trying to poll for connections using select statement. Why is that if i use an infinite loop, no connection is accepted. It works fine without while(1)
I think you are being bitten by I/O buffering here. Perl buffers all input and output. It generally doesn't print to the terminal until it has received an entire line.
Your code is probably working with the while(1), but you can't see the output of your debug print statements because the output to the terminal is being buffered. Once you get to the second time through the loop, $read_select->can_read() blocks forever, so you never see the output of the print statements.
You can probably fix this just by adding \n to the end of each print statement. Another option is setting $| = 1;. This disables buffering. See perlvar's discussion of $| for more information on buffering.