I basically want to be able to send the apache log file line by line (tail) in between two servers (unidirectionally, from one two one), I want to use perl.
Any idea?, I would like be able to do things with each line of apache log in real time but in another server.
Thanks you!
Not sure about Perl (you can probably wrap this up in a bit of Perl so you can manipulate the data), but netcat (or nc for short) (should be available on most systems).
On one server
tail -f filename | nc -l 12345
On the other server
nc hostname 12345
Of course you can use a different port number. So I guess in Perl you would exec these commands (ssh to the remote server etc.). Hopefully this has given you some ideas! nc has loads of options so you should be able to find something.
If you want to write netcat in Perl then that's a slightly different story.
You can use piped logs. This way your perl script will get every log line on standard input and then it's up to you, how you are going to send them (SSH, FTP, HTTP or maybe even connect to SQL etc).
A very simple solution is to tail the log file from the remote host via SFTP using Net::SFTP::Foreign.
The module contains a sample script implementing the remote tail: sftp_tail.pl
Related
I am trying to build a perl based server that will accept incoming requests and, based on that request, read/write data to server's STDIN/OUT. So I need to accept data from the client STDIN, write it to the server's STDIN (where it is handled by another process), capture the results of that request from the server's STDOUT and then ship it to the client.
I was hoping to build this off of Net::Server but I cannot, for the life of me, figure out how to direct data specifically to the server's STDIN. In my ideal world I'd have a set of file handles like CLISTDIN, CLISTDOUT, SRVSTDIN, and SRVSTDOUT that I could discretely address and manage. I'm just at a loss of how to go about it in Net::Server.
I don't have to use Net::Server so other suggestions are welcome. Net::Server just has a number of other features I would like to use.
Thanks for any insight.
Have you thought about just using ssh? With ssh your can run commands on a remote computer (server) which redirects your stdin to that command, and the stdout from that command to yours. Simple demo:
$ echo "hello, world" | md5sum
22c3683b094136c3398391ae71b20f04 -
$ echo "hello, world" | ssh myserver.com md5sum
22c3683b094136c3398391ae71b20f04 -
Or
$ echo "hello, world" | ssh user#myserver.com "/path/any/command --with --args x y z"
(You can set up ssh for automatic login with ip check by ssh-keygen and ~/.ssh/authorized_keys wont say more about that here)
Turns out I could do this by overloading post_accept to change how STDIN/STDOUT were being used. I was able to create two distinct filehandles so I could refer to the client IN/OUT without breaking the servers IO. That said, Net::Server kept getting in the way of what IO wanted to do so I abandoned it and just rolled my own forking server. Ended up being easier overall.
I need some heads up here. I need to write a CGI script which has to connect unix host and execute set of perl scripts.
I am new to CGI world and I have couple of questions in my mind for which I don't know perl solution.
How will connect to Unix host from CGI script. I believe using Net::SSH ,is there any better module to do this ?
Lets assume I have connected to the server , now how would I execute the script and how would I now the status (running/success/failure)
of the script.
a. When its running I would like to see the output that gets generated. Is it possible to view the script output in realtime?
b. If its a failure then I should be notified and reason for failure and should not trigger the next script in sequence.
If someone has similar setup already available and ready to show the code/setup , I would be much happier :)
I have a shell script which archives log files based on the whether the process is running or not. If the log file is not used by the process then I archive it. Until now, I'm using lsof to get the log file being used but in future, I have decided to use perl to do this function.
Is there a perl module similar to what lsof in linux can perform ?
There is a perl module, which wraps around lsof. See Unix::Lsof.
As I see it, the big problem with not using lsof is that one would need to work in a way that is independent of the operating system. Using lsof allows the perl programmer to work with a consistent application allowing for operating system independence.
To have a perl module developer to write lsof would, in effect, be writing lsof as a library and then link that into perl - which is much more work than just using the existing binary.
One could also use the fuser command, which shows the process IDs with the file handle. There is also a module which seeks to implement the same functionality. Note from the perldoc:
The way that this works is highly unlikely to work on any other OS
other than Linux and even then it may not work on other than 2.2.*
kernels.
One might try walking /proc/*/fd and looking at the file descriptors in there to see if any are pointing to the file in question. If it is known what the process ID of a running process that would be opening the log file, it would be just as easy to look at that process. Note, that this is how the fuser module works.
That said, it should be asked "why do you want to move away from lsof"?
I am on an embedded platform (mipsel architecture, Linux 2.6 kernel) where I need to monitor IPC between two closed-source processes (router firmware) in order to react to a certain event (dynamic IP change because of DSL reconnect). What I found out so far via strace is that whenever the IP changes, the DSL daemon writes a special message into a UNIX domain socket bound to a specific file name. The message is consumed by another daemon.
Now here is my requirement: I want to monitor the data flow through that specific UNIX domain socket and trigger an event (call a shell script) if a certain message is detected. I tried to monitor the file name with inotify, but it does not work on socket files. I know I could run strace all the time, filtering its output and react to changes in the filtered log file, but that would be too heavy a solution because strace really slows down the system. I also know I could just poll for the IP address change via cron, but I want a watchdog, not a polling solution. And I am interested in finding out whether there is a tool which can specifically monitor UNIX domain sockets and react to specific messages flowing through in a predefined direction. I imagine something similar to inotifywait, i.e. the tool should wait for a certain event, then exit, so I can react to the event and loop back into starting the tool again, waiting for the next event of the same type.
Is there any existing Linux tool capable of doing that? Or is there some simple C code for a stand-alone binary which I could compile on my platform (uClibc, not glibc)? I am not a C expert, but capable of running a makefile. Using a binary from the shell is no problem, I know enough about shell programming.
It has been a while since I was dealing with this topic and did not actually get around to testing what an acquaintance of mine, Denys Vlasenko, maintainer of Busybox, proposed as a solution to me several months ago. Because I just checked my account here on StackOverflow and saw the question again, let me share his insights with you. Maybe it is helpful for somebody:
One relatively easy hack I can propose is to do the following:
I assume that you have a running server app which opened a Unix domain listening socket (say, /tmp/some.socket), and client programs connect to it and talk to the server.
rename /tmp/some.socket -> /tmp/some.socket1
create a new socket /tmp/some.socket
listen on it for new client connections
for every such connection, open another connection to /tmp/some.socket1 to original server process
pump data (client<->server) over resulting pairs of sockets (code to do so is very similar to what telnetd server does) until EOF from either side.
While you are pumping data, it's easy to look at it, to save it, and even to modify it if you need to.
The downside is that this sniffer program needs to be restarted every time the original server program is restarted.
This is similar to what Celada also answered. Thanks to him as well! Denys's answer was a bit more concrete, though.
I asked back:
This sounds hacky, yes, because of the restart necessity, but feasible.
Me not being a C programmer, I keep wondering though if you know a
command line tool which could do the pass-through and protocolling or
event-based triggering work for me. I have one guy from our project in
mind who could hack a little C binary for that, but I am unsure if he
likes to do it. If there is something pre-fab, I would prefer it. Can it
even be done with a (combination of) BusyBox applet(s), maybe?
Denys answered again:
You need to build busybox with CONFIG_FEATURE_UNIX_LOCAL=y.
Run the following as intercepting server:
busybox tcpsvd -vvvE local:/tmp/socket 0 ./script.sh
Where script.sh is a simple passthrough connection
to the "original server":
#!/bin/sh
busybox nc -o /tmp/hexdump.$$ local:/tmp/socket1 0
As an example, I added hex logging to file (-o FILE option).
Test it by running an emulated "original server":
busybox tcpsvd -vvvE local:/tmp/socket1 0 sh -c 'echo PID:$$'
and by connecting to "intercepting server":
echo Hello world | busybox nc local:/tmp/socket 0
You should see "PID:19094" message and have a new /tmp/hexdump.19093 file
with the dumped data. Both tcpsvd processes should print some log too
(they are run with -vvv verbosity).
If you need more complex processing, replace nc invocation in script.sh
with a custom program.
I don't think there is anything that will let you cleanly sniff UNIX socket traffic. Here are some options:
Arrange for the sender process to connect to a different socket where you are listening. Also connect to the original socket as a client. On receipt of data, notice the data you want to notice and also pass everything along to the original socket.
Monitor the system for IP address changes yourself using a netlink socket (RTM_NEWADDR, RTM_NEWLINK, etc...).
Run ip monitor as an external process and take action when it writes messages about added & removed IP addresses on its standard output.
is there a way to query a server for its OS type in Perl? For example, if I knew that a remote server was running Windows, I might send it a winver from my local machine and get the output to determine which version of Windows it's running. Yet, is there a way to be even more abstract and simply ask "what are you?"
Since CPAN is huge, I was wondering if there were a module that encapsulated this sort of functionality.
If you can get command-line access on the remove server, then you should be able to use %ENV:
jmaney> perl -e 'print "$ENV{OSTYPE}\n";'
linux
Edit: It looks as though the key in Windows (or, at least on Windows 7 on my laptop) is OS. So, unfortunately, the exact solution via %ENV is OS-dependent... You could, however, check to see which of $ENV{OS} or $ENV{OSTYPE} is defined (and if they're both defined, then canonically pick which one you want to use), and proceed accordingly.
There is no foolproof way to do this, but the HTTP Server header -- which the server isn't required to send -- often contains the OS. For example, it may look like this (from Wikipedia):
Server: Apache/1.3.27 (Unix) (Red-Hat/Linux)
The Perl CGI module has an http function that gets the HTTP headers. You could use it like this:
my $server = $q->http('Server');
# Test $server for Windows, *nix, etc
# My Perl experience is minimal and I haven't used it in
# a while, so I'm not going to give an example here, but
# someone can feel free to edit one in.
CPAN probably has a module to do the testing on the Server header for you.