Perl. Fetching file from remote server, only core modules - perl

I need help: is there a way to fetch a file from the remote server using only core modules of perl 5.8.8? File::Fetch became core module only from 5.9.

This comes up all the time. Take a look at the classic yes, even you can use CPAN. If you have the ability to create and run a Perl script, then you also have the ability to put a module in your local directory and use it. The requirement to use only core modules is entirely artificial.
In your case, LWP::Simple's getstore() function will do what you want. While it is technically not core, LWP::Simple is included by default with many Perl distributions. You may well already have it.
Update: so, you want to do this on 1000 servers? No need to manually install the module on each server. Use CPAN programmatically to download and install the module(s) you need (some tweaking will be needed to get CPAN to install it locally rather than in the root module library). Also Leon Timmermans's suggestion of fatpacking the module is another option.
If you really don't want to do it this way, then basically the answer is no: there is no simple way to fetch a remote file via HTTP using neither the appropriate modules, nor a system command (I didn't consider writing your own HTTP client to be a simple method, but that's fine if it works for you).
The only other potential solution I see would be a different approach to your problem, such as:
Using a script in a single location to get the file, then distribute
it to all 1000 servers via FTP.
Or, putting the file on an FTP server, then using a simple Perl
script on each server to fetch it via FTP.

As Dan already said, yes, even you can use CPAN. One approach his link doesn't mention is writing it as a normal CPAN-using distribution, and then fatpack it. Fatpacker combines a script with all its (pure-perl) dependencies, creating a single easy to distribute file.

You could use:
my $wgetoutput = `wget "$myFileToGet"`;
Stuff in backticks (`) will be given to the default shell, so you can call whatever you want (and are allowed) there.
Caveat: $myFileToGet could have stuff like "&& rm -rf *" in it, so dont forget to sanitize!

Related

Giving Credit for Perl Module

I wrote a script for my company and I am using some libraries I obtained from CPAN. My manager wanted me to consolidate and remove the extra libraries - which is a little funny because I include them for the script to work.
A few notes:
I do not have root access on this server nor can I request access
To use CPAN modules w/o root I have them installed to my user directory
To allow other users to run my scripts I usually include a folder called 'libs' and inside of my script's directory and in the script I have: use 'libs'; at the top before I use my CPAN modules.
The only solution I have right now is to literally put the contents of the perl modules inside of my perl script. However I want to give credit where it is due and also not get in trouble for including opensource code w/o proper credit to its authors and organizations.
Therefore, how should I go about this? I am not trying to get away with anything.. I honestly want to go about doing this the right way.
All three modules say "licensed under the same terms as Perl itself" but I feel like it shouldn't be this easy.
I would also like to explore any other ideas too!
The modules are:
Text::Table
Text::Aligner
Term::ANSIColor
Is using PAR Packager an option for you? That would generate a standalone executable.
If the modules are pure Perl modules, you may be able to simply append the code (including those package statements) into your program. I'd also include the POD which would include the copyright statements and the names of the authors too. That should satisfy the Artistic License Requirement (but may not satisfy GNU licensing requirements).
Another possibility is to use Perlbrew which will allow you to install a user version of Perl on the system. This way, you can install CPAN modules without needing Administrative permission, and you can tell other users to use Perlbrew too.
I use it because I can install and switch between various versions of Perl which allows me to test my Perl scripts in various versions of Perl. I've also used it on our servers where I need a newer version of Perl or modules that weren't included in the standard release.
You need to get your IT approval before installing Perlbrew, but a lot of times they're relieved that they no longer have to be bothered with maintaining and installing CPAN modules for your use.
Interesting question & perspective. I don't understand what is against using libraries or modules, but I'll let your manager do the thinking ;-)
Regarding copyright, you're best to consult a lawyer if you want to be sure, but as far as I understand it, you can combine the work of others provided you retain the copyright notices. The combined work may not be covered by copyleft, so you may be able to use it commercially (i.e., distribute it without disclosing the source). But do check with a lawyer.
But, since you said you wanted to explore other ideas, App::Staticperl may be a solution? I do not have experience with it, but I tried it with a simple example and got a working executable.
App::Staticperl builds a stand-alone executable from the Perl interpreter with embedded CPAN modules. The steps I followed were roughly (you'll need to adapt, because obviously I couldn't test with your script):
latest version of App::Staticperl is 1.43: https://cpan.metacpan.org/authors/id/M/ML/MLEHMANN/App-Staticperl-1.43.tar.gz
either install the module via CPAN, or simply extract bin/staticperl from the tar - it's a standalone script
edit staticperl to change EMAIL and CPAN (optional, but you may want to change the CPAN mirror)
./staticperl install downloads and builds Perl; it ended with an error message on my box, but did produce a working Perl
./staticperl cpan enters an interactive CPAN prompt; install Text::Table, install Term::ANSIColor, and whatever else you need
./staticperl mkapp my_app --boot path/to/your/script -MText::Table -MText::Aligner -MTerm::ANSIColor
try the app: ./my_app - it will most likely fail with an error message about missing modules; repeat the previous step and include the missing modules in the -M flags
Good luck!
Can you reduce the unnecessary code (to satisfy your manager's concerns). Leave in tact the needed code in the file it came in - and give the author's credit within that module/package.
Eg: This was inspired (stolen) from Joe E Perl.

How to get/put files from SFTP server using Net::SSH::Perl module?

It might be simple query, but still I'm not clear regarding this.
In my script I'm connecting to SFTP client server using Net::SSH::Perl module. Previously I was using Net::SFTP but removed from the script, as all of a sudden it stopped working and throwing an error.
When I used Net::SFTP, most often I will use the below command to Put/get files from remote server.
$sftp->put("/home/ftpford/ftpcon/conout/$file","/Uti_Integrator/READYFORPICKUP/PENDING/$file");
But I'm not sure of how to get/put files using Net::SSH::Perl.
Can any one suggest? I have tried many ways and even I tried to search in Google but I'm not clear of any thing.
And please note that I don't have privileges to install new modules in my server.
I want to get/put files using the above Module.
I don't think that's a standard feature of Net::SSH::Perl.
One option is to use other modules that do (you don't necessarily need to install them in the system directories, you can have them in your own directories if you include them via PERL5LIB or use lib.
The other option (which answers your question stricto sensu) is to emulate it. You can try to simply run cat >$destination_file on the remote box via Net::SSH::Perl, and then send the contents of the file over the ssh connection.
Of course, error handling and the like might not be very straightforward...
BTW, you tell us you have tried many things, but you don't tell us which, and what problems you encountered.

How to run multiple Perl installs on one machine?

Is it possible to run multiple installs of Perl (in "containers") on one machine?
Reason is that I have different Perl-based server side web applications and wish to schedule updates to them independently.
For example, bugzilla upgrades seem to me to be very invasive, downloading all manner or module updates and lengthy, too (thereby increasing the chance of unpredictable behavior on other applications that depend on those modules, during the time that the upgrade is still partial).
I think it should be possible to run multiple independent server-side CGI Perl applications on one server, I'd rather not be told to separate them onto different machines - I think that's wasteful and I don't have that resource anyway.
Investigate PerlBrew and cpanm:
http://qa.celogeek.com/programming/perl/for/developer/overview
Edit, more info:
http://www.bryanesmith.com/documents/a2pm/perlbrew-june-14-2011.pdf
http://www.dagolden.com/index.php/1384/parallel-make-for-perlbrew/
http://www.perlbrew.pl/
It's easy to install and manage multiple perls. Simply install them in different places and use each perl's tools. I talk about this in The Effective Perler.
Some people suggest perlbrew without realizing that it doesn't really give you any benefit. It can download a perl, configure and install it, and switch around symbolic links to make one of those the default. It doesn't do anything magical, though.
Downloading and installing aren't a problem though. You've never needed root or sudo to do that, and if you do, you'll still need it for perlbrew. You can always install into any directory where you have permission. perlbrew doesn't get around that at all. From the source directory, you have two simple commands to run:
$./Configure -des -Dprefix=/where/you/want/to/install
$ make install
For you, that might mean Bugzilla gets its own perl:
$./Configure -des -Dprefix=/where/you/want/to/install/bugzilla-perl
$ make install
From there, you have a completely self-contained perl installation. When it matters to me which perl I use, I give the program the full path to it:
#!/where/you/want/to/install/bugzilla-perl/bin/perl
It's much easier to make these per-applications installations without perlbrew, which wants to do as much as it can for you, including deciding the directory name, which it prefers you didn't know at all.
perlbrew's main advantage is not the compilation and installation, but it's switch feature to let you make one perl the default. You probably don't want that feature though because you want bugzilla, CGI programs, and so on using only the perl you want them to use, not whatever default perl you last specified.
When you want to update the bugzilla-perl, just use it's tools, which already have adjusted shebang lines to find the right perl:
$ /where/you/want/to/install/bugzilla-perl/bin/cpan ...
I don't like all of those long paths, though, which is why I make links to them all. Then I can just call them with whatever naming scheme I decide, which might be:
$ bugzilla-cpan ...
There's never a question about which tool or version I'm using.

How do I install Perl modules on machines without an Internet connection?

I need to install my Perl-based software on networked machines which aren't connected to the internet. Therefore, I would like to download specific versions and/or latest versions of the Perl modules and I would also like to know if there is an install procedure required for these modules.
Background:
The machines aren't connected to the internet for security reasons and its deemed unnecessary also.
I would place the downloaded modules on a machine that I call the 'install server' and it contains my Perl based software and would also contain the local copies of the Perl modules.
I call a machine that I want to install my Perl-based software on, the 'target machine', also not connected to the internet. There can be several target machines, each can run this software that I want to install. I log onto the target machine and run an install script which would connect to the install machine via the local network to obtain the Perl-based software and dependent Perl modules and installs them.
So I need to know:
How/Where to get specific versions of Perl modules, e.g. CGI.pm etc
How to install these Perl modules. Is it a case of just placing them in a directory somewhere, e.g. a library path and making sure that this directory path is in the #INC library path environmental variable, if it is not already?
I would prefer not to have to do anything like make install etc. as part of installing the modules. I would like to modules to be pre-compiled or prepared as necessary so it is as simple as possible to install them. I want to avoid additional dependencies like make and its configuration, and having to parse its output to check whether it was successful.
Please help me by asking the above specific questions as I am not able to change the concept of 'install machine' and 'target machine' which aren't connected to the internet - I have to provide a solution that works within this arrangement.
The usual way to solve "I want to install stuff from CPAN but without network" problems is to use a minicpan as David Dorward wrote in his answer. But since you're going one step further, saying that you'd rather not do any real installation on the client (target) machines at all, and that you want to use precompiled modules if possible, I urge you to check out PAR and specifically PAR::Repository (server) and PAR::Repository::Client.
Since this approach needs some research before you're up to speed, I wouldn't suggest it for "I just need Foo.pm" like problems. Once you're talking about a handful of dependencies and at least a handful of clients, then it becomes a more appropriate solution.
For an outline of how it works, check out the slides of my talk at YAPC::EU 2008. It also hints at solutions to the bootstrapping problem of making the PAR::Repository::Client module available on the clients (hint: PAR can generate self-contained executables).
You can create a MiniCPAN that has just the latest versions of everything from CPAN. You can insert additional, non-public modules into it with CPAN::Mini::Inject. If you need to greater control over versions (i.e. not choosing the latest versions), you might want to create a DPAN.
With any of these solutions, you can configure your CPAN client to pull from your local source. That could be a directory you know ahead of time or something that you figure out dynamically, like a CD or a thumb-drive. It's just a matter of setting up the configuration correctly.
You might be able to get away with creating operating-system packages for most of your work, but that still means you have to compile them at least the first time.
1) How/Where to get specific versions of Perl modules, e.g. CGI.pm etc
http://search.cpan.org/
If you don't want the latest version, you can get an earlier version by following the link in the breadcrumbs.
http://img.skitch.com/20091209-bu7kt3bj65374k7iijfnhrue2y.png
2) How to install these Perl modules. Is it a case of just placing them
in a directory somewhere, e.g. a library path and making sure that this
directory path is in the #INC library path environmental variable, if
it is not already?
That sometimes work, but you really should go through the perl Makefile.PL && make && make test && make install process.
Doing this would require that you manually chase all the dependencies though. You would probably be better off with something like minicpan.

How do I tell CPAN.pm to use an application-specific mini-cpan?

Is there a way to use app-specific mini-CPANs (to ensure specific module versions for different apps and make an install work without net access). The 'standard' mini-cpan way of things relies on setting 'o conf urllist' with a file url in your CPAN/Config.pm
That's fine with one single mini-cpan. However, I need to automate having a CPAN install from a different directory for each app (local to the app), as each app has different version requirements.
The following almost works, but then has only a partially working shell, and still fetches from the web instead of using the mini-cpan from $file_url location:
use CPAN;
use FindBin qw($Bin);
my $file_url="file:///$Bin/../cpan";
CPAN::Shell->o(qw( conf urllist ),$file_url);
CPAN::shell();
You load a different configuration file for each application. The latest cpan command has a -j switch to do just that:
$ cpan -j some/app/Config.pm -i Foo::Bar
This latest feature isn't included in the CPAN.pm distribution yet since it's experimental. I've been using it for several months, however, as part of my DPAN work.
If you don't like any of that, you just have to provide your application-specific CPAN::Config module somewhere Perl will find it before it finds your personal or site-wide version. Copy your current CPAN/Config.pm into your application modules directory and ensure that the directory is first in #INC (perhaps using the PERL5LIB environment variable). CPAN.pm should find that one first and use it. It only uses the first one it finds. This is handy when the configuration is fixed, although it's a bit flexible since you can run code in the module. It's configuration as Perl code.
If that is not good enough for you, you can override CPAN::HandleConfig() in your application so CPAN.pm doesn't load any files. You then set the values yourself. This is especially handy when you have a lot of application logic to resolve to figure out some of the configurations values. The CPAN.pm configuration is just the %CPAN::Config hash, so you just have to set the right values for the appropriate keys.
In any case, you probably want to set the "connect_to_internet_ok" configuration to a false value and clean out your "urllist" to have only the local minicpans.