How does PAR::Packer work? - perl

I was using PAR::Packer and this question popped up in my mind. How does PAR::Packer work in Perl? Does it actually compile the Perl script to .exe like g++ compiles C++ Sources to .exe or does it work like py2exe in Python that packs the interpreter and the script into an .exe?

To make this absolutely clear:
Tools like PAR::Packer do not “compile” your Perl script. They bundle the perl interpreter together with your source files and any required modules into a big fat executable file. When it is run, the original sources are extracted and fed to the enclosed perl.
This works reasonably well, but does not yield a speed improvement (on the contrary…). The only advantage is that you can distribute your programs as a single (albeit quite large) file, without dependencies.
There is a very experimental tool called perlcc that is able to translate some Perl programs to C or a Perl bytecode serialization. As the docs put it:
The code generated in this way is not guaranteed to work. The whole codegen suite (perlcc included) should be considered very experimental. Use for production purposes is strongly discouraged.
This is because the Perl language does not support static compilation. It needs to be able to execute code during parsing for some dynamic features during the same session where the main execution phase takes place.
There are other, commercial tools, that usually fall in the same category as PAR::Packer (creating fat executables).
Summary: If you want a single executable, use PAR::Packer. If you want speed, inline some C (or use XS). There is no tool that can compile all Perl scripts to machine code.

I was using PAR::Packer and this question popped up in my mind. How
does PAR::Packer work in Perl?
Does it actually compile the Perl
script to .exe like g++ compiles C++ Sources to .exe
no pp and perl2exe doesn't. (though pp is free).
but it looks like perlcc does
or does it work
like py2exe in Python that packs the interpreter and the script into
an .exe?
pp and perl2exe, yes
As an example- sendemail.exe that looks like something done in PAR Packer or Perl2exe
It packs the interpreter into the exe.
You can open sendemail.exe in 7-zip! There are some folders there but one can't really see it actual files.
I suppose it's some form of self extracting executable, but, that executes code.
You can monitor it with process monitor, and you see.
Or with, process explorer and see if it uses any dll from the temp directory that it creates..
It creates a temp directory e.g.
C:\Users\user\AppData\Local\Temp\pdk-user <--- Win7
or
C:\Documents and Settings\user\Local Settings\Temp\pdk-user <-- WinXP
The temporary directory it creates contains a bunch of strange named DLLs and a DLL called perl58.dll which is no doubt, the perl interpreter
I'm sure at one point I saw it had a dll there with a normal name. SSLEAY32.DLL and interestingly when I ran the pl file it used a dll with a similar name from my c:\perl64 directory. So the EXE looks like a bit of a hack really. It's more reliable to run the pl file directly.
DIR /s/b within that directory, shows-
178c2b604baa8a7f1ebc80539f378bff.dll
1823e8f62785746fd29cf0b06c636600.dll
465d2954d90fe6225ea61b3907c91da8.dll
6145f78a34d5ced8200800f1455d578a <-- the directory with the perl58. dll
9c50b5816b0e35f047e41f5899721d46.dll
f4e2e0db77ed1e6572c2f490479cd815.dll
f72f556d99dfb6b0c3bb37f123e2ee96.dll
6145f78a34d5ced8200800f1455d578a\perl58.dll
no normal named DLLs showing other than perl58.dll (though I have seen a normal one there in the past).
if you look in process explorer, you see it using perl58.dll and other normal named dlls
If you look at the PAR website, a page describes perl2exe
Perl2Exe is commercial, command-line application that can build
standalone executables from perl sources. It works by creating an
executable that contains
A standalone perl interpreter (that is capable of grokking perl 5.8.x)
Your perl script and All perl modules that are referenced by your perl
script.
pp - PAR Packager provides that same functionality, but is free.
And if you look here
http://www.perlmonks.org/?node_id=237943
...the perl2exe tool is not a way to hide your source code.
Now it's even come to the attention of the security community (reported in bugtraq, for example).
For details, see the report from net-security's page.
Please stop supporting perl2exe. Please use PAR for a complete installation package, or perlcc to simply compile the top-level program.
-- Randal L. Schwartz, Perl hacker
So, PP and perl2exe combine the interpreter into the EXE, And you can get an idea that there's something funny atypical about the EXE when it opens in 7zip!
perlcc compiles properly.
it appears to have been unmaintained for a while http://www.perlmonks.org/?node_id=654568 , people said it was buggy, but some work was done on it as recently as June 2014 http://www.yapcna.org/yn2014/talk/5603 It gets better performance than interpreted perl. (unlike the EXEs that pack an interpreter, which are slower than running a perl script normally).

Related

How to solve dll missing in perl script?

I installed perl 5.14 version just before, when I executed script display errors perl58.dll missing. How can solve the problem..
how can I solve this problem?
It sounds to me that the thing you're describing as a 'perl script' is actually an executable built from a perl script using ActiveState's perlapp with the --dependent flag. By default, perlapp executables are self contained, but the perl{version number here}.dll can be left out to save space.
Possible courses of action (roughly in order of ease + sanity):
Get the original source from the guy who wrote it, & just run it.
Get the guy who wrote it to update the perlapp executable to a non-dependent version or a version that runs under your installed version of ActivePerl.
Pony up for ActiveState's 'business' subscription to download & install ActivePerl 5.8 (and no other version) on that machine.
Find a copy of the perl58.dll and place it where windows will find it. (may require subscription, or the guy who built the perlapp may have a copy lying about)
Extract the original script from within the .exe and either build a newer (and non-dependent) perlapp or just run it from the source as Larry intended.
Depending on what options were used to build the perlapp executable you may have the origional source lying about in a temp directory. Otherwise I believe you may be able to open it up with a program that understands .zip files, but I've never tried this.
It is also possible that you've got a directory in your path that contains a v5.8 perl.exe but not it's accompanying .dll and that you've asked your newly installed v5.14 perl not to place its directories at the beginning of your path. I don't expect this is the case.

Perl to generate one executable file for a script which uses any number of modules and libraries

I am working on creating an agent in perl which does several actions. It uses several modules which are in .pm format and also few libraries. Now i want to convert it as one executable file so that i can install in n number of servers by copying that single file. Is it something i can achieve in perl? I am just a beginner in perl, perhaps my question might sound dumb but it will teach me something.
pp script provided with PAR::Packer is able to create single-file executables. An example from its page:
pp -o foo foo.pl bar.pl # Pack 'foo.pl' and 'bar.pl' into 'foo'
Some modules are included with Perl, so even though they're separate modules, they will work on other Perl installs without installing those modules. These include File::Copy, File::Find, Time::Piece.
You can see the listing of all standard modules on the Perldoc home page. Be sure to set the drop down version field (located on the left side) to the version that you're using. It goes all the way back to Perl 5.8.8 which is on Solaris.
It is entirely possible that the modules you need are already included in the standard Perl distribution, so there's no need to worry. Sometimes, you can substitute a non-standard module that's being used for one that's a standard module with little rewriting.
Some modules include compiled C code and can't be redistributed. They must be compiled on the machine they'r running on and installed. However, most modules are pure Perl modules, and can be redistributed with a program.
If a module isn't a standard module, and it's a pure Perl module, there are two ways it can be redistributed:
Perl has an #INC list that says what directories to search for when you search for modules. There's a Perl use lib pragma that allows you to add directories. You could include modules as sub directories for your program, and then zip up the entire structure. Users would unzip the entire directory tree which would include your program and the modules you need. By the way, the default #INC usually includes the current directory.
The other way is to append the modules to your program and then remove the use statement for that module (since it's now part of the file). This is a bit tricky, but it means a single program file.
Just remember that a module might require another module, so check thoroughly.
Another thing you can do is check for the module, and if it isn't there, download it via CPAN. Testing is easy:
BEGIN {
eval {
require My::Module; Module->import( LIST );
};
if ($#) {
die qq(Module doesn't exist);
}
}
Of course, doing a die is sort of silly because use would do that. However, it might be possible instead of dying to load the module via the CPAN module programmer's interface. I've never done that, and I don't know people who have. But, it is possible.
So, your best bet is to check to see if your program uses standard Perl modules, and if not, see if you can modify the program to use them. For example, if your program uses Archive::Zip, you might be able to modify it to use IO::Uncompress::Unzip and IO::Compress::Zip instead.
Otherwise, your choice is to try include those modules for installation (and watch for recursiveness and non-Pure Perl modules) or to try to detect that a module isn't installed, and programmatically install it.
The answer is a bit complicated.
The nature of Perl makes it practically impossible to compile a perl script in most use cases, so that a single executable could be distributed (with executable in the Windows sense). There are ways to do something similar, but sadly I don't know them.
But you can actually embed the Perl interpreter inside any C application, including the Perl source (your scripts + modules). When you statically link all C libraries, this should work as well. You can then use the Perl API to call your scripts.
If all of the servers you target are guaranteed to run the exact same OS, using the exact same libraries, and are preferably a *nix of some sort, it would be possible to pack all required files into an archive and write an install script. It is possible to write self-extracting shell scripts that contain the archive they are about to unpack. Same goes with perl, using the special __DATA__ command and the DATA filehandle:
#!/usr/bin/perl
print for <DATA>;
__DATA__
1
2
3
prints
1
2
3
Works great for piping data to tar as well.
You should include all dependent modules and all compiled libraries into the file and figure out a metadata system to install all files to the correct place.
As a general rule, software should rather be compiled on the target system itself, than just copying the binary files. It is too easy to overlook architecture differencies, configuration files or special registration entries hidden from view.
If you have to target different systems, it might be better to write a script that delegates the bulk of the installation to cpan or whatever perl package manager you prefer. This will be more flexible than hard-coding filepaths.
#!/bin/bash
cpan install Foo::Bar
cpan install Acme
cpan install ...
# etc.
I would stick with that.
The most elegant solution would be to create your own package or distribution like the ones you download from CPAN. As you would include a metadata file referencing all your dependencies, cpan would figure out everything by itself and do possibly neccessary compilation. I don't think this exactly is a beginners topic, but it would give you max flexibility and maintainability (easy upgrades!). This should make it fairly easy to include some installation tests.
This is just for starters, I am sure the internet or somebody else with more knowledge will elaborate.

Can I move a Perl installation from one computer to another computer?

I am trying to set up an application dependant on few Perl modules, but the server I am installing to, does not have Internet connection. I read about offline module installs via ppd files, however I would have to resolve all the dependencies one by one.. All the more tedious considering I don't have direct internet connection.
I am hoping to find a solution, where I install ActivePerl on my PC and install all the libraries that I want and then copy paste the directories to my server. If it is just a matter of fixing some environment variables, that would be fine. Just want to know the definitive list of variables to modify. Not sure whether it is mandatory to install the perl libraries on the computer in which it is intended to run? (One is 32 bit platform and other one is 64 bit, but the server is already running various 32 bit applications so I hope it is not a major problem) For best compatibility, I plan to install ActivePerl on both the systems and merge the library directories to be identical.
The answer was on Perl FAQ, my bad didn't go through it properly.
I copied the perl binary from one machine to another, but scripts don't work.
That's probably because you forgot libraries, or library paths differ.
You really should build the whole distribution on the machine it will
eventually live on, and then type "make install". Most other approaches
are doomed to failure.
One simple way to check that things are in the right place is to print
out the hard-coded #INC that perl looks through for libraries:
% perl -le 'print for #INC'
If this command lists any paths that don't exist on your system, then
you may need to move the appropriate libraries to these locations, or
create symbolic links, aliases, or shortcuts appropriately. #INC is also
printed as part of the output of
% perl -V
You might also want to check out "How do I keep my own module/library
directory?" in perlfaq8.
From this link
Occasionally, you will not be able to
use any of the methods to install
modules. This may be the case if you
are a particularly under-privileged
user - perhaps you are renting web
space on a server, where you are not
given rights to do anything.
It is possible, for some modules, to
install the module without compiling
anything, and so you can just drop the
file in place and have it work.
Without going into a lot of the
detail, some Perl modules contain a
portion written in some other language
(such as C or C++) and some are
written in just in Perl. It is the
latter type that this method will work
for. How will you know? Well, if there
are no files called something.c and
something.h in the package, chances
are that it is a module that contains
only Perl code.
In these cases, you can just unpack
the file, and then copy just the *.pm
files to a directory from which you
will run the modules. Two examples of
this should suffice to illustrate how
this is done.
IniConf.pm is a wonderful little
module that allows you to read
configuration information out of a
.ini-style config file. IniConf.pm is
written only in Perl, and has no C
portion. When you unpack the .tar.gz
file that you got from CPAN, you will
find several files in there, and one
of them is called IniConf.pm. This is
the only file that you are actually
interested in. Copy that file to the
directory where you have the Perl
programs that will be using this
module. You can then use the module as
you would if it was installed
``correctly,'' with just the line:
use IniConf;
Time::CTime is another very handy
module that lets you print times in
any format that strikes your fancy. It
is written just in Perl, without a C
component. You will install it just
the same way as you did with IniConf,
except that the file, called CTime.pm,
must be placed in a subdirectory
called Time. The colons, as well as
indicating an organization of modules,
also indicates a directory structure
on your file system.

How do I start a new Perl module distribution?

I'm trying to set up a large-ish project, written in Perl. The IBM MakeMaker tutorial has been very helpful so far, but I don't understand how to link all the modules into the main program. In my project root, I have MANIFEST, Makefile.PL, README, a bin directory, and a lib directory. In my bin directory, I have my main script (Main.pl). In the lib directory, I have each of my modules, divided up into their own respective directories (i.e. Utils::Util1 and Utils::Utils2 in the utils directory, etc). In each module directory, there is also a t directory, containing tests
My MANIFEST file has the following:
bin/Main.pl
lib/Utils/Util1.pm
lib/Utils/Util2.pm
lib/Utils/t/Utils1.t
lib/Utils/t/Utils2.t
Makefile.PL
MANIFEST
README
Makefile.PL is the following:
use ExtUtils::MakeMaker;
WriteMakefile(
'NAME'=>'Foo',
'VERSION_FROM'=>'bin/Main.pl',
'PREREQ_PM'=>{
"XML::Simple"=> 2.18}, #The libraries that we need and their
#minimum version numbers
'EXE_FILES' =>[("bin/Main.pl")]
);
After I make and run, the program crashes, complaining that it cannot find Utils::Util1, and when I run 'make test, it says no tests defined. Can anyone make any suggestions? I have never done a large scale project like this in perl, and I will need to add many more modules
If you are just starting to create Perl modules (which is also Perl's equivalent of a project), don't use Makemaker. Module::Build is the way to go, and it's now part of the standard library. Makemaker is for us old salts who haven't converted to Module::Build yet. :) I'll strike that now that Module::Build is unmaintained and out of favor; I still use MakeMaker.
You should never start off a Perl project by trying to create the structure yourself. It's too much work and you'll always forget something.
There's h2xs, a program that comes with perl and was supposed to be a tool to convert .h files into Perl's glue language XS. It works fine, but its advantage is that it comes with perl:
% h2xs -AXn Module::Name
Something like Module::Starter is a bit more sophisticated, although you have to get it from CPAN. It's the tool we use in Intermediate Perl because it's simple. It fills in some templates with your information:
% module-starter --author=... --email=... --module=...
If you are doing to do this quite a bit, you might then convert that to Distribution::Cooker so you can customize your files and contents. It's a dinky utility I wrote for myself so I could use my own templates.
% dist_cooker Module::Name
If you're really hard core, you might want Dist::Zilla, but that's more for people who already know what they are doing.
Might I also suggest module-starter? It'll automatically create a skeleton project which "Just Works". I learned what little I know about Perl modules organization by reading the generated skeleton files. It's all well-documented, and quite easy to use as a base for growing a larger project in. You can check out the getting-started docs to see what it gives you.
Running module-starter will give you a Perl distribution, consisting of a number of modules (use the command line option --module, such as:
module-starter --distro=Project --module=Project::Module::A,Project::Module::B [...]
to create multiple modules in a single distribution). It's then up to you whether you'd prefer to organize your project as a single distribution consisting of a number of modules working together, or as a number of distributions which can be released separately but which depend on each other (as configured in your Build or Makefile.PL file) to provide a complete system.
Try this structure:
bin/Main.pl
lib/Utils/Util1.pm
lib/Utils/Util2.pm
Makefile.PL
MANIFEST
README
t/Utils1.t
t/Utils2.t
As ysth said, make does not install your modules, it just builds them in a blib directory. (In your case it just copies them there, but if you had XS code, it would be compiled with a C compiler.) Use make install to install your modules for regular scripts to use.
If you want to run your script between make and make install, you can do:
perl -Mblib bin/Main.pl
The -Mblib instructs perl to temporarily add the appropriate directories to the search path, so you can try out an uninstalled module. (make test does that automatically.)
By default, tests are looked for in a top-level t directory (or a test.pl file, but that has some limitations, so should be avoided).
You say "After I make and run"...make puts things into a blib directory structure ready to be installed, but doesn't do anything special to make running a script access them. (make test is special; it does add appropriate paths from blib to perl's #INC to be able to run the tests.) You will need to do a "make install" to install the modules where your script will find them (or use a tool like PAR to package them together with your script).

Is there a way to package my unit tests with PAR or PerlApp?

I have an app that I pack into "binary" form using PerlApp for distribution. Since my clients want a simple install for their Win32 systems, this works very nicely.
Now a client has decided that they need to run all unit tests, like in a standard install. However, they still will not install a normal Perl.
So, I find myself in need of a way to package my unit tests for operation on my client's systems.
My first thought was that I could pack up prove in one file and pack each of my tests separately. Then ship a zip file with the appropriate structure.
A bit of research showed that Test::Harness::Straps invokes perl from the command line.
Is there an existing tool that helps with this process?
Perhaps I could use PAR::Packer's parl tool to handle invocation of my test scripts.
I'm interested in thoughts on how to apply either PAR or PerlApp, as well as any thoughts on how to approach overriding Test::Harness and friends.
Thanks.
Update: I don't have my heart set on PAR or PerlApp. Those are just the tools I am familiar with. If you have an idea or a solution that requires a different packager (such as Cava Packager), I would love to hear about it.
Update 2: tsee pointed out a great new feature in PAR that gets me close. Are there any TAP experts out there that can supply some ideas or pointers on where to look in the new Test::Harness distribution?
I'm probably not breaking big news if I tell you that PAR (and probably also perlapp) aren't meant to package the whole test suite and plethora of CPAN-module build byproducts. They're intended to package stand-alone applications or binary JAR-like module libraries.
This being said, you can add arbitrary files to a PAR archive (both to .par libraries and stand-alone .exe's) using pp's -a switch. In case of the stand-alone executable, the contents will be extracted to $ENV{PAR_TEMP}."/inc" at run-time.
That leaves you with the problem of reusing the PAR-packaged executable to run the test harness (and letting that run your executable as a "perl"). Now, I have no ready and done solution for that, but I've recently worked on making PAR-packaged executables re-useable as more-or-less general purpose perl interpreters. Two gotchas before I explain how you can use that:
Your application isn't going to magically be called "perl" and add itself to your $PATH.
The "reuse" of the application as a general purpose perl requires special options and does not currently support the normal perl options (those in perlrun). It can simply run an external perl script of your choice.
Unfortunately, the latter problem is what may kill this approach for you. Support for perl command line options is something I've been thinking about, but won't implement any time soon.
Here's the recipe how you get PAR with "reusable exe" support:
Install the newest version of PAR from CPAN.
Install the newest, developer version of PAR::Packer from CPAN (0.992_02 or 03).
Add the "--reusable" option to your pp command line.
Run your executable with the following options to run an external script "foo.pl":
./myapp --par-options --reuse foo.pl FOO-PL-OPTIONS-HERE
Unfortunately, how you're going to teach Test::Harness that "./myapp --par-options --reuse" is a perl interpreter is beyond me.
Cava Packager allows you to package test scripts with your packaged executables. This is primarily to allow you to run tests against the packaged code before distribution. However the option is there to also distribute the tests and test capability to your end users.
Note: As indicated by my name, I am affiliated with Cava Packager.