This is probably a multi-part question. Background: we have a native (c++) library that is part of our application and we have managed to use SWIG to generate a perl wrapper for this library. We'd now like to distribute this perl module as part of our application.
My first question - how should I distribute this module? Is there a standard way to package pre-built perl modules? I know there is ppm for the ActiveState distro, but I also need to distribute this for linux systems. I'm not even sure what files are required to distribute, but I'm guessing it's the pm and so files, at a minimum.
My next question - it looks like I might need to build my module project for each version of perl that I want to support. How do I know which perl versions I should build for? Are there any standard guidelines... or better yet, a way to build a package that will work with multiple versions of perl?
Sorry if my questions make no sense - I'm fairly new to the compiled module aspects of perl.
CLARIFICATION: the underlying compiled source is proprietary (closed source), so I can't just ship source code and the appropriate make artifacts for the package. Wish I could, but it's not going to happen in this case. Thus, I need a sane scheme for packaging prebuilt binary files for my module.
I look after DBD::Informix, one of the Perl Database Driver modules that works with the DBI (Perl Database Interface). The underlying libraries used to connect to IBM Informix Dynamic Server (IDS) are proprietary, but the DBD::Informix code itself is not. I distribute that code on CPAN, just the same as any other Perl module. People can download that source, and (provided that they have the Informix ClientSDK installed on their machine - and Perl and DBI and so on), they can build DBD::Informix to work with their installed Perl.
I would strongly counsel that you arrange that your Perl interface code be made available in source form, even though the library that it interfaces to is proprietary. This allows people to install the code with any version of Perl they have - without requiring you to deal with inconsistencies.
If you still want to provide binary support, you are going to have to work out which platforms you want to support, and build the module with the standard version of Perl on each such platform. This gets messy. You need access to an instance of each machine. Granted, virtual machines make this easier, but it is still fiddly and the number of platforms and versions only grows. But you still need to support people who don't use the standard version of Perl on their machine - that's why the Perl wrapper interface needs to be provided in source form.
DISCLAIMER: I have next to no experience creating binary packages that can easily be installed. Therefore, I am making this post CW to make it easier for others to add their advice.
You should make the distribution available in source form so it can be compiled on each system tailored according to the specifics of that system. I really like Module::Build for that purpose.
For ActiveState users on Windows, you probably want to have four or six PPMs based on whether you want to support 5.6. Package both 32-bit and 64-bit versions for each of 5.6, 5.8 and 5.10. Use the version of mingw you can install using ppm to compile the modules to preserve binary compatibility.
Another option is to use PAR::Packer and distribute your application in a PAR archive. In that context, PAR::WebStart might be useful although I have not tried it. I have had success with PAR archives in the past, though.
Related
I am a very new to porting.
I was trying to port perl to a netbsd system. Since its a custom made build, we wont be able to run configure or make on the target netbsd system. So we are trying to cross-compile it in a host pc and copy the binary over target machine. And in order to do so, we have to make a makefile from scratch, since the format for the makefile in our build is different.
I have some basic doubts regarding this,
Firstly, In order to create a perl makefile for my custom build, what are the basic things will come. Such as ccflags, library paths etc.,?
There are some files like DynaLoader, uudmap.h, myConfig, Config.pm which gets generated while "make". How can i generate them using custum makefile.
How to set various library paths and what are they ?
The #INC, shows the perl search paths, how can i create it ?
Where exactly Perl modules get installed and when it happens?
A perl build normally involves building a stripped down version of perl named miniperl, which is then used extensively in the remainder of the process of building perl and the bundled modules.
There are two basic approaches to cross-compiling: to build miniperl for the target machine and build the modules, etc., there, or to build miniperl for the host and use it to build perl and modules for the target.
The WinCE port uses the latter approach; the rudimentary (last I knew, anyway) support for a -Dusecrosscompile switch to Configure uses the former.
I recommend you ask for advice and help on the perl5-porters mailing list: http://lists.perl.org/list/perl5-porters.html
And be prepared for hard work.
NetBSD's pkgsrc system has perl in it already and has the ability to generate binary packages that you can then install on a target machine.
I've written any number of perl modules in the past, and more than a few stand-alone perl programs, but I've never released a multi-file perl program into the wild before.
I have a perl program that is almost at the beta stage and is going to be released open source. It requires a number of data files, as well as some external perl modules -- some I've written myself, and some from CPAN -- that I'll have to bundle with it so as to ensure that someone can just download my program and install it without worrying about hunting for obscure modules.
So, it sounds to me like I need to write an installer to copy all the files to standard locations so that a user can easily install everything. The trouble is, I have no idea what the standard practice would be for this. I have found lots of tutorials on perl module standards, but none on perl program standards.
Does anyone have any pointers to standard paths, installation proceedures, etc, for perl programs? This is going to be complicated by the fact that the program is multi-platform. I've been testing it in Linux, but its designed to work equally well in Windows.
Take a look at PAR and PAR::Packer. You can bundle all of your requirements (even non-Perl requirements) into one file. With PAR::Packer, the user doesn't even need to have Perl installed for it to work.
You might also look at how the various App::* distributions are setup.
The standard installers for modules (ExtUtils::MakeMaker, Module::Build, Module::Install) also work the same way for scripts.
Using such a standard Perl tool will help you to:
distribute your application on the
CPAN (and you'll benefit from
automated tests on various platforms
by CPAN Testers), and so your app
will be installable (with all its dependencies) from the CPAN
shell
help packagers of Linux/BSD distributions to make packages for your product
Being very new to Perl but not to dynamic languages, I'm a bit surprised at how not straight forward the manage of modules is.
Sure, cpan X does theoretically work, but I'm working on the same project from three different machines and OSs (at work, at home, testing in an external environment).
At work (Windows 7) I have problem using cpan because of our firewall that makes ftp unusable
At home (Mac OS X) it does work
In the external environment (Linux CentOs) it worked after hours because I don't have root access and I had to configure cpan to operate as a non-root user
I've tried on another server where I have an access. If the previous external environment is a VPS and so I have a shell access, this other one is a cheap shared hosting where I have no way to install new modules other than the ones pre-installed
At the moment I still can't install Template under Windows. I've seen that as an alternative I could compile it and I've also tried ActiveState's PPM but the module is not existent there.
Now, my perplexity is about Perl being a dynamic language. I've had all these kind of problems while working, for example, with C where I had to compile all the libraries for all the platform, but I thought that with Perl the approach would have been very similar to Python's or PHP's where in 90% of the cases copying the module in a directory and importing it simply works.
So, my question: if Perl's modules are written in Perl, why the copy/paste approach will not work? If some (or some part) of the modules have to be compiled, how to see in CPAN if a module is Perl-only or it relies upon compiled libraries? Isn't there a way to download the module (tar, zip...) and use cpan to deploy it? This would solve my problem under Windows.
Now, Perl is a dynamic language, but that doesn't imply that everything that people write is portable across platforms. That's not the fault of the language. It's not even the fault of the programmer. Some things, like Win32::OLE shouldn't work on Unix. :)
Other dynamic languages will have some of the same problems. If you have to compile C code, you won't be able to merely copy files to another machine. Some distributions configure the code slightly differently depending on your operating system, etc.
Even if you could copy files, you have to ensure that you copy all of the files that you need. Do you know everything that you need for a particular module? Remember, many of them have dependencies.
Most of the problems you're having aren't anything to do with the language. You're having trouble with the tools. If you want a zero conf CPAN tool that makes all the decisions for you, try cpanminus. It's mostly the same thing that you'd get out of cpan (although different code), but it makes all of the decisions for you. It doesn't run any of the distribution tests, and it installs into your user directory. When you need something that gives you control, come back to cpan.
In the external environment (Linux CentOs) it worked after hours because I don't have root access and I had to configure cpan to operate as a non-root user
This is one of those times when it helps to know The Trick. In this case local::lib, which lets you configure a non-root install area and all the ENV variables in about three minutes.
if perl's modules are written in perl, why the copy/past approach will not work?
Some are written in Pure Perl, but many are written partially in C (using Perl's XS API) for efficiency.
Sometimes you end up with situations like JSON::XS, JSON::PP and JSON::Any to autoselect the best one that is installed.
Isn't there a way to download the module (tar, zip...) and use cpan to deploy it?
The cpan program is all about getting things from the Internet. You can download the package (there will be a link along the lines of "Download: CGI.pm-3.49.tar.gz" on the right hand side of the CPAN page), untar it, then
perl Makefile.PL
make
make install
You would probably be better off configuring your cpan installation to use only HTTP sources (in the urllist config option). Possibly going to far as to create a mini CPAN mirror inside your network.
I have several scripts that I supply to users as tools for their engineering projects. These scripts contain a lot of duplicated code. I want to extract this code and maintain it in a set of modules. However, in order for this to work, the users would have to install these modules. I don't want to have to tell my users to "make install", etc., as I'm sure none of them would have the patience for that.
I understand that one option is to package everything together with PAR, but ideally the user would be able to open up && edit these scripts if they need to, just like they can now. They also need to be able to move them to whatever directory they want, and I don't want them to have to move a bunch of library files as well.
Is it possible to make a double-click file that installs some bundled Perl modules?
I distribute my script as modules, and then use the normal CPAN toolchain to install them. See my articles Scripts as Modules and Automating Script Distributions with scriptdist. Once you have them in a conventional module distribution, you can install them from their current directory with cpan:
% cpan . # install from distribution in the current directory
Depending on how complex your situation is, you might want to create a DPAN, which is a private version of CPAN that your CPAN tools can draw from. You put all of your private modules there, point your CPAN tools at it, and everything happens as it does with a real CPAN mirror. You never have to share your work though.
yeah package with either PAR or Shipwright (not sure about binaries). Also use scandeps.pl along the way.
If the users are using systems with a packaging system (dpkg, cygwin, etc.), consider using that.
If you don't mind spending some green, one of the better bet is Perl Dev Kit from Activestate.
From their own description of the product,
Develop and deploy your Perl programs
to anyone on any platform with
PerlApp's new cross-platform
wrapping.
Deliver code as
executables or as Windows Services,
ActiveX components, .NET assemblies
or in the System Tray.
Easily create
MSI files using Perl code.
You may also consider using sparrow - scripts distribution system. Sparrow plays nice with Perl as it writen on it. It supports CPAN modules dependencies via carton tool.
PS Disclaimer - I am the tool author
It seems that most of the installers for Perl are centered around installing Perl modules, not applications. Things like ExtUtils::MakeMaker and Module::Build are very well suited for modules, but require some additional work for Web Apps.
Ideally it would be nice to be able to do the following after checking out the source from the repository:
Have missing dependencies detected
Download and install dependencies from CPAN
Run a command to "Build" the source into a final state (perform any source parsing or configuration necessary for the local environment).
Run a command to install the built files into the appropriate locations. Not only the perl modules, but also things like template (.tt) files, and CGI scripts, JS and image files that should be web-accessible.
Make sure proper permissions are set on installed files (and SELinux context if necessary).
Right now we have a system based on Module::Build that does most of this. The work was done by done by my co-worker who was learning to use Module::Build at the time, and we'd like some advice on generalizing our solution, since it's fairly app-specific right now. In particular, our system requires us to install dependencies by hand (although it does detect them).
Is there any particular system you've used that's been particularly successful? Do you have to write an installer based on Module::Build or ExtUtils::MakeMaker that's particular to your application, or is something more general available?
EDIT: To answer brian's questions below:
We can log into the machines
We do not have root access to the machines
The machines are all (ostensibly) identical builds of RHEL5 with SELinux enabled
Currently, the people installing the machines are only programmers from our group, and our source is not available to the general public. However, it's conceivable our source could eventually be installed on someone else's machines in our organization, to be installed by their programmers or systems people.
We install by checking out from the repository, though we'd like to have the option of using a distributed archive (see above).
The answer suggesting RPM is definitely a good one. Using your system's package manager can definitely make your life easier. However, it might mean you also need to package up a bunch of other Perl modules.
You might also take a look at Shipwright. This is a Perl-based tool for packaging up an app and all its Perl module dependencies. It's early days yet, but it looks promising.
As far as installing dependencies, it wouldn't be hard to simply package up a bunch of tarballs and then have you Module::Build-based solution install them. You should take a look at pip, which makes installing a module from a tarball quite trivial. You could package this with your code base and simply call it from your own installer to handle the deps.
I question whether relying on CPAN is a good idea. The CPAN shell always fetches the latest version of a distro, rather than a specific version. If you're interested in ensuring repeatable installs, it's not the right tool.
What are your limitations for installing web apps? Can you log into the machine? Are all of the machines running the same thing? Are the people installing the web apps co-workers or random people from the general public? Are the people installing this sysadmins, programmers, web managers, or something else? Do you install by distributed an archive or checking out from source control?
For most of my stuff, which involves sysadmins familiar with Perl installing in control environments, I just use MakeMaker. It's easy to get it to do all the things you listed if you know a little about MakeMaker. If you want to know more about that, ask a another question. ;) Module::Build is just as easy, though, and the way to go if you don't already like using MakeMaker.
Module::Build would be a good way to go to handle lots of different situations if the people are moderately clueful about the command line and installing software. You'll have a lot of flexibility with Module::Build, but also a bit more work. And, the cpan tool (which comes with Perl), can install from the current directory and handle dependencies for you. Just tell it to install the current directory:
$ cpan .
If you only have to install on a single platorm, you'll probably have an easier time making a package in the native format. You could even have Module::Build make that package for you so the developers have the flexibility of Module::Build, but the installers have the ease of the native process. Sticking with Module::Build also means that you could create different packages for different platforms from a single build tool.
If the people installing the web application really have no idea about command lines, CPAN, and other things, you'll probably want to use a packager and installer that doesn't scare them or make them think about what is going on, and can accurately report problems to you automatically.
As Dave points out, using a real CPAN mirror always gets you the latest version of a module, but you can also make your own "fake" CPAN mirror with exactly the distributions you want and have the normal CPAN tools install from that. For our customers, we make "CPAN on a CD" (although thumb drives are good now too). With a simple "run me" script everything gets installed in exactly the versions they need. See, for instance, my Making my own CPAN talk if you're interested in that. Again, consider the audience when you think about that. It's not something you'd hand to the general public.
Good luck, :)
I'd recommend seriously considering a package system such as RPM to do this. Even if you're running on Windows I'd consider RPM and cygwin to do the installation. You could even set up a yum or apt repository to deliver the packages to remote systems.
If you're looking for a general installer for customers running any number of OSes and distros, then the problem becomes much harder.
Take a look at PAR.
Jonathan Rockway as a small section on using this with Catalyst in his book.