tl;dr I want to ship a package.json with my Perl library, run yarn install (or npm install during the installation) and install the downloaded JavaScript dependencies with the Perl modules.
I have the following dist.ini:
name = Foobar
version = 1.2.3
license = Perl_5
copyright_holder = Yours Truly
copyright_year = 2018
[#Filter]
-bundle = #Basic
-remove = GatherDir
[Git::GatherDir]
[Web::FileHeader]
header_filename = EMM-include.pm
file_match = ^Makefile\.PL$
The file EMM-include.pm contains a MY::postamble method:
package MY;
use strict;
use Cwd qw(abs_path);
use File::Spec;
sub postamble {
my ($self) = #_;
my $here = Cwd::abs_path();
my $libdir = File::Spec->catdir($here, 'lib', 'Foobar');
chdir $libdir or die;
0 == system 'yarn', 'install' or die;
chdir $here or die;
return '';
}
The plug-in [Web::FileHeader] takes the file and patches it to the beginning of Makefile.PL.
Then there is a lib/Foobar/package.json:
{
"name": "foobar",
"version": "1.2.3",
"main": "index.js",
"dependencies": {
"ajv": "^6.5.4"
}
}
The MY::postamble section from EMM-include.pm invokes yarn install (replace it with npm install if you don't have yarn) and populate the directory lib/Foobar/node_modules with ajv and its dependencies.
Finally, there must be a module lib/Foobar.pm:
package Foobar;
# ABSTRACT: Just a test.
1;
That almost works as intended: The distribution can be created with dzil build. In the distribution directory, perl Makefile.PL invokes yarn install, the directory lib/Foobar/node_modules gets populated but the files in there are not installed with make install.
If I run perl Makefile.PL a second time, everything works, the JavaScript dependencies make it into blib/ and make install would install the JavaScript modules alongside the Perl modules.
Shipping the JavaScript dependencies with the distribution is not an option. They are already too many and they may have conflicting licenses (I am using GPLv3 here). Downloading the deps at runtime, after the installation will mostly fail because of missing privileges.
True, this has not that much to do with Dist::Zilla, it's rather a problem with ExtUtils::MakeMaker. But I am actually using Dist::Zilla here.
In case it matters, the real distribution is https://github.com/gflohr/qgoda and the last commit at the time of this writing is https://github.com/gflohr/qgoda/commit/3f34a3dfec8da665061432c3a8f7bd8eef28b95e.
First, instead of using [Web::FileHeader] to alter your Makefile.PL, replace [MakeMaker] (used by #Basic) with [MakeMaker::Awesome], which allows you to modify Makefile.PL directly, and correctly enables dynamic_config since your distribution needs it. Also, don't give your include-file a .pm extension since it's not a perl module, and exclude it from being gathered into the resulting distribution so it doesn't accidentally get installed.
[#Filter]
-bundle = #Basic
-remove = GatherDir
-remove = MakeMaker
[Git::GatherDir]
exclude_filename = EMM-include
[MakeMaker::Awesome]
header_file = EMM-include
I strongly suggest using my #Starter bundle instead of the outdated #Basic, but if not at least add [MetaJSON] so you have modern metadata.
[#Starter::Git]
revision = 3
installer = MakeMaker::Awesome
Git::GatherDir.exclude_filename[] = EMM-include
MakeMaker::Awesome.header_file = EMM-include
Regarding what needs to be done at install time. First I will caution that requiring an internet connection to install is not something you can always rely on, nor is having yarn available of course. But the Alien series of modules for installing external libraries does this sort of thing often. Since you don't need to compile this code you probably don't need the whole Alien::Build/Alien::Base setup, but it may turn out to be an easier way to solve your problem than Makefile hacking described below. Basically you would first release an Alien distribution which installs your javascript library if necessary, and then this distribution could depend on that to load the library. If you decide to pursue this direction, check out Alien::Build, and the IRC channel #native on irc.perl.org.
The postamble section for ExtUtils::MakeMaker is not for running arbitrary code; it's for adding custom rules to the Makefile it generates; this is the way you need to influence the make process. I know very little about Makefiles so I can't help you further here, all I can suggest is to read all of the EUMM docs and note that the postamble is a function from MM_Any which you override to add your text, among other options from MM_Any and MM_Unix. You may be able to find people to help you in this direction on the IRC channel #toolchain on irc.perl.org.
Related
I have a perl program that uses WWW::Mechanize::Firefox on windows 7 32bit with strawberry perl.
It works fine with the command C:\>perl testcase.pl. When I compile it with C:\>pp -o testcase.exe testcase.pl it compiles with no errors.
When I run the testcase.exe it gives me the error:
Failed to connect to , Can't locate object method "setup" via package "MozRepl::Client" at MozRepl.pm line 224
The code I am using for testcase.pl is:
#!perl
use MozRepl;
use WWW::Mechanize::Firefox;
use warnings;
system('start firefox');
sleep(5);
$mech = WWW::Mechanize::Firefox->new;
Also note that a program without WWW::Mechanize::Firefox and MozRepl does work fine.
The problem has obviously been narrowed down to PAR::Packer not liking MozRepl, any idea what it might be?
PAR::Packer sometimes has a hard time identifying which modules need to be included in a PAR package in order to fulfill all of the requirements of the program you are trying to package.
It copes OK if the dependancies are loaded via plain 'use', or 'require' statements where the module to be loaded is a literal string, but it won't have much chance if the module is being loaded dynamically with something like:
require $myModuleToLoad;
Browsing the source code of MozRepl and related modules shows that they make heavy use of plugins loaded dynamically. I suspect that some of these are not being packaged.
You can manually specify module(s) to be included in the PAR package by adding -M Module::Name to the pp command line for each of the modules to be added (replacing Module::Name with the actual module name of course).
The hard part might be identifying which modules to include. One way to do this is to temporarily add something like this to the end of your script:
END { print "$_ -> $INC{$_}\n" foreach sort keys %INC; }
then run your script normally, not through PAR. It should list all the modules that were loaded. You can compare that with the actual modules present in the PAR package and add the missing ones using the -M option to pp.
You can see the modules inside your PAR file by opening it with an unzipping tool, such as 7zip. Or in Linux:
unzip -l {parfile}
I am familiar with using package.json with node.js, Gemfile for Ruby, Podfile for Objective-C, et al.
What is the equivalent file for Perl and what is the syntax used?
I've installed a couple packages using cpanm and would like to save the package names and version in a single file that can be executed by team members.
For simple use cases, writing a cpanfile is a good choice. A sample file might look like
requires 'Marpa::R2', '2.078';
requires 'String::Escape', '2010.002';
requires 'Moo', '1.003001';
requires 'Eval::Closure', '0.11';
on test => sub {
requires 'Test::More', '0.98';
};
That is, it's actually a Perl script, not a data format. The dependencies can then be installed like
$ cd /path/to/your/module
$ cpanm --installdeps .
This does not install your module! But it makes sure that all dependencies are satisfied, so we can do:
use lib '/path/to/your-module/lib'; # add the location as a module search root
use Your::Module; # works! yay
This is usually sufficient e.g. for a git repository which you want others to tinker with.
If you want to create a tarball that can be distributed and installed easily, I'd recommend Dist::Zilla (although it's geared towards CPAN releases). Instead of a cpanfile we use a dist.ini:
name = Your-Module
version = 1.2.3
author = Your Self <you#example.com>
license = GPL_3
copyright_holder = Your Self
[#Basic]
[Prereqs]
Marpa::R2 = 2.078
String::Escape = 2010.002
Moo = 1.003001
Eval::Closure = 0.11
[Prereqs / TestRequires]
Test::More = 0.98
Then:
$ dzil test # sanity checks, and runs your tests
$ dzil build # creates a tarball
Dist::Zilla takes care of creating a Makefile.PL and other infrastructure that is needed to install the module.
You can then distribute that tarball, and install it like cpanm Your-Module-1.2.3.tar.gz. Dependencies are resolved, your packages are copied to a permanent location, and you can now use Your::Module in any script without having to specify the location.
Note that you should adhere to the standard directory layout for Perl modules:
./
lib/
Your/
Module.pm # package Your::Module
Module/
Helper.pm # package Your::Module::Helper
t/ # tests to verify the module works on the target syste,
foo.t
bar.t
xt/ # optional: Author tests that are not run on installation
baz.t
bin/ # optional: scripts that will later end up in the target system's $PATH
command-line-tool
Makefile.PL usually (along with a few other files; Perl has had packages for longer then any of the other languages you mention and suffers from a bit of inelegance here).
Module Starter is a sensible way to start writing a package. It has a getting started guide.
Instead of adding or modifying files in the directory where the sources of a Perl module are unpacked, I would like to build everything in a separate directory. Is this easily achievable with a fairly standard Makefile.PL that uses ExtUtils::MakeMaker? (By easy, I mean something like one or a few command line parameters.) If no, does any of the other build systems support this?
Update / Reason: The Perl module is a binding to a library whose build system is autoconf/automake/libtool-based. The Perl module is shipped together with this library and calling make in the top directory eventually also builds the Perl library. I am interested in building the entire project in a separate build tree. At the moment I do something similar to what runrig suggested, only using symlinks. (cp -sru $(srcdir)/. $(builddir)/.). This has worked so far, but if there is a more elegant solution, I'd like to read about it.
MakeMaker already copies the sources and builds them in a separate directory (that's what blib/ is). You can control the build location with the INST_* set of arguments to WriteMakefile(). This example changes the location from blib/ to foo/.
INST_ARCHLIB => "foo/arch",
INST_LIB => "foo/lib",
INST_BIN => "foo/bin",
INST_SCRIPT => "foo/script",
INST_MAN1DIR => 'foo/man1',
INST_MAN3DIR => 'foo/man3',
In addition you have to tell MakeMaker to cleanup the new build directory.
clean => {
FILES => 'foo'
},
See "Determination of Perl Library and Installation Locations" in the ExtUtils::MakeMaker docs for more info.
cp -R Module-Directory-0.01 Module-Directory-0.01.copy
cd Module-Directory-0.01.copy
perl Makefile.PL
make
make test
...etc.
I ended up using symlinks:
The library to which the Perl module provides bindings uses an
autoconf/automake/libtool-based build system. Makefile.PL is
generated from Makefile.PL.in by configure. Makefile.PL
generates Makefile-pl (Makefile has already been taken by
autoconf/automake).
This is the relevant part of Makefile.am:
all: Makefile-pl src_deps
$(MAKE) -f Makefile-pl
Makefile-pl: Makefile.PL
perl Makefile.PL INSTALLDIRS=$(INSTALLDIRS) PREFIX=$(prefix)
I changed the second target to:
Makefile-pl: Makefile.PL
-[ $(srcdir) != $(builddir) ] && cp -rsu $(abs_srcdir)/. $(builddir)/.
perl Makefile.PL INSTALLDIRS=$(INSTALLDIRS) PREFIX=$(prefix)
This should work as long as building or installing the Perl module
does not result in any files being modified in-place.
I'm working on installing a perl module (not using CPAN) on a Linux machine. When I run the command:
perl Build.PL
I get the following error:
ERROR: Missing required field 'dist_abstract' for metafile
Could not get valid metadata. Error is: Invalid metadata structure.
Errors: Missing mandatory field, 'abstract' (abstract) [Validation: 1.4],
value is an undefined string (abstract) [Validation: 1.4]
at /usr/local/share/perl5/Module/Build/Base.pm line 4559
Could not create MYMETA files
I've tried Googling bits and pieces of this error but haven't found any solutions. Just looking for a clue as to what might be causing this error.
Here's a link to a zip file containing the files required to install it:
https://oncourse.iu.edu/access/content/user/brilewis/Filemanager_Public_Files/DataDownloader.zip
First at all please make sure you have package Module::Build installed.
You need ungzip few gzipped files in this package. I don't realize why author gzipped them:
gzip -d *.gz
I really don't know why author archived each install file. It looks like some mistake to me.
Than you can install all dependencies (this module requires some):
./Build installdeps
And then finally install module itself:
./Build
./Build test
./Build install
However I must warn you that this module packaged in a bit strange way and there's no guarantee it works.
The NAME section of the module does not have a - in it, e.g.,
=head1 NAME
Foo::Bar implements a Foo framework.
will fail, but if you make it
=head1 NAME
Foo::Bar - implements a Foo framework.
then it will work.
Do you have root access on your machine? Can you use the cpan utility to build and install your module. Using cpan is fairly straight forward:
$ cpan
After that, it will do a lot of configuration, simply take the default values. When it finishes, it'll come to a cpan> prompt. All you have to do there is type this:
cpan> install Module::Name
Where Module::Name is the module you're trying to install. Check the CPAN archive to get the name of your module.
If there are any dependencies, CPAN will ask if you want to download and install those. Say Yes, and CPAN will install the dependencies, then your module.
Using cpan is the best way to install third party modules you find in the CPAN archive. It takes care of all the dependencies, testing, and building for you.
Try installing through CPAN, and then see if you still have your issues.
Having volunteered to maintain a stagnant CPAN package (GnuPG) I'd like improve the install files such that they exit gracefully if the gpg binary (which GnuPG is a wrapper for) cannot be found. After a bit of seeking inspiration from other packages, I've come up with adding this to Makefile.PL:
my #paths = grep { -x "$_/gpg" } split /:/, $ENV{PATH}, $ENV{PGP_PATH};
unless ( scalar #paths ) {
print <<EOD;
I can't find the gpg binary on your system. If it's not installed in your usual PATH, set $ENV{PGP_PATH} to include where it can be found and try installing again.
EOD
exit(0);
}
WriteMakefile(
'NAME' => 'GnuPG',
'VERSION_FROM' => 'GnuPG.pm',
'EXE_FILES' => [ gpgmailtunl ],
'LICENSE' => 'GPL',
'LIBS' => [ #paths ],
);
Does that look sane?
If you're using Module::Install or part of that family, you can use do
requires_external_bin 'gpg';
See Module::Install::External for details.
No good reason to reinvent the wheel.
The general concept is fine - if what you need to work is not present, don't create the makefile. The CPAN Testers have rules about exiting with a zero status on failure (which annoys me a lot, but never mind; I hate failing with a success status!).
Question: do you keep a record of where PGP was found at install time, so that if somebody else uses the Perl module without the location on their path, the module can still run?
For DBD::Informix, I have rigid dependencies without which the module cannot be compiled; the Makefile.PL is a major production in its own right because of that. It also tries to handle versions of the software covering over 15 years; that complicates its life, too. If the pre-requisites (some Perl modules; some non-Perl software) is not available, it won't install.
Wouldn't it make more sense to just print a warning? Is gpg necessary for the installation itself?
The code itself looks fine by me. But perhaps there's a built-in 'which' functionality. :).
For better accuracy you should look at File::Which or at least use File::Spec->path().
File::Which would be a crossplatform solution. You will need to either bundle it into inc/ directory or require it to be installed with configure_requires. For EU::MM it can be done with
META_MERGE => {
configure_requires => {
'File::Which' => 0,
Module::Install is also a good solution, but you will need to release new version of distribution each time new version of Module::Install is released and changes are important.