Defining required packages and versions for a perl project - perl

I am familiar with using package.json with node.js, Gemfile for Ruby, Podfile for Objective-C, et al.
What is the equivalent file for Perl and what is the syntax used?
I've installed a couple packages using cpanm and would like to save the package names and version in a single file that can be executed by team members.

For simple use cases, writing a cpanfile is a good choice. A sample file might look like
requires 'Marpa::R2', '2.078';
requires 'String::Escape', '2010.002';
requires 'Moo', '1.003001';
requires 'Eval::Closure', '0.11';
on test => sub {
requires 'Test::More', '0.98';
};
That is, it's actually a Perl script, not a data format. The dependencies can then be installed like
$ cd /path/to/your/module
$ cpanm --installdeps .
This does not install your module! But it makes sure that all dependencies are satisfied, so we can do:
use lib '/path/to/your-module/lib'; # add the location as a module search root
use Your::Module; # works! yay
This is usually sufficient e.g. for a git repository which you want others to tinker with.
If you want to create a tarball that can be distributed and installed easily, I'd recommend Dist::Zilla (although it's geared towards CPAN releases). Instead of a cpanfile we use a dist.ini:
name = Your-Module
version = 1.2.3
author = Your Self <you#example.com>
license = GPL_3
copyright_holder = Your Self
[#Basic]
[Prereqs]
Marpa::R2 = 2.078
String::Escape = 2010.002
Moo = 1.003001
Eval::Closure = 0.11
[Prereqs / TestRequires]
Test::More = 0.98
Then:
$ dzil test # sanity checks, and runs your tests
$ dzil build # creates a tarball
Dist::Zilla takes care of creating a Makefile.PL and other infrastructure that is needed to install the module.
You can then distribute that tarball, and install it like cpanm Your-Module-1.2.3.tar.gz. Dependencies are resolved, your packages are copied to a permanent location, and you can now use Your::Module in any script without having to specify the location.
Note that you should adhere to the standard directory layout for Perl modules:
./
lib/
Your/
Module.pm # package Your::Module
Module/
Helper.pm # package Your::Module::Helper
t/ # tests to verify the module works on the target syste,
foo.t
bar.t
xt/ # optional: Author tests that are not run on installation
baz.t
bin/ # optional: scripts that will later end up in the target system's $PATH
command-line-tool

Makefile.PL usually (along with a few other files; Perl has had packages for longer then any of the other languages you mention and suffers from a bit of inelegance here).
Module Starter is a sensible way to start writing a package. It has a getting started guide.

Related

How to write a minimally working pyproject.toml file that can install packages?

Pip supports the pyproject.toml file but so far all practical usage of the new schema requires a 3rd party tool that auto-generates these files (e.g., poetry and pip). Unlike setup.py which is already human-writeable, pyproject.toml is not (yet).
From setuptools docs,
[build-system]
requires = [
"setuptools >= 40.9.0",
"wheel",
]
build-backend = "setuptools.build_meta"
However, this file does not include package dependencies (as outlined in PEP 621). Pip does support installing packages using pyproject.toml but nowhere does pep specify how to write package dependencies in pyproject.toml for the official build system setuptools.
How do I write package dependencies in pyproject.toml?
Related StackOverflow Questions:
How to init the pyproject.toml file
This question asks for a method to auto-generate pyproject.toml, my question differ because I ask for a human-written pyproject.toml.
my question differ because I ask for a human-written pyproject.toml
First, the pyproject.toml file is always "human-writable".
Then, it is important to know that in this context setuptools and Poetry take the role of what are called "PEP 517 build back-ends", and there are many such back-ends available today, setuptools and Poetry are just two examples of them.
As of today, it seems like most (if not all) of the build back-ends I know of expect their configuration (including dependencies) to be written in pyproject.toml.
PEP 621
There is a standard called PEP 621 that specifies how a project's metadata, including dependencies, should be laid out in the pyproject.toml file.
Here is a list of build back-ends I know of that have support for PEP 621:
enscons
flit_core (see flit)
hatchling (see hatch)
pdm-pep517 (see pdm)
setuptools (experimental support since version 61.0.0)
trampolim
whey
For all PEP 621 compatible build back-ends, the dependencies should be written in pyproject.toml file like in the following:
[project]
name = "Thing"
version = "1.2.3"
# ...
dependencies = [
"SomeLibrary ~= 2.2",
]
References:
https://peps.python.org/pep-0621/
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html (experimental support for PEP 621 has been added to setuptools version 61.0.0, released 2022-03-24)
setuptools (before version 61.0.0)
In setuptools before version 61.0.0 there is no support for writing the configuration in pyproject.toml (in other words: no PEP 621 support). You have to either write a setup.cfg, or a setup.py, or a combination of both.
My recommendation is to write as much as possible in setup.cfg. Such a setup.cfg could look like this:
[metadata]
name = Thing
version = 1.2.3
[options]
install_requires =
SomeLibrary ~= 2.2
packages = find:
and in most cases the setup.py can be omitted completely or it can be as short as:
import setuptools
setuptools.setup()
References about the dependencies specifically:
https://setuptools.readthedocs.io/en/latest/userguide/dependency_management.html
https://www.python.org/dev/peps/pep-0508/
https://www.python.org/dev/peps/pep-0440/
Again, note that in most cases it is possible to omit the setup.py file entirely, one of the conditions is that the setup.cfg file and a pyproject.toml file are present and contain all the necessary information. Here is an example of pyproject.toml that works well for a setuptools build backend:
[build-system]
build-backend = 'setuptools.build_meta'
requires = [
'setuptools >= 43.0.0',
]
poetry
In poetry everything is defined in pyproject.toml, but it uses poetry-specific sections. In other words, Poetry does not currently use the PEP 621 standard, but there are some plans to move to this standard in the future.
This file can be hand-written. As far as I can tell, there is no strict need to ever explicitly install poetry itself (commands such as pip install and pip wheel can get you far enough).
The pyproject.toml file can be as simple as:
[tool.poetry]
name = 'Thing'
version = '1.2.3'
[tool.poetry.dependencies]
python = '^3.6'
SomeLibrary = '~2.2'
[build-system]
requires = ['poetry-core~=1.0']
build-backend = 'poetry.core.masonry.api'
References:
https://python-poetry.org/docs/pyproject/
https://python-poetry.org/docs/dependency-specification/

Installing additional files at install time with ExtUtils::MakeMaker/Dist::Zilla (dzil)

tl;dr I want to ship a package.json with my Perl library, run yarn install (or npm install during the installation) and install the downloaded JavaScript dependencies with the Perl modules.
I have the following dist.ini:
name = Foobar
version = 1.2.3
license = Perl_5
copyright_holder = Yours Truly
copyright_year = 2018
[#Filter]
-bundle = #Basic
-remove = GatherDir
[Git::GatherDir]
[Web::FileHeader]
header_filename = EMM-include.pm
file_match = ^Makefile\.PL$
The file EMM-include.pm contains a MY::postamble method:
package MY;
use strict;
use Cwd qw(abs_path);
use File::Spec;
sub postamble {
my ($self) = #_;
my $here = Cwd::abs_path();
my $libdir = File::Spec->catdir($here, 'lib', 'Foobar');
chdir $libdir or die;
0 == system 'yarn', 'install' or die;
chdir $here or die;
return '';
}
The plug-in [Web::FileHeader] takes the file and patches it to the beginning of Makefile.PL.
Then there is a lib/Foobar/package.json:
{
"name": "foobar",
"version": "1.2.3",
"main": "index.js",
"dependencies": {
"ajv": "^6.5.4"
}
}
The MY::postamble section from EMM-include.pm invokes yarn install (replace it with npm install if you don't have yarn) and populate the directory lib/Foobar/node_modules with ajv and its dependencies.
Finally, there must be a module lib/Foobar.pm:
package Foobar;
# ABSTRACT: Just a test.
1;
That almost works as intended: The distribution can be created with dzil build. In the distribution directory, perl Makefile.PL invokes yarn install, the directory lib/Foobar/node_modules gets populated but the files in there are not installed with make install.
If I run perl Makefile.PL a second time, everything works, the JavaScript dependencies make it into blib/ and make install would install the JavaScript modules alongside the Perl modules.
Shipping the JavaScript dependencies with the distribution is not an option. They are already too many and they may have conflicting licenses (I am using GPLv3 here). Downloading the deps at runtime, after the installation will mostly fail because of missing privileges.
True, this has not that much to do with Dist::Zilla, it's rather a problem with ExtUtils::MakeMaker. But I am actually using Dist::Zilla here.
In case it matters, the real distribution is https://github.com/gflohr/qgoda and the last commit at the time of this writing is https://github.com/gflohr/qgoda/commit/3f34a3dfec8da665061432c3a8f7bd8eef28b95e.
First, instead of using [Web::FileHeader] to alter your Makefile.PL, replace [MakeMaker] (used by #Basic) with [MakeMaker::Awesome], which allows you to modify Makefile.PL directly, and correctly enables dynamic_config since your distribution needs it. Also, don't give your include-file a .pm extension since it's not a perl module, and exclude it from being gathered into the resulting distribution so it doesn't accidentally get installed.
[#Filter]
-bundle = #Basic
-remove = GatherDir
-remove = MakeMaker
[Git::GatherDir]
exclude_filename = EMM-include
[MakeMaker::Awesome]
header_file = EMM-include
I strongly suggest using my #Starter bundle instead of the outdated #Basic, but if not at least add [MetaJSON] so you have modern metadata.
[#Starter::Git]
revision = 3
installer = MakeMaker::Awesome
Git::GatherDir.exclude_filename[] = EMM-include
MakeMaker::Awesome.header_file = EMM-include
Regarding what needs to be done at install time. First I will caution that requiring an internet connection to install is not something you can always rely on, nor is having yarn available of course. But the Alien series of modules for installing external libraries does this sort of thing often. Since you don't need to compile this code you probably don't need the whole Alien::Build/Alien::Base setup, but it may turn out to be an easier way to solve your problem than Makefile hacking described below. Basically you would first release an Alien distribution which installs your javascript library if necessary, and then this distribution could depend on that to load the library. If you decide to pursue this direction, check out Alien::Build, and the IRC channel #native on irc.perl.org.
The postamble section for ExtUtils::MakeMaker is not for running arbitrary code; it's for adding custom rules to the Makefile it generates; this is the way you need to influence the make process. I know very little about Makefiles so I can't help you further here, all I can suggest is to read all of the EUMM docs and note that the postamble is a function from MM_Any which you override to add your text, among other options from MM_Any and MM_Unix. You may be able to find people to help you in this direction on the IRC channel #toolchain on irc.perl.org.

How to install local tar file dependencies together with dependencies from CPAN?

Let's assume there is a distribution "Example" that is packed in example-1.0.tar.gz which contains a Makefile.PL that points to all dependencies. After extracting, "Example" could be easily installed (including all its dependencies) by calling cpanm . if all its dependencies were available on CPAN.
How do I install a simialar file (let's call it example-2.0.tar.gz) if I am given a folder "deps" containing most of the distribution's dependencies
dependency-1.tar.gz
dependency-2.tar.gz
...
dependency-N.tar.gz
where N is a huge number?
Further assumptions are:
dependency-*tar.gz files have to be preferred over the available CPAN modules
some dependencies are not available on CPAN but only in this "deps" folder
the folder "deps" is incomplete and further dependencies have to be installed from CPAN.
Is there a simple way to install example-2.0.tar.gz?
I would like to know if there is some command line like cpanm --use-local-deps=~/deps example-2.0.tar.gz which does not need further workarounds.
You could use this tool http://search.cpan.org/~bingos/CPANPLUS-0.9152/bin/cpan2dist . It lets you specify filters to define what modules are loaded locally and what is fetched from CPAN. It's part of the CPANPLUS module, which should cone with newer versions of Perl I think.

Can I build Perl modules with ExtUtils::MakeMaker-based build system "out of tree"?

Instead of adding or modifying files in the directory where the sources of a Perl module are unpacked, I would like to build everything in a separate directory. Is this easily achievable with a fairly standard Makefile.PL that uses ExtUtils::MakeMaker? (By easy, I mean something like one or a few command line parameters.) If no, does any of the other build systems support this?
Update / Reason: The Perl module is a binding to a library whose build system is autoconf/automake/libtool-based. The Perl module is shipped together with this library and calling make in the top directory eventually also builds the Perl library. I am interested in building the entire project in a separate build tree. At the moment I do something similar to what runrig suggested, only using symlinks. (cp -sru $(srcdir)/. $(builddir)/.). This has worked so far, but if there is a more elegant solution, I'd like to read about it.
MakeMaker already copies the sources and builds them in a separate directory (that's what blib/ is). You can control the build location with the INST_* set of arguments to WriteMakefile(). This example changes the location from blib/ to foo/.
INST_ARCHLIB => "foo/arch",
INST_LIB => "foo/lib",
INST_BIN => "foo/bin",
INST_SCRIPT => "foo/script",
INST_MAN1DIR => 'foo/man1',
INST_MAN3DIR => 'foo/man3',
In addition you have to tell MakeMaker to cleanup the new build directory.
clean => {
FILES => 'foo'
},
See "Determination of Perl Library and Installation Locations" in the ExtUtils::MakeMaker docs for more info.
cp -R Module-Directory-0.01 Module-Directory-0.01.copy
cd Module-Directory-0.01.copy
perl Makefile.PL
make
make test
...etc.
I ended up using symlinks:
The library to which the Perl module provides bindings uses an
autoconf/automake/libtool-based build system. Makefile.PL is
generated from Makefile.PL.in by configure. Makefile.PL
generates Makefile-pl (Makefile has already been taken by
autoconf/automake).
This is the relevant part of Makefile.am:
all: Makefile-pl src_deps
$(MAKE) -f Makefile-pl
Makefile-pl: Makefile.PL
perl Makefile.PL INSTALLDIRS=$(INSTALLDIRS) PREFIX=$(prefix)
I changed the second target to:
Makefile-pl: Makefile.PL
-[ $(srcdir) != $(builddir) ] && cp -rsu $(abs_srcdir)/. $(builddir)/.
perl Makefile.PL INSTALLDIRS=$(INSTALLDIRS) PREFIX=$(prefix)
This should work as long as building or installing the Perl module
does not result in any files being modified in-place.

Building perl module gives metafile error

I'm working on installing a perl module (not using CPAN) on a Linux machine. When I run the command:
perl Build.PL
I get the following error:
ERROR: Missing required field 'dist_abstract' for metafile
Could not get valid metadata. Error is: Invalid metadata structure.
Errors: Missing mandatory field, 'abstract' (abstract) [Validation: 1.4],
value is an undefined string (abstract) [Validation: 1.4]
at /usr/local/share/perl5/Module/Build/Base.pm line 4559
Could not create MYMETA files
I've tried Googling bits and pieces of this error but haven't found any solutions. Just looking for a clue as to what might be causing this error.
Here's a link to a zip file containing the files required to install it:
https://oncourse.iu.edu/access/content/user/brilewis/Filemanager_Public_Files/DataDownloader.zip
First at all please make sure you have package Module::Build installed.
You need ungzip few gzipped files in this package. I don't realize why author gzipped them:
gzip -d *.gz
I really don't know why author archived each install file. It looks like some mistake to me.
Than you can install all dependencies (this module requires some):
./Build installdeps
And then finally install module itself:
./Build
./Build test
./Build install
However I must warn you that this module packaged in a bit strange way and there's no guarantee it works.
The NAME section of the module does not have a - in it, e.g.,
=head1 NAME
Foo::Bar implements a Foo framework.
will fail, but if you make it
=head1 NAME
Foo::Bar - implements a Foo framework.
then it will work.
Do you have root access on your machine? Can you use the cpan utility to build and install your module. Using cpan is fairly straight forward:
$ cpan
After that, it will do a lot of configuration, simply take the default values. When it finishes, it'll come to a cpan> prompt. All you have to do there is type this:
cpan> install Module::Name
Where Module::Name is the module you're trying to install. Check the CPAN archive to get the name of your module.
If there are any dependencies, CPAN will ask if you want to download and install those. Say Yes, and CPAN will install the dependencies, then your module.
Using cpan is the best way to install third party modules you find in the CPAN archive. It takes care of all the dependencies, testing, and building for you.
Try installing through CPAN, and then see if you still have your issues.