The default behavior of rpmbuild seems to be to scan every file in the buildroot for #! lines and add those interpreters as hard requirements. Is it possible to disable this? I've run into a situation where people are committing scripts with #!/path/to/local/install/of/something and that requirement is becoming embedded in the rpm.
I haven't been able to find a command line or spec file option that suppresses this behavior. Hopefully I just didn't look hard enough.
I think the question has an answer in Where does rpm look for dependencies. At least AutoReqProv: no disables scanning and then you would need manual Requires: xxx values; not the least work solution, but a solution. I would also guess that something like .rpmmacros has a way to disable the script scanning. Take a look at /usr/lib/rpm/find-requires.
__provides_exclude_from, __provides_exclude, __requires_provides_from and __requires_provides also works in Fedora / Centos / RHEL
%global __provides_exclude_from %{buildroot}%{_javadir}/%{name}/lib/
where %{buildroot}%{_javadir}/%{name}/lib/ is one directory
a more complex example :
# Filter away the deps om bundled libs and those substituted
# by symlinks and explicit Requires:.
%global __requires_exclude ^libssl.so.1.0.0
%global __requires_exclude %__requires_exclude|^libcrypto.so.1.0.0
%global __requires_exclude %__requires_exclude|^libcef.so
%global __requires_exclude %__requires_exclude|^libcurl.so.4
%global __requires_exclude %__requires_exclude|^libcurl-gnutls.so.4
%global __requires_exclude %__requires_exclude|^libwidevinecdm.so.*
#global __requires_exclude #__requires_exclude|[.]so[.][0-2][a-f]
Related
I'm using ExtUtils::MakeMaker to distribute my perl module, which is composed of a PM file and an executable.
The executable have to load a configuration file.
I want my Makefile.PL to :
generate the configuration file ;
install it in the correct PREFIX ;
modify the executable to set the real path of the configuration file.
But I have no idea of how to do this.
This is where Makemaker is a real pain. You have to know how to do this in make.
First, Makefile.PL is just a Perl program, so you can do anything you want with that.
Second, you can use a .PL file as a program that runs to generate the real file. For instance, you could have a lib/Module.pm.PL. At build time, the system runs that program and uses the output to create blib/lib/Module.pm. See the documentation for PL_FILES.
Third, you can add tasks to the make targets so your processing happens at the right time. Double colon targets add to the tasks already defined.
all ::
perl create_config.pl > config.pl
install ... config.pl
To add this to Makefile from Makefile.PL, you have to add special subroutines to the MY namespace. It's all documented in Makemaker, but the docs assume you are comfortable with make.
But, all of that is really a pain in the ass even if you know how to do it. You think you have it figured out, then someone has a different setup or a different sort of make. You spend all your time with fiddly bits to make it compatible everywhere. This is one of the major reasons the Perl gods invented Module::Build. It's much easier to add custom processing and modify build targets, and you get to do it all in Perl! The rule of thumb is that if you don't know how to do it with Makemaker, it's time to use Module::Build.
I would like to factor out some of the stuff in Crypt::SSLeay's Makefile.PL into a couple of separate classes. These classes would only be used by Makefile.PL. As such, I do want them
not to be indexed by the PAUSE indexer
not to be installed as part of the module.
Should I just put them in inc the way Module::Install does? What else should I pay attention to?
PAUSE looks for a no_index parameter in the META.yml file (specifications: v1.4, v2). The default META.yml that ExtUtils::MakeMaker makes contains
no_index:
directory:
- t
- inc
but you can add more data to it if you'd like
package:
- Some::Package::Used::For::Building::But::Not::To::Be::Installed
file:
- a-file/with/a/package/statement/that-should-be/ignored.pm
There are really two issues here. One is to prevent PAUSE from indexing the extra modules in your distribution so they don't show up in the 02packages, and how not to fool a user into installing them.
The PAUSE answer is a combination of the proper fix, the no_index stuff, and the old folklore of faking out mldistwatch. PAUSE wants to discover which packages are in your distro. It looks for the package on the same line as a namespace. If it's not on the same line, PAUSE passes over it. So, you'll see in some older "hidden" modules lines like:
package # separate lines get past PAUSE
Some::Helper::Module;
If you like looking behind the curtain, the relevant code is in PAUSE::mldistwatch::filter_pms() in (the PAUSE github repo).
The other problem is to not install extra things that are in the distro. Indexing has nothing to do with that. Build files move a lot of stuff into blib (the build library) to prepare them for installation. Anything ending up in there gets installed. The trick is to not let the builder put your helper modules in there. That's usually not a problem as long as you don't put them at the top level of your distro or in the lib directory.
Do I have to run make and make install each time I change a .pm file for Perl? I'm doing a ton of testing and this is becoming cumbersome.
You don't have to install the module to test it.
If I'm testing inside my distribution directory, I just use the test target:
% make test
Or, if I'm using Module::Build:
% ./Build test
Since make is a dependency management tool, it also takes care of any other steps it needs to perform so it can run the test target. You don't need to run each target separately. Module::Build does the same thing.
If I want to test a single file, I combine the make command with a call to perl that also uses the blib module to set the right #INC:
% make; perl -Mblib t/single_test.t
Some people like using prove for the same thing. No matter which method I use, I'm probably using the arrow keys to move back to a previous command line to re-run it. I do very little typing in any of this.
It depends on module setup, but under the standard MakeMaker I use, "make test" runs a "make" if any files have been modified, so when doing intra-module development "make test" is the only command you need until you've finished.
Evan Carroll got it basically right. To expand on his answer: use the testing tools that come with Perl to tighten the workflow.
Let's say you are in your project directory and you hack on the files in its lib/ subdirectory. Execute prove -l to run all tests. That's easier than messing with absolute paths in the PERL5LIB environment variable.
Presumably you're editing a lib module in a non-lib location, rather than clobbering a global library for each modification - do the sensible thing and change the library path perl uses with PERL5LIB, which will append internally to #INC (the use search path):
PERL5LIB=/home/user/code/perl/project/lib perl myapp.pl
If your program isn't pure-perl and requires a make system, there is no way to do this short of rebuilding, but pure-perl (PP) doesn't really require make under normal circumstances. If you do it this way, running perl under a normal environment will yield the predictable and tested results, running it with your PERL5LIB will allow you to test the program.
I'm currently using the PAR Packer (pp) to package a couple of pl scripts such that they can be copied to a machine and "just work" without my client having to muck with CPAN.
Unfortunately, the PAR Packer doesn't work for deep dependencies. For example, a script imports CHI::Driver::File but does not explicitly import Log::Any::Adapter::Null which CHI::Driver::File requires. PAR does not pick up this dependency and running the generated script will crash with the following error message:
Can't locate Log/Any/Adapter/Null.pm in #INC (#INC contains: CODE(0x874aab8) /tmp/par-apache/cache-7b4508ab92efb43271da1629e8eb654c1572cc55/inc/lib /tmp/par-apache/cache-7b4508ab92efb43271da1629e8eb654c1572cc55/inc CODE(0x87e8f54) CODE(0x87e9194)) at (eval 215) line 3.
Compilation failed in require at CHI/Driver/File.pm line 11.
BEGIN failed--compilation aborted at CHI/Driver/File.pm line 11.
My work-around is to explicitly import "Log::Any::Adapter::Null" but there must be a better way. Perhaps this is a bug with the PAR Packer? I have installed the latest version (0.994).
Well, have you read the documentation? It gives a number of command-line options to include dependencies, and even the option to manually force modules to be included.
If you want a stand-alone script as opposed to a stand-alone binary (which is the default mode for pp), then add the -P option. I'd advise against it, though. It's the least-tested mode of operation.
Note also that PAR::Packer DOES work for deep dependencies. The subject of your question is a bit pretentious. In fact, PAR::Packer doesn't really check any dependencies itself but delegates this to Module::ScanDeps. Now, Module::ScanDeps is a heuristic approach to dependency scanning and it can be broken by ugly fuzzing with dynamic loading of dependencies. (i.e. generating module names at run time and then using them in an eval).
Indeed, if you look at the sources for Log::Any (which is used by CHI::Driver::File), you'll quickly see that it uses dynamic loading of modules. This is why the ::Adapter::Null module isn't being picked up.
Generally, we fix these issues by adding a special case to Module::ScanDeps for such modules whose author thought it'd be a good idea to defeat any sort of static analysis. Until you get a fixed version of Module::ScanDeps, you can use the -c or -x options to pp to have the dependency resolution use compilation or execution of the program instead of only relying on static analysis. The augmented Module::ScanDeps has version 0.95 and should be available from CPAN within the day.
I want another developer to run a Perl script I have written. The script uses many CPAN modules that have to be installed before the script can be run. Is it possible to make the script (or the perl binary) to dump a list of all the missing modules? Perl prints out the missing modules’ names when I attempt to run the script, but this is verbose and does not list all the missing modules at once. I’d like to do something like:
$ cpan -i `said-script --list-deps`
Or even:
$ list-deps said-script > required-modules # on my machine
$ cpan -i `cat required-modules` # on his machine
Is there a simple way to do it? This is not a show stopper, but I would like to make the other developer’s life easier. (The required modules are sprinkled across several files, so that it’s not easy for me to make the list by hand without missing anything. I know about PAR, but it seems a bit too complicated for what I want.)
Update: Thanks, Manni, that will do. I did not know about %INC, I only knew about #INC. I settled with something like this:
print join("\n", map { s|/|::|g; s|\.pm$||; $_ } keys %INC);
Which prints out:
Moose::Meta::TypeConstraint::Registry
Moose::Meta::Role::Application::ToClass
Class::C3
List::Util
Imager::Color
…
Looks like this will work.
Check out Module::ScanDeps and the "scandeps.pl" utility that comes with it. It can do a static (and recursive) analysis of your code for dependencies as well as the %INC dump either after compiling or running the program.
Please note that the static source scanning always errs on the side of including too many dependencies. (It is the dependency scanner used by PAR and aims at being easiest on the end-user.)
Finally, you could choose to distribute your script as a CPAN distribution. That sounds much more complicated than it really is. You can use something like Module::Starter to set up a basic skeleton of a tentative App::YourScript distribution. Put your script in the bin/ subdirectory and edit the Makefile.PL to reference all of your direct dependencies. Then, for distribution you do:
perl Makefile.PL
make
make dist
The last step generates a nice App-YourScript-VERSION.tar.gz
Now, when the client wants to install all dependencies, he does the following:
Set up the CPAN client correctly. Simply run it and answer the questions. But you're requiring that already anyway.
"tar -xz App-YourScript-VERSION.tar.gz && cd App-YourScript-VERSION"
Run "cpan ."
The CPAN client will now install all direct dependencies and the dependencies of those distributions automatically. Depending on how you set it up, it will either follow the prerequisites recursively automatically or prompt with a y/n each time.
As an example of this, you might check out a few of the App::* distributions on CPAN. I would think App::Ack is a good example. Maybe one of the App::* distributions from my CPAN directory (SMUELLER).
You could dump %INC at the end of your script. It will contain all used and required modules. But of course, this will only be helpful if you don't require modules conditionally (require Foo if $bar).
For quick-and-dirty, infrequent use, the %INC is the best way to go. If you have to do this with continuous integration testing or something more robust, there are some other tools to help.
Steffen already mentioned the Module::ScanDeps.
The code in Test::Prereq does this, but it has an additional layer that ensures that your Makefile.PL or Build.PL lists them as a dependency. If you make your scripts look like a normal Perl distribution, that makes it fairly easy to check for new dependencies; just run the test suite again.
Aside from that, you might use a tool such as Module::Extract::Use, which parses the static code looking for use and require statements (although it won't find them in string evals). That gets you just the modules you told your script to load.
Also, once you know which modules you loaded, you can combine that with David Cantrell's CPANdeps tool that has already created the dependency tree for most CPAN modules.
Note that you also have to think about optional features too. Your code in this case my not have them, but sometimes you don't load a module until you need it:
sub foo
{
require Bar; # don't load until we need to use it
....
}
If you don't exercise that feature in your trial run or test, you won't see that you need Bar for that feature. A similar problem comes up when a module loads a different set of dependency modules in a different environment (say, mod_perl or Windows, and so on).
There's not a good, automated way of testing optional features like that so you can get their dependencies. However, I think that should be on my To Do list since it sounds like an interesting problem.
Another tool in this area, which is used by Dist::Zilla and its AutoPrereqs plugin, is Perl::PrereqScanner. It installs a scan-perl-prereqs program that will use PPI and a few plugins to search for most kinds of prereq declaration, using the minimum versions you define. In general, I suggest this over scanning %INC, which can bring in bogus requirements and ignores versions.
Today I develop my Perl apps as CPAN-like distributions using Dist::Zilla that can take care of the dependencies through the AutoPrereq plugin. Another interesting piece of code in this area is carton.