Testing for LibreSSL in a Perl build script - perl

I released Net::NSCAng::Client a while ago and am getting a lot of test failures on OpenBSD. The reason for that is that the NSCAng protocol uses OpenSSL in preshared-key mode (RFC4279), something the folks at LibreSSL (default on OpenBSD now) have ripped out. However, they seem to have been hell-bent on doing this the most intransparent way: the include files have all the functions defined, just the shared library is missing the corresponding symbols, so compilation works fine but the tests fail.
There is a compatibility package on OpenBSD called eopenssl, and by testing for this first in Makefile.PL (using ExtUtils::PkgConfig) I can make it work if the compatibility library is installed. If it isn't, things still fail.
I could check for the CPP symbol OPENSSL_NO_PSK, but as the includes always come from LibreSSL, this fails even if linking with eopenssl would work fine. The only idea I have left is to try and have a test program run as part of the compilation phase as autoconf does it. Is that even possible with ExtUtils::MakeMaker (or something else -- I wouldn't mind switching the build system if necessary)?

It's easy to write feature tests with Devel::CheckLib. Something like the following can be used to check for the presence of function your_func (in Makefile.PL):
my $your_func_exists = check_lib(
header => 'your_header.h',
function => 'return your_func ? 1 : 0;',
);
If you simply want to abort compilation if the function is missing:
check_lib(
...
) or warn('your_func is missing'), exit;
Exiting with 0 should avoid a CPAN Tester's 'FAIL' report.

Related

mod_perl cannot use threaded MPM without ithreads enabled Perl

Firstly, I have asked a similar previous question years ago for an older version Compile Perl without threads but my previous answer doesn't seem to work now.
My set up is a custom compiled version of Perl
This is perl 5, version 34, subversion 1 (v5.34.1) built for x86_64-linux
CFLAGS='-m64 -mtune=nocona' ./Configure -des -A ccflags=-fPIC -Dprefix=/opt/perl
Which to my understanding shows that threads aren't enabled (I think it would say at the end if so)
I have trying to get this running with Apache 2.4.53 (but I don't think it's getting that far anyway).
Apache config options..
./configure --enable-proxy --enable-rewrite --enable-headers --enable-ssl --with-apr=/usr/local/apr/
As far as I'm aware, and previous answer, was if using Perl without threads, you can compile mod_perl without threads also, this is done by using
mod_perl version is 2.0.12
MP_NO_THREADS=1
So my full command for trying to compile mod_perl is
perl Makefile.PL MP_NO_THREADS=1 MP_APXS=/usr/local/apache2/bin/apxs
the readme says...
# For httpd-2.4, we can't use mpm_is_threaded(), because MPMs are loadable
# modules. We therefore treat httpd as a whole project as threaded. It is
# still possible to disable threading by using MP_NO_THREADS=1
I can also see a bit of code in the Makefile.PL
if ($build->{MP_NO_THREADS}) {
$build_threaded = 0;
}
and I can see $build_threaded indeed gets set to 0.
However, when I
make && make test
I get
[Thu Apr 28 12:18:02.392480 2022] [perl:error] [pid 38185:tid 140616570507840]
cannot use threaded MPM without ithreads enabled Perl
I can't really see anything in the code (other than checks) that does anything different depending on that code though, but I'm not familiar with it at all, so probably missing something important ?
I have also tried with -Uuseithreads -Uusethreads -Dusethreads=undef
Edit: the bit of code producing the error looks like
## src/modules/perl/mod_perl.c
#ifndef USE_ITHREADS
if (modperl_threaded_mpm()) {
ap_log_error(APLOG_MARK, APLOG_ERR, 0, base_server,
"cannot use threaded MPM without ithreads enabled Perl");
exit(1);
}
#endif
and
Apache2::Build::PERL_HAS_ITHREADS ? "w/" : "w/o";
shows as "w/o" when running
Digging further I see the code seems to call ap_mpm_query() so checking with Apache and in apachectl -v this looks incorrect...I think threaded should be no, so checking on why that is..
Server MPM: event
threaded: yes (fixed thread count)
forked: yes (variable process count)
Which is odd..so I think
Looks like something changed with Apache, so seem to have to specify the mpm mode, so if I change Apache compile to --with-mpm=prefork this seems to work.
./configure --enable-proxy --enable-rewrite --enable-headers --enable-ssl --with-apr=/usr/local/apr/ --with-mpm=prefork

Missing a lot of files in Perl kit

EDIT: Solution was "apt-get install libwww-per" from charlesbridge in the comments
EDIT: Attempted to install by both going into the cpan console and running install LWP::Simple as well as perl -MCPAN -e'install "LWP::Simple"' (from here).
The first time I ran CPAN I just let it configure itself with all the defaults. The below is not a description of the dependencies but all of the missing dependencies are included in this warning message, so I figure if I can solve that then I can solve the rest of it.
Original post:
I've been trying to install some missing Perl packages (namely LWP::Simple) but every time I do it spits back a trillion errors and ends with "Missing 17 dependencies". Most (if not all) are listed below at the very start of the install process. I've tried installing individual ones but I get similar dependency errors. Is there some way to fix this?
Checking if your kit is complete...
Warning: the following files are missing in your kit:
eg/hanchors
eg/hdump
eg/hform
eg/hlc
eg/hrefsub
eg/hstrip
eg/htext
eg/htextsub
eg/htitle
hints/solaris.pl
lib/HTML/Entities.pm
lib/HTML/Filter.pm
lib/HTML/HeadParser.pm
lib/HTML/LinkExtor.pm
lib/HTML/PullParser.pm
lib/HTML/TokeParser.pm
t/api_version.t
t/argspec-bad.t
t/argspec.t
t/argspec2.t
t/attr-encoded.t
t/callback.t
t/case-sensitive.t
t/cases.t
t/comment.t
t/crashme.t
t/declaration.t
t/default.t
t/document.t
t/dtext.t
t/entities.t
t/entities2.t
t/filter-methods.t
t/filter.t
t/handler-eof.t
t/handler.t
t/headparser-http.t
t/headparser.t
t/ignore.t
t/largetags.t
t/linkextor-base.t
t/linkextor-rel.t
t/magic.t
t/marked-sect.t
t/msie-compat.t
t/offset.t
t/options.t
t/parsefile.t
t/parser.t
t/plaintext.t
t/pod.t
t/process.t
t/pullparser.t
t/script.t
t/skipped-text.t
t/stack-realloc.t
t/textarea.t
t/threads.t
t/tokeparser.t
t/uentities.t
t/unbroken-text.t
t/unicode-bom.t
t/unicode.t
t/xml-mode.t
Please inform the author.

Why can't I debug my puppet code?

I'm learning Puppet and the biggest frustration I have with the entire paradigm is the try/run/fix development process I'm using to build functional Puppet code. My background is in Java and I'm naturally use to debugging my code to find errors instead of just running the program to see where it bombs making development much faster but I can't seem to find a way to do this using Puppet and Eclipse. I know writing a debugger for Puppet would require some creativity given its nature but I think this is something the community could really benefit from.
I've written debuggers and know the Eclipse SDK but unfortunately it does not map cleanly to the Puppet architecture which is a bit awkward in the sense its runtime stack and execution flow does not happen in natural order as well as the fact the runtime requires a target machine to apply changes on.
I'm curious if the community has done any development work on trying to create some kind of debugger where code can be stepped. To write this I think it would make sense to extend Eclipse with a new Puppet debug configuration type where you specify a target sandbox host to test your code as well as a puppet project in your workspace you want to debug (leveraging existing Gepetto tooling). Then when you start a new Puppet debugging session Eclipse could connect to the remote host, execute puppet apply with some additional debug arguments and somehow provide feedback from the runtime about what line of code is currently being executed.
This still might be awkward but would allow puppet developers to quickly see things like oh duh.. I can't create this directory because the parent path does not exist, wait... why is this if statement not going here like I planned, oh I see here that Puppet is not very clear on single or double quotes or now I see why this fails because this class was not executed first etc. etc.
Instead all we get is a big ugly output on the agent console that yes can give us insight on errors but does not cleanly map exceptions to our code that in my view shows an underlying pain and weakness of Puppet, can you at least give me a stack trace and line number so I know where to look? Nope sorry.
Don't get me wrong, I love how Puppet can make me look very productive throughout the work week when all I'm doing is running Puppet apply on new machines which my manager has not yet figured out but I think for Puppet to really be useful this lack of debugging support is something that needs to be addressed.
Does anyone else feel this pain? - Duncan Krebs
It would be impossible to "step through" puppet code, unless you want to debug against the ruby codebase itself. It's not just that the order of "execution" is unclear, its that the manifest themselves are never executed at a single time. They are actually evaluated in multiple phases throughout execution.
There are ways to simplify finding problems though. The biggest one is writing unit tests using rspec-puppet. It lets you essentially test the compilation phase of puppet, helping you catch errors like circular dependencies, incorrect conditional logic, etc.
There is a new tool called the puppet-debugger which allows you to set breakpoints in your puppet code in order to step through it. So this is no longer "impossible" as it has been available for about 8 months.
You will first need to install the puppet-debugger gem
https://github.com/nwops/puppet-debugger
Then install the debug module, include it in your fixtures or just ensure it is in your module path.
https://forge.puppet.com/nwops/debug
Then just set a breakpoint in your code using debug::break() function.
Ruby Version: 2.0.0
Puppet Version: 4.9.4
Puppet Debugger Version: 0.6.0
Created by: NWOps
Type "exit", "functions", "vars", "krt", "whereami", "facts", "resources", "classes",
"play", "classification", "types", "datatypes", "reset", or "help" for more information.
>> $vars = ['one', 'two', 'three']
=> [
[0] "one",
[1] "two",
[2] "three"
]
>> $vars.each | String $var | {
debug::break()
notify{$var:}
}
From file: puppet_debugger_input20170417-97123-qjwbaj.pp
1: $vars.each | String $var | {
=> 2: debug::break()
3: notify{$var:}
4: }
1:>> $var
=> "one"
2:>> exit
From file: puppet_debugger_input20170417-97123-qjwbaj.pp
1: $vars.each | String $var | {
=> 2: debug::break()
3: notify{$var:}
4: }
1:>> $var
=> "two"
2:>> exit
From file: puppet_debugger_input20170417-97123-qjwbaj.pp
1: $vars.each | String $var | {
=> 2: debug::break()
3: notify{$var:}
4: }
1:>> $var
=> "three"

I keep getting failures installing modules at BEGIN { plan tests => 5 }. What do I need to get past this?

The module it's failing to install is JSON::XS. Really it's failing to install anything
that has the following code:
BEGIN { plan tests => 5 };
From the build.log:
syntax error at t/04_dwiw_encode.t line 13, near "plan tests"
The offending line:
13 BEGIN { plan tests => 5 }
I read that there's a problem with Test.pm but there are quite a few modules
using it and furthermore this just started happening recently.
I just tried reinstalling perlbrew and also tried updating outdated modules
but I keep getting the same failures.
Any one have an idea what might have caused this and how to fix it?
I suspect you either have an older-than-expected version of the Test module, or you created your own module named Test.pm and it's getting picked up instead of the expected module.
You can address the first issue by upgrading Test.
cpan Test
You should address the second issue by renaming your Test.pm to something else, but you might also be able to address it by changing directory and temporarily clearing the PERL5LIB env var.
pushd / ; PERL5LIB= cpan JSON::XS ; popd

DBD::Informix connection issues

I'm having somewhat weird problem with DBD::Informix. If I run a simple script like that:
use DBI;
my $dbh = DBI->connect_cached('dbi:Informix:database', '', '');
my $sth = $dbh->prepare('select foo from bar');
...
It works all right. But if I try to do exactly the same from a test script it fails with the following message:
SQL: -25588: The appl process cannot connect to the database server cms_ol.
ISAM: 22: Invalid argument
The only difference I see is that test script is quite heavy on module usage; it is based on Test::More and loads a lot of submodules that are to be tested.
Turning on DBI trace does not provide anything useful (for me, at least). Simple script runs along just fine:
DBI 1.616-nothread default trace level set to 0x0/1 (pid 9685 pi 0) at test_ifx.pl line 6
Note: perl is running without the recommended perl -w option
-> DBI->connect(dbi:Informix:cms#cms_ol, , ****, HASH(0x13fad0))
-> DBI->install_driver(Informix) for solaris perl=5.008009 pid=9685 ruid=106 euid=106
install_driver: DBD::Informix version 2011.0612 loaded from /cms/webdash/lib/arch/DBD/Informix.pm
<- install_driver= DBI::dr=HASH(0x1c8070)
!! warn: 0 CLEARED by call to connect method
-->> DBD::Informix::dbd_ix_db_connect()
CONNECT TO 'cms#cms_ol' - no user info
-->> DBD::Informix::dbd_ix_db_check_for_autocommit()
... and the only difference in trace of the problematic script I see is that it just fails:
DBI 1.616-nothread default trace level set to 0x0/1 (pid 9687 pi 0) at 22_report.t line 5 via 22_report.t line 6
Note: perl is running without the recommended perl -w option
-> DBI->connect_cached(dbi:Informix:cms, , ****)
-> DBI->install_driver(Informix) for solaris perl=5.008009 pid=9687 ruid=106 euid=106
install_driver: DBD::Informix version 2011.0612 loaded from /cms/webdash/lib/arch/DBD/Informix.pm
<- install_driver= DBI::dr=HASH(0xb619bc)
!! warn: 0 CLEARED by call to connect_cached method
-->> DBD::Informix::dbd_ix_db_connect()
CONNECT TO 'cms' - no user info
***ERROR***
SQL: -25588: The appl process cannot connect to the database server cms_ol.
ISAM: 22: Invalid argument
<<-- dbd_ix_db_connect (**ERROR-1**)
<<-- DBD::Informix::dbd_ix_db_connect()
I'm running custom Perl 5.8.9 build in Solaris 9, with latest DBI and DBD::Informix versions, against Informix IDS 9.40UC.
Update: If I try to be a smartass and put a block like that at the top of the heavy test script:
use DBI;
BEGIN { my $dbh = DBI->connect_cached( ... ); print "Connected!\n" if $dbh; }
... it prints like this:
Connected!
Out of memory!
Callback called exit.
END failed--call queue aborted at t/22_report.t line 20.
Callback called exit at t/22_report.t line 20.
BEGIN failed--compilation aborted at t/22_report.t line 24.
My guess is that DBD::Informix conflicts with some of the modules loaded after the connection is made. But which one? That's the question...
Another update: It appears that the above trick does something unwieldy. I tried to load all the modules explicitly by replacing 'use Module' with 'require Module; Module->import'. Pure Perl modules are OK but whenever XS module using XSLoader appears, Perl goes boom with friendly 'Out of memory' message. And if I move Informix connection below module initialization, it works all right - except DBD::Informix fails with the same -25588 error. Boomer. I'm at loss. :(
Another another update: I tried to run the same script with standard Perl 5.6.1 shipped with Solaris 9, using DBI 1.601 (the latest that would compile with Perl 5.6) and DBD::Informix 2011.0612. Same thing, so it's not custom Perl giving me trouble.
I can also add that the test module in question was prototyped using DBD::SQLite and fully works. It is the final test with DBD::Informix that is failing... As usual. :/
Workaround: following e-mail discussion with Jonathan, a workaround was found: addition of streams-based 'onipcstr' connection to Informix server allowed DBD::Informix to connect. Apparently, some XS modules interfere with default shmem-based connection method, although the culprit is unknown at the moment.
Further discussion
Custom-built Perl is, in my experience, easier than the system Perl. I never modify the system's Perl installation (I don't want to break it) so I always build my own.
You appear to have:
Solaris 9 (SPARC?)
Perl 5.8.9
DBI 1.616
DBD::Informix 2011.0612
ESQL/C (CSDK) 2.81
Informix Dynamic Server 9.40
We don't have the detailed sub-version of ESQL/C and IDS (2.81.UC2, 9.40.UC5, or whatever). There's a hint that you are using a 32-bit version of IDS, so probably everything is 32-bit. You are probably aware that 9.40 is no longer supported by IBM (and, indeed, its successor version 10.00 is also out of support). However, superficially, none of that should matter very much. The failing t91lvarchar.t is not a big issue.
Can you run the connect in working and non-working modes with DBI_TRACE=9 set in the environment.
If the trace for the connect operation is too voluminous to go into an update to the question, we'd better take this off-line to the DBD::Informix support channels (that's me, but by email).
The 'ISAM' error of 22 (Invalid argument) is puzzling. I'm curious about what is in your sqlhosts file for this server - the entry for cms_ol specifically. I'm not sure it will reveal anything, not least because you say the sample ESQL/C below (in the 'First hypothesis' section) works OK, and sometimes the Perl connects and sometimes it does not.
I wonder if there is a name conflict somewhere between functions in the shared libraries? That sort of thing will be hell to track.
First hypothesis
Further information received shows that this was not the crucial distinction.
The difference appears to be:
Works: CONNECT TO 'cms#cms_ol' - no user info
Fails: CONNECT TO 'cms' - no user info
The tricky part to explain is why the second fails, especially as the error goes on to mention cms_ol.
The workaround is to specify the server name in the connect string:
DBI->connect(dbi:Informix:cms#cms_ol, , ****, HASH(0x13fad0))
DBI->connect_cached(dbi:Informix:cms, , ****)
The underlying problem is more likely at the ESQL/C level than anything to do with other Perl modules. That is, if you compiled and executed this ESQL/C program, it would fail on cms and work on cms#cms_ol:
int main(int argc, char **argv)
{
$ char *dbs = "cms";
if (argc > 1)
dbs = argv[1];
$ whenever error stop;
$ connect to :dbs;
return 0;
}
You could run it without an explicit database name (or with an explicit 'cms'), and I would expect it to fail. You could run it with 'cms#cms_ol' and I would expect it to pass. The program will say nothing if it passes; it will be obvious when it fails (though the messages will not be beautiful).
There is an outside chance it is something to do with connect_cached; that is a service provided by the DBI module and not by the DBD::Informix module. On the whole though, it is more likely something happening at the ESQL/C level.