Why can't I debug my puppet code? - eclipse

I'm learning Puppet and the biggest frustration I have with the entire paradigm is the try/run/fix development process I'm using to build functional Puppet code. My background is in Java and I'm naturally use to debugging my code to find errors instead of just running the program to see where it bombs making development much faster but I can't seem to find a way to do this using Puppet and Eclipse. I know writing a debugger for Puppet would require some creativity given its nature but I think this is something the community could really benefit from.
I've written debuggers and know the Eclipse SDK but unfortunately it does not map cleanly to the Puppet architecture which is a bit awkward in the sense its runtime stack and execution flow does not happen in natural order as well as the fact the runtime requires a target machine to apply changes on.
I'm curious if the community has done any development work on trying to create some kind of debugger where code can be stepped. To write this I think it would make sense to extend Eclipse with a new Puppet debug configuration type where you specify a target sandbox host to test your code as well as a puppet project in your workspace you want to debug (leveraging existing Gepetto tooling). Then when you start a new Puppet debugging session Eclipse could connect to the remote host, execute puppet apply with some additional debug arguments and somehow provide feedback from the runtime about what line of code is currently being executed.
This still might be awkward but would allow puppet developers to quickly see things like oh duh.. I can't create this directory because the parent path does not exist, wait... why is this if statement not going here like I planned, oh I see here that Puppet is not very clear on single or double quotes or now I see why this fails because this class was not executed first etc. etc.
Instead all we get is a big ugly output on the agent console that yes can give us insight on errors but does not cleanly map exceptions to our code that in my view shows an underlying pain and weakness of Puppet, can you at least give me a stack trace and line number so I know where to look? Nope sorry.
Don't get me wrong, I love how Puppet can make me look very productive throughout the work week when all I'm doing is running Puppet apply on new machines which my manager has not yet figured out but I think for Puppet to really be useful this lack of debugging support is something that needs to be addressed.
Does anyone else feel this pain? - Duncan Krebs

It would be impossible to "step through" puppet code, unless you want to debug against the ruby codebase itself. It's not just that the order of "execution" is unclear, its that the manifest themselves are never executed at a single time. They are actually evaluated in multiple phases throughout execution.
There are ways to simplify finding problems though. The biggest one is writing unit tests using rspec-puppet. It lets you essentially test the compilation phase of puppet, helping you catch errors like circular dependencies, incorrect conditional logic, etc.

There is a new tool called the puppet-debugger which allows you to set breakpoints in your puppet code in order to step through it. So this is no longer "impossible" as it has been available for about 8 months.
You will first need to install the puppet-debugger gem
https://github.com/nwops/puppet-debugger
Then install the debug module, include it in your fixtures or just ensure it is in your module path.
https://forge.puppet.com/nwops/debug
Then just set a breakpoint in your code using debug::break() function.
Ruby Version: 2.0.0
Puppet Version: 4.9.4
Puppet Debugger Version: 0.6.0
Created by: NWOps
Type "exit", "functions", "vars", "krt", "whereami", "facts", "resources", "classes",
"play", "classification", "types", "datatypes", "reset", or "help" for more information.
>> $vars = ['one', 'two', 'three']
=> [
[0] "one",
[1] "two",
[2] "three"
]
>> $vars.each | String $var | {
debug::break()
notify{$var:}
}
From file: puppet_debugger_input20170417-97123-qjwbaj.pp
1: $vars.each | String $var | {
=> 2: debug::break()
3: notify{$var:}
4: }
1:>> $var
=> "one"
2:>> exit
From file: puppet_debugger_input20170417-97123-qjwbaj.pp
1: $vars.each | String $var | {
=> 2: debug::break()
3: notify{$var:}
4: }
1:>> $var
=> "two"
2:>> exit
From file: puppet_debugger_input20170417-97123-qjwbaj.pp
1: $vars.each | String $var | {
=> 2: debug::break()
3: notify{$var:}
4: }
1:>> $var
=> "three"

Related

Testing for LibreSSL in a Perl build script

I released Net::NSCAng::Client a while ago and am getting a lot of test failures on OpenBSD. The reason for that is that the NSCAng protocol uses OpenSSL in preshared-key mode (RFC4279), something the folks at LibreSSL (default on OpenBSD now) have ripped out. However, they seem to have been hell-bent on doing this the most intransparent way: the include files have all the functions defined, just the shared library is missing the corresponding symbols, so compilation works fine but the tests fail.
There is a compatibility package on OpenBSD called eopenssl, and by testing for this first in Makefile.PL (using ExtUtils::PkgConfig) I can make it work if the compatibility library is installed. If it isn't, things still fail.
I could check for the CPP symbol OPENSSL_NO_PSK, but as the includes always come from LibreSSL, this fails even if linking with eopenssl would work fine. The only idea I have left is to try and have a test program run as part of the compilation phase as autoconf does it. Is that even possible with ExtUtils::MakeMaker (or something else -- I wouldn't mind switching the build system if necessary)?
It's easy to write feature tests with Devel::CheckLib. Something like the following can be used to check for the presence of function your_func (in Makefile.PL):
my $your_func_exists = check_lib(
header => 'your_header.h',
function => 'return your_func ? 1 : 0;',
);
If you simply want to abort compilation if the function is missing:
check_lib(
...
) or warn('your_func is missing'), exit;
Exiting with 0 should avoid a CPAN Tester's 'FAIL' report.

How to get Resharpers InspectCode to recognize Plugins?

I am trying to run ReSharpers command line tool InspectCode.exe. It's running fine doing it's job with the expected output.
However after my earlier attempt to get plugins to work, this time with the new version it is supposed to be supported. There is a switch in the command line interface that allows to specify the extension you want to use.
/extensions (/x) – allows using ReSharper extensions that affect code analysis. To use an extension, specify its ID, which you can find by opening the extension package page in the ReSharper Gallery, and then the Package Statistics page. Multiple values are separated with the semicolon.
But I cannot get it to work properly. I cannot even provoke any reaction to the /x switch at all. No matter how or what I pass, I get no feedback from the executable and the output is identical. I don't even get an error message when passing obvious garbage.
I tried the following commandlines for the exact same result:
inspectcode.exe /o="rcli.xml" /swea /x="ReSharper.StyleCop" "my.sln"
inspectcode.exe /o="rcli.xml" /swea /x=ReSharper.StyleCop "my.sln"
inspectcode.exe /o="rcli.xml" /swea "my.sln"
inspectcode.exe /o="rcli.xml" /swea /x=ABCDEFG "my.sln"
Result
JetBrains Inspect Code 9.1.1
Running in 64-bit mode, .NET runtime 4.0.30319.18444 under Microsoft Windows NT
6.1.7601 Service Pack 1
Enabled solution-wide analysis according to Inspect Code command line Setting.
Analyzing files
[files]
Inspection report was written to rcli.xml
What am I doing wrong? How to get extensions to work?
I already tried the R# forums, but it took them more then 24h to approve my post and so far I'm not sure someone else even read it.
Unfortunately, the support for extensions was dropped in 9.0 due to the refactorings in the "ReSharper platform". I hope that JetBrains will bring it back soon.
See RSRP-436208.
This is a late answer that might help future readers (like myself). Currently inspectcode.exe will automatically look for and use any NuGet packages that are in the same folder as the executable (source).
Example for CleanCode extension:
if you have a R# instance on some machine and install the extension, it will be placed in C:\Users\{user}\AppData\Local\JetBrains\plugins\MO.CleanCode.5.6.15
copy MO.CleanCode.5.6.15.nupkg and paste it next to inspectcode.exe
when running inspectcode with verbosity = VERBOSE, the extension should appear in the Zones list:
$cmd = "..\JetBrains.ReSharper.CommandLineTools.2019.3.4\inspectcode.exe"
$outputFile = "..\Output\$($outputName).xml"
& $cmd -o="$outputFile" $sln --verbosity=VERBOSE
Zones: (52pcs)[CodeInspectionPageImplZone, DaemonEngineZone,
DaemonZone, IAmd64CpuArchitectureHostZone, IAspMvcZone,
IBatchToolEnvironmentZone, IClrImplementationHost Zone,
IClrPsiLanguageZone, ICodeEditingOptionsPageImplZone,
IConsoleEnvironmentZone, ICppProductZone, ICpuArchitectureHostZone,
IDocumentModelZone, IEnvironmentZone, IHostSolutionZone,
IInspectCodeConsoleEnvironmentZone, IInspectCodeEnvironmentZone,
IInspectCodeZone, ILanguageAspZone, ILanguageBuildScriptsZone,
ILanguageCppZone, I LanguageCSharpZone, ILanguageCssZone,
ILanguageHtmlZone, ILanguageIlZone, ILanguageJavaScriptZone,
ILanguageMsBuildZone, ILanguageNAntZone, ILanguageProtobufZone, ILa
nguageRazorZone, ILanguageRegExpZone, ILanguageResxZone,
ILanguageVBZone, ILanguageXamlZone, INetFrameworkHostZone, INuGetZone,
IOperatingSystemHostZone, IProjectMode lZone,
IPsiAssemblyFileLoaderImplZone, IPsiLanguageZone,
IPublicVisibilityZone, IRdFrameworkZone, IRiderModelZone,
ISinceClr2HostZone, ISinceClr4HostZone, ITextContro lsZone,
IToolsOptionsPageImplZone, IWebPsiLanguageZone, IWindowsNtHostZone,
PsiFeaturesImplZone, ReplaceableByIntelliJPlatformZone, SweaZone]
Packages: (23pcs)[JetBrains.ExternalAnnotations,
JetBrains.Platform.Core.Ide, JetBrains.Platform.Core.IisExpress,
JetBrains.Platform.Core.MsBuild, JetBrains.Platform. Core.Shell,
JetBrains.Platform.Core.Text, JetBrains.Platform.Interop.CommandLine,
JetBrains.Platform.Interop.dotMemoryUnit.Framework,
JetBrains.Platform.Interop.dotMe moryUnit.Interop.Console,
JetBrains.Platform.Interop.dotMemoryUnit.Interop.Ide,
JetBrains.Platform.RdProtocol, JetBrains.Psi.Features.Core,
JetBrains.Psi.Features.Cpp .Src.Core, JetBrains.Psi.Features.src,
JetBrains.Psi.Features.Tasks, JetBrains.Psi.Features.UnitTesting,
JetBrains.Psi.Features.Web.Core, JetBrains.ReSharperAutomatio
nTools.src.CleanupCode,
JetBrains.ReSharperAutomationTools.src.CommandLineCore,
JetBrains.ReSharperAutomationTools.src.CommandLineProducts,
JetBrains.ReSharperAutomat ionTools.src.DuplicatesFinder,
JetBrains.ReSharperAutomationTools.src.InspectCode, MO.CleanCode]

I keep getting failures installing modules at BEGIN { plan tests => 5 }. What do I need to get past this?

The module it's failing to install is JSON::XS. Really it's failing to install anything
that has the following code:
BEGIN { plan tests => 5 };
From the build.log:
syntax error at t/04_dwiw_encode.t line 13, near "plan tests"
The offending line:
13 BEGIN { plan tests => 5 }
I read that there's a problem with Test.pm but there are quite a few modules
using it and furthermore this just started happening recently.
I just tried reinstalling perlbrew and also tried updating outdated modules
but I keep getting the same failures.
Any one have an idea what might have caused this and how to fix it?
I suspect you either have an older-than-expected version of the Test module, or you created your own module named Test.pm and it's getting picked up instead of the expected module.
You can address the first issue by upgrading Test.
cpan Test
You should address the second issue by renaming your Test.pm to something else, but you might also be able to address it by changing directory and temporarily clearing the PERL5LIB env var.
pushd / ; PERL5LIB= cpan JSON::XS ; popd

DBD::Informix connection issues

I'm having somewhat weird problem with DBD::Informix. If I run a simple script like that:
use DBI;
my $dbh = DBI->connect_cached('dbi:Informix:database', '', '');
my $sth = $dbh->prepare('select foo from bar');
...
It works all right. But if I try to do exactly the same from a test script it fails with the following message:
SQL: -25588: The appl process cannot connect to the database server cms_ol.
ISAM: 22: Invalid argument
The only difference I see is that test script is quite heavy on module usage; it is based on Test::More and loads a lot of submodules that are to be tested.
Turning on DBI trace does not provide anything useful (for me, at least). Simple script runs along just fine:
DBI 1.616-nothread default trace level set to 0x0/1 (pid 9685 pi 0) at test_ifx.pl line 6
Note: perl is running without the recommended perl -w option
-> DBI->connect(dbi:Informix:cms#cms_ol, , ****, HASH(0x13fad0))
-> DBI->install_driver(Informix) for solaris perl=5.008009 pid=9685 ruid=106 euid=106
install_driver: DBD::Informix version 2011.0612 loaded from /cms/webdash/lib/arch/DBD/Informix.pm
<- install_driver= DBI::dr=HASH(0x1c8070)
!! warn: 0 CLEARED by call to connect method
-->> DBD::Informix::dbd_ix_db_connect()
CONNECT TO 'cms#cms_ol' - no user info
-->> DBD::Informix::dbd_ix_db_check_for_autocommit()
... and the only difference in trace of the problematic script I see is that it just fails:
DBI 1.616-nothread default trace level set to 0x0/1 (pid 9687 pi 0) at 22_report.t line 5 via 22_report.t line 6
Note: perl is running without the recommended perl -w option
-> DBI->connect_cached(dbi:Informix:cms, , ****)
-> DBI->install_driver(Informix) for solaris perl=5.008009 pid=9687 ruid=106 euid=106
install_driver: DBD::Informix version 2011.0612 loaded from /cms/webdash/lib/arch/DBD/Informix.pm
<- install_driver= DBI::dr=HASH(0xb619bc)
!! warn: 0 CLEARED by call to connect_cached method
-->> DBD::Informix::dbd_ix_db_connect()
CONNECT TO 'cms' - no user info
***ERROR***
SQL: -25588: The appl process cannot connect to the database server cms_ol.
ISAM: 22: Invalid argument
<<-- dbd_ix_db_connect (**ERROR-1**)
<<-- DBD::Informix::dbd_ix_db_connect()
I'm running custom Perl 5.8.9 build in Solaris 9, with latest DBI and DBD::Informix versions, against Informix IDS 9.40UC.
Update: If I try to be a smartass and put a block like that at the top of the heavy test script:
use DBI;
BEGIN { my $dbh = DBI->connect_cached( ... ); print "Connected!\n" if $dbh; }
... it prints like this:
Connected!
Out of memory!
Callback called exit.
END failed--call queue aborted at t/22_report.t line 20.
Callback called exit at t/22_report.t line 20.
BEGIN failed--compilation aborted at t/22_report.t line 24.
My guess is that DBD::Informix conflicts with some of the modules loaded after the connection is made. But which one? That's the question...
Another update: It appears that the above trick does something unwieldy. I tried to load all the modules explicitly by replacing 'use Module' with 'require Module; Module->import'. Pure Perl modules are OK but whenever XS module using XSLoader appears, Perl goes boom with friendly 'Out of memory' message. And if I move Informix connection below module initialization, it works all right - except DBD::Informix fails with the same -25588 error. Boomer. I'm at loss. :(
Another another update: I tried to run the same script with standard Perl 5.6.1 shipped with Solaris 9, using DBI 1.601 (the latest that would compile with Perl 5.6) and DBD::Informix 2011.0612. Same thing, so it's not custom Perl giving me trouble.
I can also add that the test module in question was prototyped using DBD::SQLite and fully works. It is the final test with DBD::Informix that is failing... As usual. :/
Workaround: following e-mail discussion with Jonathan, a workaround was found: addition of streams-based 'onipcstr' connection to Informix server allowed DBD::Informix to connect. Apparently, some XS modules interfere with default shmem-based connection method, although the culprit is unknown at the moment.
Further discussion
Custom-built Perl is, in my experience, easier than the system Perl. I never modify the system's Perl installation (I don't want to break it) so I always build my own.
You appear to have:
Solaris 9 (SPARC?)
Perl 5.8.9
DBI 1.616
DBD::Informix 2011.0612
ESQL/C (CSDK) 2.81
Informix Dynamic Server 9.40
We don't have the detailed sub-version of ESQL/C and IDS (2.81.UC2, 9.40.UC5, or whatever). There's a hint that you are using a 32-bit version of IDS, so probably everything is 32-bit. You are probably aware that 9.40 is no longer supported by IBM (and, indeed, its successor version 10.00 is also out of support). However, superficially, none of that should matter very much. The failing t91lvarchar.t is not a big issue.
Can you run the connect in working and non-working modes with DBI_TRACE=9 set in the environment.
If the trace for the connect operation is too voluminous to go into an update to the question, we'd better take this off-line to the DBD::Informix support channels (that's me, but by email).
The 'ISAM' error of 22 (Invalid argument) is puzzling. I'm curious about what is in your sqlhosts file for this server - the entry for cms_ol specifically. I'm not sure it will reveal anything, not least because you say the sample ESQL/C below (in the 'First hypothesis' section) works OK, and sometimes the Perl connects and sometimes it does not.
I wonder if there is a name conflict somewhere between functions in the shared libraries? That sort of thing will be hell to track.
First hypothesis
Further information received shows that this was not the crucial distinction.
The difference appears to be:
Works: CONNECT TO 'cms#cms_ol' - no user info
Fails: CONNECT TO 'cms' - no user info
The tricky part to explain is why the second fails, especially as the error goes on to mention cms_ol.
The workaround is to specify the server name in the connect string:
DBI->connect(dbi:Informix:cms#cms_ol, , ****, HASH(0x13fad0))
DBI->connect_cached(dbi:Informix:cms, , ****)
The underlying problem is more likely at the ESQL/C level than anything to do with other Perl modules. That is, if you compiled and executed this ESQL/C program, it would fail on cms and work on cms#cms_ol:
int main(int argc, char **argv)
{
$ char *dbs = "cms";
if (argc > 1)
dbs = argv[1];
$ whenever error stop;
$ connect to :dbs;
return 0;
}
You could run it without an explicit database name (or with an explicit 'cms'), and I would expect it to fail. You could run it with 'cms#cms_ol' and I would expect it to pass. The program will say nothing if it passes; it will be obvious when it fails (though the messages will not be beautiful).
There is an outside chance it is something to do with connect_cached; that is a service provided by the DBI module and not by the DBD::Informix module. On the whole though, it is more likely something happening at the ESQL/C level.

Erlang: How to access CLI flags (arguments) as application environment variables?

How does one access command line flag (arguments) as environment variables in Erlang. (As flags, not ARGV) For example:
RabbitMQ cli looks something like:
erl \
...
-sasl errlog_type error \
-sasl sasl_error_logger '{file,"'${RABBITMQ_SASL_LOGS}'"}' \
... # more stuff here
If one looks at sasl.erl you see the line:
get_sasl_error_logger() ->
case application:get_env(sasl, sasl_error_logger) of
% ... etc
By some unknown magic the sasl_error_logger variable becomes an erlang tuple! I've tried replicating this in my own erlang application, but I seem to be only able to access these values via init:get_argument, which returns the value as a string.
How does one pass in values via the commandline and be able to access them easily as erlang terms?
UPDATE Also for anyone looking, to use environment variables in the 'regular' way use os:getenv("THE_VAR")
Make sure you set up an application configuration file
{application, fred,
[{description, "Your application"},
{vsn, "1.0"},
{modules, []},
{registered,[]},
{applications, [kernel,stdlib]},
{env, [
{param, 'fred'}
]
...
and then you can set your command line up like this:
-fred param 'billy'
I think you need to have the parameter in your application configuration to do this - I've never done it any other way...
Some more info (easier than putting it in a comment)
Given this
{emxconfig, {ets, [{keypos, 2}]}},
I can certainly do this:
{ok, {StorageType, Config}} = application:get_env(emxconfig),
but (and this may be important) my application is started at this time (may actually just need to be loaded and not actually started from looking at the application_controller code).