I'm currently debugging a Dockerfile and it would be really helpful if I could print things on the console during the build process.
To be specific: I have two different build-scripts depending on what branch is beeing build and I have to make the distinction inside the Dockerfile. As I understand it, I need to use Powershell syntax to put functionality into the Dockerfile, so I wrote the following:
ARG branch
RUN IF ($branch -eq 'alpha') {./alphabuild.bat}
RUN IF ($branch -eq 'rc') {./rcbuild.bat}
While the syntax finally seems to be correct (I never did Powershell scripting before), it still doesn't actually execute the build script when calling
docker build -t myapp:test --build-arg branch=alpha .
It would be really helpful to be able to print the result of the conditions, or the build argument itself to find out where the bug is.
Edit: Not a duplicate of this question, as I'm building a Windows container and I would really like to print stuff. Had several situations where that would come in handy.
Related
I am attempting to use the SVN_Load_Dirs.PL script file (https://svn.apache.org/repos/asf/subversion/trunk/contrib/client-side/svn_load_dirs/) to attempt to merge a platform drop.
However, I can't get the --glob_ignores flag to behave as I'd expect, and I know so little perl that I can't dig into the script to understand why. The format I am using is:
--glob_ignores=*.jazzignore
Where I want to ignore all .jazzignore files (although I am fine with anything with "jazzignore" in either the extension or name being ignored. I've looked for examples but can't find any actual usage of this flag anywhere. What I am looking for is a way to ignore all .jazzignore files and a few entire directories (like jazz5 for an example)
I assumed the flag would then be --glob_ignores="*.jazzignore *.jazz5" but that doesn't appear to be working.
It turns out that the script was failing silently because I was running it from a windows cmd (and therefore never getting to the flag option). Apparently there are some issues running it this way and it was designed to be run from a linux command line. After switching it works perfectly.
So the larger context of this problem is that it isn't possible, for whatever reason, to decrypt this file using, say, Bouncy Castle, so we're trying to do an automated command line with the normal gpg utility instead... I originally thought that would be quicker than trying to figure out why Bouncy Castle doesn't believe this is a real PGP-encrypted file, but I might have been wrong.
Here's the pipeline:
echo password | gpg --batch --yes --passphrase-fd 0 "filename"
This works perfectly in Powershell. Actually, several variations on this work perfectly in Powershell, but that's not the point...
The point is that I'm trying to run this in cmd.exe and it doesn't work there. Instead, I get an error saying that there has been no password provided and that, therefore, there is no secret key available and that, therefore, the file cannot be decrypted.
Given that the instructions I read for this are specifically for cmd.exe (not Powershell), I'm more than a little confused. Any idea what's going on here?
Apparently, the problem is that the password being passed through the pipeline includes a space--the one that appears between our hypothetical "d" and the pipe symbol itself. :)
So, for future reference, this works:
echo password|gpg --batch --yes --passphrase-fd 0 "filename"
Which, by the way, is exactly what the guide had said, but which I never caught onto because I did my initial testing in Powershell and didn't realize how picky cmd's echo command could be.
I have this command that I load (example.sh) works well in the unix command line.
However, if I execute it in Perl using the system or ` syntax, it doesn't work.
I am guessing certain settings like environment variables and other external sh files weren't loaded.
Is there an example coding to ensure it will work?
More Updates on coding execution failure (I have been trying with different codes):
push (#JOBSTORUN, "cd $a/$b/$c/$d; loadproject cats; sleep 60;");
...
my $pm = new Parallel::ForkManager(3);
foreach my $job (#JOBSTORUN) {
$pm->start and next;
print(`$job`);
$pm->finish;
}
print "\n\n[DONE] FINISHED EXECUTING JOBS\n";
Output Messages:
sh: loadproject: command not found
Can you show us what you have tried so far? How are you running this program?
My first suspicion wouldn't be the environment if you are running it from a login shell. any Perl script you start (well, any program, really) inherits the same environment. However, if you are running the program through cron, then that's a different story.
The other mistakes I usually make in these situations is specifying the relative paths incorrectly. The paths are fine from the command line, but my Perl script has some other current working directory.
For general advice, see Interact with the system when Perl isn't enough. There's also a chapter in Learning Perl about this.
That's about the best advice you can hope for given the very limited information you've shared with us.
I've just recently completed my first nightly build script (first significant anything script, really) in powershell. I seem to have things working well, if not yet robustly (I haven't handled significant error-checking yet), but I found myself falling into an idiom around the Invoke-Expression cmdlet, and I'm wondering if I'm using it properly.
Specifically, I use a series of variables to build up command-lines that I will use to build the solution, then run the solution's unit tests. e.g., something like:
$tmpDir = "C:\Users\<myuser>\Development\Autobuild"
$solutionPath=$tmpDir+"\MyProj\MyProj.sln"
$devenv="C:\Program Files (x86)\Microsoft Visual Studio 10.0\common7\ide\devenv"
$releaseProfile="Release"
$releaseCommandLine="`"$devenv`" `"$solutionPath`" /build `"$releaseProfile`""
This works well enough, $releaseCommandLine contains the command line that I want to execute when I'm done. I then execute it via this line:
$output = Invoke-Expression "& $releaseCommandLine"
Is this the proper way to execute a manually-built command line from a powershell script? I thought initially that Invoke-Command would do it, but I must have been doing something wrong because I couldn't get that working at all for half an hour, and I got this working almost immediately.
I've followed this same pattern a few other times in this same script. Is this a best-practice?
Looks fine to me. Only thing I'd change is to use more Powershell features in place of fragile assumptions. E.g.:
use Join-Path instead of string concatenation
use the Env:\ provider to look up the %programfiles(x86)% dir (or better yet, use the HKML:\ provider to find the path - it's in SOFTWARE\Microsoft\VisualStudio\\InstallDir)
when I have to write a string that contains literal doublequotes and variable expansion, I usually fall back to the syntax below. Personal preference, obviously.
'"{0}" "{1}" /build "{2}"' -f $devenv, $solutionPath, $releaseProfile
In some cases I'd be inclined to use Process.Start() so that I could capture the stdout & stderr streams independently (and maybe even control stdin interactively, depending on the application).
PS - the '&' is not strictly necessary.
I think it is unnecessary to use Invoke-Expression here. I've done this with a lot of build scripts and it usually looks like this:
$vsroot = "$env:ProgramFiles(x86)\Microsoft Visual Studio 9.0"
$devenv = "$vsroot\Common7\IDE\devenv.exe"
$sln = Join-Path <source_root> Source\MyProj\MyProj.sln
& $devenv $sln /build Release
or
& $devenv $sln /build "Release|Any CPU"
Although lately, I have had some troubles with using devenv.exe (mis-behaving add-ins, etc), so now I use msbuild.exe:
$msbuild = 'C:\Windows\Microsoft.NET\Framework\v3.5\MSBuild.exe'
& $msbuild $sln /p:Configuration=Release
Currently MSBuild can handle C#, VB and C++ (invokes vcbuild) but it can't handle solutions with setup & deployment projects in them. However, I have found it to be more reliable than using devenv.exe.
BTW you typically need to invoke other tools (sn.exe, signtool.exe, mt.exe, etc) in a build script that are specific to the version of Visual Studio/.NET you want to build against. So it is usually best to configure your environment variables in the same way that the VS 2008 command prompt does. With the PowerShell Community Extensions installed, you can enable one line in the PSCX profile header to enable this for .NET 3.5/VS 2008 settings:
$Pscx:Preferences["ImportVisualStudioVars"] = $true
I have a perl script (part of the XMLTV family of "grabbers", specifically tv_grab_oztivo).
I can successfully run it like this:
/sw/bin/perl /path/to/tv_grab_oztivo --output /path/to/tv.xml
I use the full paths to everything to eliminate issues with the Working Directory. Permissions shouldn't be a problem.
So, if I run it from the Terminal (Mac OSX) it works just fine.
But when I set it to run via a cron job, nothing appears to happen at all. No output is created etc.
There isn't anything wrong with the crontab as far as I can see, because if I substitute a helloworld.pl for the actual script, it runs just fine at the right time.
So, what can I do to debug? I can see from looking at %ENV in the two cases that the environment is very different, but what other approaches can I take to debugging? How can I see the output of the cron job, which might be some kind of perl "die" message or "not found" message from the shell or whatever?
Or should I be trying to somehow give the cron version of the command the same environment as when it's running as me?
It's often because you don't get the full environment when running under cron. Best bet is to capture the ouput by using the command:
( /sw/bin/perl /path/to/tv_grab_oztivo ... ) >/tmp/qq 2>&1
and then have a look at /tmp/qq.
If it does turn out to be a missing environment, then you may need to put:
. ~/.profile
or something similar, into the execution chain of your cron job, such as:
( . ~/.profile ; /sw/bin/perl /path/to/tv_grab_oztivo ... ) >/tmp/qq 2>&1
If you're looking at %ENV in the two cases, I'd suggest that, as a first step in your perl script, set %ENV to what it is in a cron job, and then trying to run it from the command line. You may need to exec yourself once for this to take full control:
BEGIN {
if (exists $ENV{something_in_your_env_not_in_cron}) {
%ENV = (...);
exec $^X, $0, #ARGV;
}
}
Now try running it, and seeing if there's anything you can do to debug it (including running under perl -d if required). Most likely, you'll find that you end up adding items back into %ENV one at a time until it magically starts working (LD_LIBRARY_PATH is a good one for this, but ORACLE_HOME or DB2HOME for Oracle or DB2 apps might be good choices, too). Then you can either set the variable in your script, or in the crontab.
I'd run a simple shell script by absolute path from the cron command.
Inside that script, I'd ensure that I trapped stdout and stderr to a known (or knowable) file. I'd also ensure that enough of your environment is set. On Unix, you get almost no environment set at all when you run a command via cron - I'm not sure about MacOS X. The standard culprit for problems is PATH. I have a separate .cronfile that sets my working environment enough that I usually don't have problems - that's an analogue of .profile.
On occasion if you can't figure out what's going wrong with your command line, the simplest way to fix it is to turn the whole thing into a shell script. Ideally you shouldn't have to do this, but it can be the fastest way to solve the problem.
File: /files/cron1.sh
#!/bin/sh
/sw/bin/perl /path/to/tv_grab_oztivo --output /path/to/tv.xml
And then in cron:
/files/cron1.sh
This allows you to test the script independent of cron. Remember though that your login shell runs with different environment variables than cron does.
cron usually captures the output of stdout and stderr and e-mailes any output to the crontab owner.
Did you double check your crontab entry to make sure it's valid and will execute at the right time?
Make sure that the script does not need any environment variables set. Otherwise wrap it in another (bash) script, where you can set the environment variables that the other script expects.