Why does a Julia system call process fail to exit? - visual-studio-code

This seems to have cropped up out of nowhere. Julia all of a sudden seems to have stopped returning from system calls. There is no indication of an error, the system command appears to successfully complete, but Julia never goes back to the command prompt. For example:
I run a simple command in the VSCode REPL, it outputs what you would expect it to, but then I have to Ctrl-C to continue. Note the "Evaluating..." and spinning arrows icon.
I don't think this happens outside of VisualStudio Code (running Julia in a terminal). I started running into this when trying to save animation files so calls to ffmpeg (direct or through Plots's anim function) and ImageMagick. The behavior I saw with anim calls looks like what is described here: [BUG] Saving animated gif hangs indefinitely #3664
This is the test script that jams up the VSCode REPL but works fine in a plain terminal-based Julia session:
using Plots
using Printf
tmp_dir = mktempdir()
for ind_png = 0:9
θ = ind_png * 36.
p = quiver( [0], [0], quiver=( [sind(θ)], [cosd(θ)] ),
xlims=[-1.,+1.], ylims=[-1.,+1.], aspect_ratio=:equal )
png( p, #sprintf( "%s/img%04i.png", tmp_dir, ind_png ) )
ind_png += 1
end
run( `convert $(tmp_dir)/img'*'.png $(pwd())/test.gif` )
rm( tmp_dir, recursive=true )
The command string that the script generates works fine when entered into a terminal. After this test script jams up, simple commands like run(`ls`) start to hang too. If I Ctrl-C, I get the following stack trace:
ERROR: InterruptException:
Stacktrace:
[1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
# Base ./task.jl:777
[2] wait()
# Base ./task.jl:837
[3] wait(c::Base.GenericCondition{Base.Threads.SpinLock})
# Base ./condition.jl:123
[4] wait(x::Base.Process)
# Base ./process.jl:627
[5] success
# ./process.jl:489 [inlined]
[6] run(::Cmd; wait::Bool)
# Base ./process.jl:446
[7] run(::Cmd)
# Base ./process.jl:444
[8] top-level scope
# REPL[3]:1
I am running Julia and VSCode on a Mac. Has anyone encountered this or know how to fix it? Please help.

Related

SCP command not working in karate project - it throws command error:cannot run program scp.exe: CreateProcess error=2 [duplicate]

I'm trying to execute bash script using karate. I'm able to execute the script from karate-config.js and also from .feature file. I'm also able to pass the arguments to the script.
The problem is, that if the script fails (exits with something else than 0) the test execution continues and finishes as succesfull.
I found out that when the script echo-es something then i can access it as a result of the script so I could possibly echo the exit value and do assertion on it (in some re-usable feature), but this seems like a workaround rather than a valid clean solution. Is there some clean way of accessing the exit code without echo-ing it? Am I missing on something?
script
#!/bin/bash
#possible solution
#echo 3
exit 3;
karate-config.js
var result = karate.exec('script.sh arg1')
feture file
def result = karate.exec('script.sh arg1')
Great timing. We very recently did some work for CLI testing which I am sure you can use effectively. Here is a thread on Twitter: https://twitter.com/maxandersen/status/1276431309276151814
And we have just released version 0.9.6.RC4 and new we have a new karate.fork() option that returns an instance of Command on which you can call exitCode
Here's an example:
* def proc = karate.fork('script.sh arg1')
* proc.waitSync()
* match proc.exitCode == 0
You can get more ideas here: https://github.com/intuit/karate/issues/1191#issuecomment-650087023
Note that the argument to karate.fork() can take multiple forms. If you are using karate.exec() (which will block until the process completes) the same arguments work.
string - full command line as seen above
string array - e.g. ['script.sh', 'arg1']
json where the keys can be
line - string (OR)
args - string array
env - optional environment properties (as JSON)
redirectErrorStream - boolean, true by default which means Sys.err appears in Sys.out
workingDir - working directory
useShell - default false, auto-prepend cmd /c or sh -c depending on OS
And since karate.fork() is async, you need to call waitSync() if needed as in the example above.
Do provide feedback and we can tweak further if needed.
EDIT: here's a very advanced example that shows how to listen to the process output / log, collect the log, and conditionally exit: fork-listener.feature
Another answer which can be a useful reference: Conditional match based on OS
And here's how to use cURL for advanced HTTP tests ! https://stackoverflow.com/a/73230200/143475
In case you need to do a lot of local file manipulation, you can use the karate.toJavaFile() utility so you can convert a relative path or a "prefixed" path to an absolute path.
* def file = karate.toJavaFile('classpath:some/file.txt')
* def path = file.getPath()

Scala Process Builder

I'm trying to execute a command on Scala, but i'm getting an error
s"git --git-dir ${repository.localLocation.get.path}/.git log --format='%h %at %s' --no-decorate" !!
I'm getting an error exit status:
java.lang.RuntimeException: Nonzero exit value: 128
at scala.sys.package$.error(package.scala:27)
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(ProcessBuilderImpl.scala:134)
But when i'm running this via terminal it works perfectly:
git --git-dir='/var/folders/mk/dc2mnd7x3db1hnqm0vfg6b800000gn/T/XXHMjm7178261334218603127.tmp/.git' log --format='%h %at %s'
If i leave only one % part, it works properly.
Can anyone help me ?
You should probably use the Seq[String] variant, because you have space characters in your args that might be used to wrongly separate the arguments. Try
Seq("git", "--git-dir", s"${repository.localLocation.get.path}/.git",
"log", "--format='%h %at %s'", "--no-decorate").!!
Also note that you will see the single ticks ' in the output. You probably want "--format=%h %at %s".

Using __END__ and DATA in Chef recipes (to run legacy shell scripts)

I'm migrating some shell scripts to Chef recipes. Some of these scripts are fairly involved, so just to make life easier in the short term and to avoid introducing bugs in rewriting everything in Chef/Ruby, I'd like to just run some of them as-is. They're all well-written and idempotent, so honestly there's no rush, but of course, the eventual goal is to rewrite them.
One cool feature of Ruby is its __END__ keyword/method: Lines below __END__ will not be executed. Those lines will be available via the special filehandle DATA.
It would be cool to ship the shell scripts as-is inside the the recipe after __END__, maybe something like the following, which I placed in chef-repo/cookbooks/ruby-data-test/recipes/default.rb:
file = Tempfile.new(File.basename(__FILE__))
file << DATA.read
bash file.path
file.unlink
__END__
echo "Hello, world"
However when I run this (with chef-solo -c solo.rb --override-runlist 'recipe[ruby-data-test]'), I get the following error:
[2014-10-03T17:14:56+00:00] ERROR: uninitialized constant Chef::Recipe::DATA
I'm pretty new to Chef, but I'm guessing the above is something about Chef wrapping my recipe in a class, and there's something simple preventing me from accessing DATA. Since it's "global" (?) I tried putting a dollar sign ($DATA) in front of it but that failed with:
NoMethodError
-------------
undefined method `read' for nil:NilClass
So the question is: How do I access DATA in my Chef recipe? Thanks!
It appears you don't have access to DATA, but you can fake it by reading in the current file yourself and splitting on __END__, like Sinatra does.
I ended up making a Chef LWRP for reuse. I don't know if I'll actually end up using this, but I wanted to figure it out. Like I said, I'm a Chef/Ruby noob, so any better ideas or suggestions welcome!
ruby_data_test/recipes/default.rb:
ruby_data_test_execute_ruby_data __FILE__
__END__
#!/bin/bash
set -o errexit
date
echo "Hello, world"
ruby_data_test/resources/execute_ruby_data.rb:
actions :execute_ruby_data
default_action :execute_ruby_data
attribute :source, :name_attribute => true, :required => true
attribute :args, :kind_of => Array
attribute :ignore_errors, :kind_of => [TrueClass, FalseClass], :default => false
ruby_data_test/providers/execute_ruby_data.rb:
def whyrun_supported?
true
end
use_inline_resources
action :execute_ruby_data do
converge_by("Executing #{#new_resource}") do
Chef::Log.info("Executing #{#new_resource}")
file_who_called_me = #new_resource.source
io = ::IO.respond_to?(:binread) ? ::IO.binread(file_who_called_me) : ::IO.read(file_who_called_me)
app, data = io.gsub("\r\n", "\n").split(/^__END__$/, 2)
data.lstrip!
file = Tempfile.new('execute_ruby_data')
file << data
file.chmod(0755)
file.close
exit_status = ::Open3.popen2e(file.path, *#new_resource.args) do |stdin, stdout_and_stderr, wait_thr|
stdout_and_stderr.each { |line| puts line }
wait_thr.value # exit status
end
if exit_status != 0 && !#new_resource.ignore_errors
throw RuntimeError
end
end
end
Here's the output:
$ chef-solo -c solo.rb --override-runlist 'recipe[ruby_data_test]'
Starting Chef Client, version 11.12.4
[2014-10-03T21:50:29+00:00] WARN: Run List override has been provided.
[2014-10-03T21:50:29+00:00] WARN: Original Run List: []
[2014-10-03T21:50:29+00:00] WARN: Overridden Run List: [recipe[ruby_data_test]]
Compiling Cookbooks...
Converging 1 resources
Recipe: ruby_data_test::default
* ruby_data_test_execute_ruby_data[/root/chef/chef-repo/cookbooks/ruby_data_test/recipes/default.rb] action execute_ruby_dataFri Oct 3 21:50:29 UTC 2014
Hello, world
- Executing ruby_data_test_execute_ruby_data[/root/chef/chef-repo/cookbooks/ruby_data_test/recipes/default.rb]
Running handlers:
Running handlers complete
Chef Client finished, 1/1 resources updated in 1.387608 seconds

Return code of scheduled task prefixed with 0x8007000 in list view, registered as 0 in the event log

I am currently trying to setup monitoring of windows scheduled tasks in Zabbix. It seemed easy enough to just monitor the Microsoft-Windows-TaskScheduler/Operational event log filtered by 201 events and regexing on the return code, but when I started simulating errors to test the monitoring, nothing happened.
It turns out that all our windows 2012 servers always log "return code 0" in the event log, even though it actually, sort of, displays it correctly in the Task Scheduler list view. When I say "sort of", it's because the "Last Run Result" actually displays 0x80070001 if the exit code of the program run by the scheduled task is 1.
I have spend a lot of time tweaking the settings, like user account, Run only when user is logged on, Run whether user is logged on or not, setting path on the action, Run with highest privileges, Configure for Vista/7/2012, etc. Nothing helped.
Finally I did some testing on my local machine, Windows 7, and a 2008R2 server, both of which just worked as expected.
The specific task I was testing ran a PowerShell script, using -Command so that it properly propagates the exit, but to rule out any PS issues I also tested with a batch file containing "exit 1" and finally with a small C# console program, that just returns whatever you supply on the command line.
PS, batch and console program all work fine on 7 and 2008, but they all fail in the same manner on 2012.
I've google this to death, but keep coming up short. Apparently 0x80070005 and other similar error codes are have some meaning, but that's not what happens in my case. In my case it seems that my exit code is bitwise or'ed with 0x80070000.
I should note that in all the cases, even 2012, the program started by the task, actually executes and run to the end, it's just the exit code which is handled weirdly.
Following is the output from the test runs:
From Powershell (my shell writes :( if $LASTEXITCODE > 0 ):
54 :( .\ExitCodeTest.exe 1
55 :( $LASTEXITCODE
1
56 :) .\ExitCodeTest.exe 10
57 :( $LASTEXITCODE
10
Windows Server 2008 R2 Standard:
Last Run Result (from list view): 0xA
Event 201 from event log Microsoft-Windows-TaskScheduler/Operational:
Task Scheduler successfully completed task "\ErrorTest" ,
instance "{b67a26cf-7fd8-461a-93d9-a5e48e72e558}" ,
action "D:\Tasks\ExitCodeTest.exe" with return code 10.
Windows Server 2012 Datacenter (notice that the return code in the event log is 0):
Last Run Result (from list view): 0x8007000A
Event 201 from event log Microsoft-Windows-TaskScheduler/Operational:
Task Scheduler successfully completed task "\error test" ,
instance "{2bde46b8-2858-4772-a7ec-d66b29d893a6}" ,
action "D:\Tasks\ExitCodeTest.exe" with return code 0.
Source for ExitCodeTest.exe:
static void Main( string[] args )
{
int exitCode = 0;
if ( args.Length > 0 )
{
exitCode = Convert.ToInt32( args[0] );
}
Environment.Exit( exitCode );
}
Please help, I am at my wits end.
Thanks,
John
(this is NOT an answer, but StackOverflow is refusing to let me add comments - when I click 'add comment', browser scrolls to top of page :-/)
You may be misinterpreting the Last Run Result column. According to Wikipedia (http://en.wikipedia.org/wiki/Windows_Task_Scheduler), LRR values of 0, 1 and 10 are common. Ignore the 0x8007 prefix - this just indicates a WIN32 error code transformed into an HRESULT (http://msdn.microsoft.com/en-us/library/gg567305.aspx).
Try running the test and forcing an exit code of something other than 1 or 10 to see if this influences LRR.
This does not explain of course why action return code is 0 in 2012. Error code 10 is defined as 'environment is incorrect'. Could it be that 2012 server does not want to run 32bit executable?
One other suggestion (and I'm a little out of my depth); according to (http://msdn.microsoft.com/en-us/library/system.environment.exit(v=vs.110).aspx): "Exit requires the caller to have permission to call unmanaged code. The return statement does not.". Might be worth re-compiling ExitCodeTest as follows:
static int Main(string[] args)
{
int exitCode = 0;
if ( args.Length > 0 )
{
exitCode = Convert.ToInt32( args[0] );
}
return exitCode;
}
I'm seeing a similar issue on Server 2012 with a batch file that looks like it succeeds, shows a return value of 0 in event log, but a Last Run Result of 0x80070001.
I see MSFT has a hotfix available for Server 2012 which might address this issue:
http://support.microsoft.com/kb/3003689
I had this problem and fixed it this way.
Instead of calling a batch file move the commands into the actions section of the scheduled task.
I realize this may not work for you as some batch files are long.
I suspect it has to do with circumventing security on a scheduled task -- if you can change the batch file then you could get a scheduled task to run as the identity without windows being the wiser.

Julia, handle keyboard interrupt

Title says it all. How can I handle or catch a SIGINT in julia? From the docs I assumed I just wanted to catch InterruptException using a try/catch block like the following
try
while true
println("go go go")
end
catch ex
println("caught something")
if isa(ex, InterruptException)
println("it was an interrupt")
end
end
But I never enter the catch block when I kill the program with ^C.
edit: The code above works as expected from the julia REPL, just not in a script.
I see the same behavior as alto, namely that SIGINT kills the entire process when my code is run as a script but that it is caught as an error when run in the REPL. My version is quite up to date and looks rather similar to that of tholy:
julia> versioninfo()
Julia Version 0.3.7
Commit cb9bcae* (2015-03-23 21:36 UTC)
Platform Info:
System: Linux (x86_64-linux-gnu)
CPU: Intel(R) Core(TM) i7-3610QM CPU # 2.30GHz
WORD_SIZE: 64
BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Sandybridge)
LAPACK: libopenblas
LIBM: libopenlibm
LLVM: libLLVM-3.3
Digging through the source, I found hints that Julia's interrupt behavior is determined by an jl_exit_on_sigint option, which can be set via a ccall. jl_exit_on_sigint is 0 for the REPL, but it looks as if init.c sets it to 1 when running a Julia program file from the command line.
Adding the appropriate ccall makes alto's code works regardless of the calling environment:
ccall(:jl_exit_on_sigint, Void, (Cint,), 0)
try
while true
println("go go go")
end
catch ex
println("caught something")
if isa(ex, InterruptException)
println("it was an interrupt")
end
end
This does seem to be a bit of a hack. Is there a more elegant way of selecting the interrupt behavior of the environment? The default seems quite sensible, but perhaps there should be a command line option to override it.
For julia >= 1.5, there is Base.exit_on_sigint. From the docs (retrieved on 20220419)
Set exit_on_sigint flag of the julia runtime. If false, Ctrl-C (SIGINT) is capturable as InterruptException in try block. This is the default behavior in REPL, any code run via -e and -E and in Julia script run with -i option.
Works for me. I'm running
julia> versioninfo()
Julia Version 0.3.0-prerelease+695
Commit 47915f3* (2013-12-27 05:27 UTC)
DEBUG build
Platform Info:
System: Linux (x86_64-linux-gnu)
CPU: Intel(R) Core(TM) i7 CPU L 640 # 2.13GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY)
LAPACK: libopenblas
LIBM: libopenlibm
but I expect it's not necessary to be fully up-to-date for this to work (I'd guess 0.2 should be fine too).