I'm writing some functions in a Jupyter notebook to prepare data, but they can take quite a while to run.
In [*]: print "hello"
df['expensive_col'] = df.apply(myfunc, axis=1)
It would be useful if myfunc would print status information such that I see how far into the execution it is. The downside of this is that it would seriously clutter the Jupyter output cell.
In [*]: print hello
df['expensive_col'] = df.apply(myfunc, axis=1)
hello
ix0
ix1
ix2
...
ix999
...
noisy output
...
Is there a way to print status updates from the long-running Jupyter cell in such a way to keep from cluttering the output cell?
For example, I could wipe everything printed by calling IPython.display.clear_output() but I might want to do a more targeted wipe, keeping stuff from before my expensive (hello, in the above example).
One simple thing you could do is to invoke the print function with the end parameter set to carriage return instead of newline, like this:
for i in range(100):
print('progress %i' % i, end='\r')
# things happen here
It effectively writes over the previous line, without cluttering the output field.
For a more advanced approach (and fancy looking progress bars!) check out the tqdm project.
I am trying to implement shell commands in knitr and display the output in the knitted pdf document as shown here:
```{r shell commands, engine="sh"}
wc -wlmc en_US.blogs.txt
```
I am not sure whether this is even being evaluated, as there is no output.
just realized that I can call this with system(), which will print to device! Therefore,
system("wc -l en_US.blogs.txt")
will print to the display.
Use intern=TRUE to return result of system() as character vector, then cat with past and collapse. Example.
x <- system("tree", intern=TRUE)
cat(paste(x, collapse="\n"))
places tree of working directory in output document using knitr.
I am using Matlab to print a small text file (temp_script.exec) that will be used to run GrADS commands. The script looks like the following:
'reinit'
'open temp_ctl.ctl'
'set lon -100 -80'
'set lat 20 30'
'define prc = var'
'set sdfwrite data_out.nc'
'sdfwrite prc'
The script is called via cshell:
#!/bin/csh -f
grads -lbc << EOF
temp_script.exec
EOF
exit
The script seems to execute properly, but the output (data_out.nc) is not generated. Strangely, if I edit it using VI and replace the first character -- the single quotation before the command "reinit" -- by typing another single quotation, then re-run the script, data is generated properly.
My question is, what could be different? The scripts look identical in several different text editors, but the "modified" script (by typing) is 1 byte larger. I am using the "fprintf" function to generate the single quotes in Matlab. Could it be some problem with that function?
Thanks for reading.
To see if the files are really the same (the generated one and the one edited with vi):
od -c -t x1 temp_script.exec > temp_script.lis
od -c -t x1 vi_script.exec > vi_script.lis
diff exec_script.lis vi_script.lis
There could be a UNICODE BOM at the beginning of the file, or a missing newline at the end of file that is causing your issue.
I am trying to use the system() function to pipe the output of the gdalinfo (version 1.10 x64) utility directly to Matlab/Octave. The function consistently returns status=0, but does not return any output. For example:
[status output] = system('"C:\Program Files\GDAL\gdalinfo.exe" "E:\DATA\image.tif"')
will only return:
status =
0
output =
''
Any idea why no output is returned?
It appears there is something strange about `gdalinfo.exe'. Several people have reported difficulty piping the output of the program to a textfile - see for example http://osgeo-org.1560.x6.nabble.com/GDALINFO-cannot-pipe-to-text-file-td3747928.html
So the first test would be - can you do something like this:
"C:\Program Files\GDAL\gdalinfo.exe" "E:\DATA\image.tif" > myFile.txt
and see whether the file is created and has any content? If it doesn't, it may be that the program is using a different way to produce output (for example, using stderr instead of stdout). If it is possible to get data into a text file but not directly to matlab, I suppose a workaround would be to write to file, then read that file in separately:
tempFile = tempname; % handy built in function to create temporary file name
execCmd = '"C:\Program Files\GDAL\gdalinfo.exe ';
targetFile = '"E:\DATA\image.tif"';
status = system([execCmd targetFile ' > ' tempFile]);
output = textread( tempFile, '%s' );
system(['del ' tempFile);
Now the output variable will be a cell array with one cell per line in the input file.
This works on my Windows machine if I am in the Octave directory:
[status output] = system('ls bin')
I was having the same issue trying to pipe the output from within C#. It turns out that the ECW plugin breaks the capability (I don't know how). If this plugin is not crucial to you, go into the plugins directory and delete gdal_ECWJP2ECW.dll. You should be able to use '>' and other stuff to dump your output to a file.
GCC version 4.6
The Problem: To find a way to feed in parameters to the executable, say a.out, from the command line - more specifically feed in an array of double precision numbers.
Attempt: Using the READ(*,*) command, which is older in the standard:
Program test.f -
PROGRAM MAIN
REAL(8) :: A,B
READ(*,*) A,B
PRINT*, A+B, COMMAND_ARGUMENT_COUNT()
END PROGRAM MAIN
The execution -
$ gfortran test.f
$ ./a.out 3.D0 1.D0
This did not work. On a bit of soul-searching, found that
$./a.out
3.d0,1.d0
4.0000000000000000 0
does work, but the second line is an input prompt, and the objective of getting this done in one-line is not achieved. Also the COMMAND_ARGUMENT_COUNT() shows that the numbers fed into the input prompt don't really count as 'command line arguments', unlike PERL.
If you want to get the arguments fed to your program on the command line, use the (since Fortran 2003) standard intrinsic subroutine GET_COMMAND_ARGUMENT. Something like this might work
PROGRAM MAIN
REAL(8) :: A,B
integer :: num_args, ix
character(len=12), dimension(:), allocatable :: args
num_args = command_argument_count()
allocate(args(num_args)) ! I've omitted checking the return status of the allocation
do ix = 1, num_args
call get_command_argument(ix,args(ix))
! now parse the argument as you wish
end do
PRINT*, A+B, COMMAND_ARGUMENT_COUNT()
END PROGRAM MAIN
Note:
The second argument to the subroutine get_command_argument is a character variable which you'll have to parse to turn into a real (or whatever). Note also that I've allowed only 12 characters in each element of the args array, you may want to fiddle around with that.
As you've already figured out read isn't used for reading command line arguments in Fortran programs.
Since you want to read an array of real numbers, you might be better off using the approach you've already figured out, that is reading them from the terminal after the program has started, it's up to you.
The easiest way is to use a library. There is FLAP or f90getopt available. Both are open source and licensed under free licenses.
The latter is written by Mark Gates and me, just one module and can be learned in minutes but contains all what is needed to parse GNU- and POSIX-like command-line options. The first is more sophisticated and can be used even in closed-source projects. Check them out.
Furthermore libraries at https://fortranwiki.org/fortran/show/Command-line+arguments
What READ (*,*) does is that it reads from the standard input. For example, the characters entered using the keyboard.
As the question shows COMMAND_ARGUMENT_COUNT() can be used to get the number of the command line arguments.
The accepted answer by High Performance Mark show how to retrieve the individual command line arguments separated by blanks as individual character strings using GET_COMMAND_ARGUMENT(). One can also get the whole command line using GET_COMMAND(). One then has to somehow parse that character-based information into the data in your program.
I very simple cases you just need the program requires, for example, two numbers, so you read one number from arg 1 and another form arg 2. That is simple. Or you can read a triplet of numbers from a single argument if they are comma-separated like 1,2,3 using a simple read(arg,*) nums(1:3).
For general complicated command line parsing one uses libraries such as those mentioned in the answer by Hani. You have set them up so that the library knows the expected syntax of the command line arguments and the data it should fill with the values.
There is a middle ground, that is still relatively simple, but one already have multiple arguments, that correspond to Fortran variables in the program, that may or may not be present. In that case one can use the namelist for the syntax and for the parsing.
Here is an example, the man point is the namelist /cmd/ name, point, flag:
implicit none
real :: point(3)
logical :: flag
character(256) :: name
character(1024) :: command_line
call read_command_line
call parse_command_line
print *, point
print *, "'",trim(name),"'"
print *, flag
contains
subroutine read_command_line
integer :: exenamelength
integer :: io, io2
command_line = ""
call get_command(command = command_line,status = io)
if (io==0) then
call get_command_argument(0,length = exenamelength,status = io2)
if (io2==0) then
command_line = "&cmd "//adjustl(trim(command_line(exenamelength+1:)))//" /"
else
command_line = "&cmd "//adjustl(trim(command_line))//" /"
end if
else
write(*,*) io,"Error getting command line."
end if
end subroutine
subroutine parse_command_line
character(256) :: msg
namelist /cmd/ name, point, flag
integer :: io
if (len_trim(command_line)>0) then
msg = ''
read(command_line,nml = cmd,iostat = io,iomsg = msg)
if (io/=0) then
error stop "Error parsing the command line or cmd.conf " // msg
end if
end if
end subroutine
end
Usage in bash:
> ./command flag=T name=\"data.txt\" point=1.0,2.0,3.0
1.00000000 2.00000000 3.00000000
'data.txt'
T
or
> ./command flag=T name='"data.txt"' point=1.0,2.0,3.0
1.00000000 2.00000000 3.00000000
'data.txt'
T
Escaping the quotes for the string is unfortunately necessary, because bash eats the first quotes.