I have some code in python that works but is unfortunately super slow. Someone at #python suggested that I can run the code through a profiler to see the lines and functions in which the code was taking the most amount of time.
The python source code that I want to profile reads from STDIN. But since the input is large I compiled the input as a file so that I can simply redirect it to the python code at the shell. So at the shell, I issue the command..
cat input | python pythonsource.py
The problem is, when I try to run the profiler in ipython I can't seem to find a way to redirect the input to the python code. At the ipython shell, I tried,
run -p -l 1.0 pythonsource.py input (didn't work. simply waits at STDIN for input)
run -p -l 1.0 pythonsource.py << input (didn't work)
run -p -l 1.0 cat input | python pythonsource.py (didn't work.)
I'm not sure how to do I can make the ipython profiler command redirect the input to STDIN for the pythonsource to read from. Could someone please tell me how to fix this? Or have I got it totally wrong? Maybe there is some other cleaner, more smarter way of profiling python code?
And maybe what I ask next should be a part of another question..but I was wondering what does ipython mean when it refers to "primitive calls" in some of the output of the ipython profiler?
Thank you.
The right way to redirect input on the shell commandline is as follows:
cat input | run -p -l 1.0 python pythonsource.py
run -p -l 1.0 python pythonsource.py < input
The syntax << string creates a "here document", it does not redirect input.
From the python prompt (and hopefully in ipython), you can redirect standard input like this:
import sys
save_stdin = sys.stdin
sys.stdin = open('input')
run -p <etc.>
sys.stdin = save_stdin # Restore the real stdin
Or (recommended) you could rewrite your source to work with a filename: If you call it as follows, "input" will be in sys.argv[1] and you can open it and read from it:
python pythonsource.py input
Related
I'm looking for a way to pass the path of a USB/serial device to my ipython profile startup file on OS X. Because the device file can have an awkward name it would be nice if I could leverage bash completion. Something like:
$ ipython --profile test /dev/tty.usb[TAB]
I could then make this in to an alias `ipython-test /dev/tty.usb[TAB]'.
I'd like to do this to allow my startup script to establish a connection with some hardware.
Other ideas welcome of course.
You can do that by evaluating a bash command there which will give you this autocompletion effect. E.g. with tab you'd like to get the first matching entry, here's a code which will do that for you:
ipython --profile test `ls /dev/tty | head -n1`
For understanding how that works see Understanding backticks in bash.
I'm trying to use named pipes in a project. I have two terminals open, Terminal A and Terminal B.
In terminal A, I issued this command:
mkfifo myFifo && tail -f myFifo | csh -s
It seems as if standard out is being redirected somewhere else, though, because my prompt disappears and some commands aren't reflected in terminal A.
For example, if in terminal B I begin a python session via issuing echo "python" > myFifo, then echo "print 'Hello, World'" > myFifo, I don't see Hello, World in terminal A.
However, if I issue echo ls > myFifo within terminal B, I see the correct output from ls in terminal A.
Does anyone know why sometimes the output appears and sometime it doesn't?
I'm running on CentOS 6.6
Thanks,
erip
You read from the FIFO with csh, if you start an interactive Python shell in csh, then it won't be reading from the FIFO because it's busy running python.
Python doesn't somehow automagically do a REPL on the FIFO. How should it even know about the FIFO? It has no knowledge of it.
You could, perhaps, tell Python to read commands from the FIFO with something like:
>>> import os, sys, time
>>> fifo = open(os.open('myFifo', os.O_NONBLOCK), 'r')
And then:
$ echo 'print(42+5)' > ! myFifo
Will give you:
>>> eval(fifo.read())
47
Perhaps there's also a way to tell Python to read commands from myFifo by overwriting sys.stdin, but I can't get that working in my testing.
It's a bit unclear to me what exactly you're trying to achieve here, though. I suspect there might be another solution which is much more appropriate to the problem you're having.
I'm running a script in solaris 11 with different results depending of the shell used.
The script has an echo redirecting to a file given by an environment value:
echo "STARTING EXEC" >> $FILE
ps. EXEC is just the message the script show, it's not using exec command.
If I execute the script without the variable FILE defined (using /usr/bin/ksh):
./start.sh[10]: : cannot open
and the script continue the execution.
The flags for ksh are:
echo $-
imsuBGEl
But if I change to /usr/xpg4/bin/sh, the script show me the echo in stdout and there is no error shown.
The flags for xpg4 sh are:
echo $-
imsu
I tried to change the flags with set +- (I can't remove El flags, but BG are removed ok), but can't get the same behavior.
Is there anything I can do to get the same result using ksh without cannot open error?
/usr/bin/ksh --version
version sh (AT&T Research) 93u 2011-02-08
I'll want the script keep going, showing the message in stdout, instead of showing the error just like it does now.
Like shellter said in the comments, the good thing to do is to check if the FILE variable is defined before doing anything. This is a script migration from an HPUX to a SOLARIS environment, and client think they must have the same result as before (we unset FILE variable before execution to test it).
You are likely running Solaris 11, not Solaris 64.
Should you want to have your scripts to work under Solaris 11 without having to search everywhere the bogus redirections, you can simply replace all shebangs (first line) by #!/usr/xpg4/bin/sh.
The final solution we are going to take is to install the ksh88 package and use it like default shell (/usr/sunos/bin/ksh). This shell have the same behavior the client had before, and we can let the scripts with no modifications.
The ksh used in solaris 11 is the 93 (http://docs.oracle.com/cd/E23824_01/html/E24456/userenv-1.html#shell-1)
Thanks #jlliagre and #shellter for your help.
In python, I can run a script and enter interactive mode in the context of that script. This lets me mess with global variables and what not to examine program state.
$ python -i hello.py
Can I do this with Coffeescript? I've tried the following:
$ coffee -i hello.coffee
doesn't load hello.coffee. It's equivalent to coffee -i
$ cat hello.coffee | coffee -i
runs the script line by line in REPL but ends REPL after the EOF.
I've recently started a project to create an advanced interactive shell for Node and associated languages like CoffeeScript. One of the features is loading a file or string in the context of the interpreter at startup which takes into account the loaded language.
http://danielgtaylor.github.com/nesh/
Example:
# Load a string
nesh -c -e 'hello = (name) -> "Hello, #{name}"'
# Load a file
nesh -c -e hello.coffee
Then in the interpreter you can access the hello function. Also a good idea to create an alias in bash:
alias cs='nesh -c'
cat foo.coffee - | coffee -i
tells cat to first output your code and then output stdin, which gives you what you're looking for I think.
I am confronted with this problem as well. The one provide by #int3 doesn't solve this problem, for CoffeeScript is one indentation based language. stdin will pass the code line by line, but the repl is not smart enough to realize this. Since you post this question, I suggest you create one issue (feature request) on CoffeeScript
In Unix, I have a process that I want to run using nohup. However this process will at some point wait at a prompt where I have to enter yes or no for it to continue. So far, in Unix I have been doing the following
nohup myprocess <<EOF
y
EOF
So I start the process 'myprocess' using nohup and pipe in a file with 'y' then close the file. The lines above are effectively three seperate commands - i.e. I hit enter on the first line in UNIX, then I get a prompt where I enter 'y' and then press enter to then finally type 'EOF' and hit return again.
I want to know execute this in Perl but I am not sure how I can execute this command as it is over three lines. I don't know if the following will work....
my $startprocess = `nohup myprocess <<EOF &
y
EOF
`
Please help - thank you!
I think your proposal will work as is. If not, try replacing the redirect with a pipe:
my $startprocess = `(echo "y" | nohup myprocess) &`;
Also, depending on WHY you are doing a nohup, please look at the following pure Perl daemonizing approach using Proc::Daemon : How can I run a Perl script as a system daemon in linux?
Expect for interactive programs can be used as well.