Killing subprocess after first line - subprocess

I am executing program which connects to external server from python.
If user is not authenticated, the program asks for username and password.
Here is how subprogram output looks:
Authentication Required
Enter authorization information for "Web API"
<username_prompt_here>
<password_prompt_here>
I want to kill subprocess right after 'Authentication Required' is printed, but the problem is, that my code works wrong - subprocess is asking for credentials and after user provides it, the subprocess is killed.
Here is my code:
with subprocess.Popen(self.command, stdout=subprocess.PIPE, shell=True, bufsize=1, universal_newlines=True) as process:
for line in process.stdout:
if 'Authentication Required' in line:
print('No authentication')
process.kill()
print(line)
What am I doing wrong?

What am I doing wrong?
Your code is ok (if you want to kill the subprocess after 'Authentication Required' line regardless its position) if the child process flushes its stdout buffer in time. See Python: read streaming input from subprocess.communicate()
The observed behavior indicates that the child uses a block-buffering mode and therefore your parent script sees the 'Authentication Required' line too late or that killing the shell with process.kill() doesn't kill its descendants (processes created by the command).
To workaround it:
See whether you could pass a command-line argument such as --line-buffered (accepted by grep), to force a line-buffered mode
Or see whether stdbuf, unbuffer, script utilities work in your case
Or provide a pseudo-tty to hoodwink the process into thinking that it runs in a terminal directly — it may also force the line-buffered mode.
See code examples in:
Python subprocess readlines() hangs
Python C program subprocess hangs at "for line in iter"
Last unbuffered line can't be read
And - not always I want to kill program after first line. Only if first line is 'Authentication required'
Assuming the block-buffering issue is fixed, to kill the child process if the first line contains Authentication Required:
with Popen(shlex.split(command),
stdout=PIPE, bufsize=1, universal_newlines=True) as process:
first_line = next(process.stdout)
if 'Authentication Required' in first_line:
process.kill()
else: # whatever
print(first_line, end='')
for line in process.stdout:
print(line, end='')
If shell=True is necessary in your case then see How to terminate a python subprocess launched with shell=True.

Related

Python Windows how to get STDOUT data in real time?

I have a windows executable that I want to run over and over. The problem is that sometimes there's an error about 1 second in, but the program doesn't exit. So what I would like to do is to be able to grab the contents of stdout, recognize there is an error, and then kill the subprocess and start it over.
When I run this executable, stuff prints to the screen just fine. But when I wrap it in a subprocess from python then the stdout stuff doesn't show up until the program terminates.
I've tried basically everything posted here with no luck:
Constantly print Subprocess output while process is running
Here's my current code, I replaced the executable with a second python program just to remove any other weird variables:
parent_program.py:
import subprocess, os, sys
program = "python "+os.path.dirname(os.path.abspath(__file__)) + "/child_program.py"
with subprocess.Popen(program, shell=True, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True) as p:
for line in p.stdout:
print(line, end='')
child_program.py:
from time import sleep
for i in range(0,10):
print(i)
sleep(1)
What I would expect is that I would see 1,2,3,4... printed one second at a time, as if I had just run python child_program.py, but instead I get nothing for 10 seconds and then get all the output at once.
I also thought about trying to run the program from the CMD prompt and piping the stdout to a file python child_program.py 2>&1 > output.txt and then having python read that file, but it's the same problem, the file doesn't get written until the program terminates.
Is there any way to fix this on windows?

problems exiting from Python using iPython/Spyder

This question has been asked before, but I have tried the solutions in related questions such as this to no avail.
I am having problems with Python's exit command, and I have ruled out a problem with my code as run by vanilla Python 3. The problem comes when I run it with iPython or in Spyder's iPython console.
When I use just a simple exit command, I get the error:
NameError: name 'exit' is not defined
I have already imported sys as suggested by the other link. The only thing that kind of works is to try sys.exit() in which case I get:
An exception has occurred, use %tb to see the full traceback.
SystemExit
C:\Users\sdewey\AppData\Local\Continuum\Anaconda3\lib\site-
packages\IPython\core\interactiveshell.py:2870: UserWarning: To exit: use
'exit', 'quit', or Ctrl-D.
warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)
I only say that that "kind of works" because the error message is smaller so it's less annoying :).
Any ideas? Seems like an issue with iPython. I have encountered a different issue in Jupyter (which uses iPython) where quit was ignored entirely, which I posted about separately here
I've run into the same issue while running scripts containing exit() in Pycharm's IPython shell.
I learned here, that exit is intended for interactive shells, so behaviour will vary depending on how the shell implements it.
I could figure out a solution which would...
not kill the kernel on exit
not display a traceback
not force you to entrench code with try/excepts
work with or without IPython, without changes in code
Just import 'exit' from the code beneath into scripts you also intend to run with IPython and calling 'exit()' should work. You can use it in jupyter as well (instead of quit, which is just another name for exit), where it doesn't exit quite as silent as in the IPython shell, by letting you know that...
An exception has occurred, use %tb to see the full traceback.
IpyExit
"""
# ipython_exit.py
Allows exit() to work if script is invoked with IPython without
raising NameError Exception. Keeps kernel alive.
Use: import variable 'exit' in target script with 'from ipython_exit import exit'
"""
import sys
from io import StringIO
from IPython import get_ipython
class IpyExit(SystemExit):
"""Exit Exception for IPython.
Exception temporarily redirects stderr to buffer.
"""
def __init__(self):
# print("exiting") # optionally print some message to stdout, too
# ... or do other stuff before exit
sys.stderr = StringIO()
def __del__(self):
sys.stderr.close()
sys.stderr = sys.__stderr__ # restore from backup
def ipy_exit():
raise IpyExit
if get_ipython(): # ...run with IPython
exit = ipy_exit # rebind to custom exit
else:
exit = exit # just make exit importable
You can use system warnings to set those warning that you do not need as ignored. Example:
the function that you call from somewhere else:
def my_function:
statements
if (something happened that you want to exit):
import warnings
warnings.filterwarnings("ignore")
sys.exit('exiting...')

Ipython script stop on shell error

Does ipython have a setting similar to '-e' in bash that stops the execution of the script if any ipython's shell command returns a non-zero value?
Not directly, no. However, after a !foo shell command, the exit code is stored as the value _exit_code. So your script could check that and throw an error.
For some reason, on my system it multiplies all the exit codes by 256. I'm not at all sure why it's doing that.

How can I detect that a command completed its output in a tty?

I'm studying the code of Mobile Terminal which is a command line for iPhone.
The projects emulates a VT100 terminal.
I can monitor everything that goes through the terminal (ascii and control characters)
but I can't figure out how the terminal knows that a command completed its output. How
does the terminal know when to display the prompt again ? Is there a special control
character that every command sends when ending ?
To me it sounds like you're running a shell in the terminal, because a VT100 doesn't show a prompt (AFAIK).
A shell creates a child process and executes the command there. The shell then simply waits until this child process is finished and then prints its prompt again.
An exception is when the command is run in the background (some_command &), the shell doesn't wait for the child to exit and immediately prints the prompt again.

How to collect the output of man command in tcl?

Can somebody suggest me, how to collect output of man command in tcl?
I am writing :-
set hello [ man {command-name}]
and when am executing the script, the program gets halted and
man commands start running in the foreground, prompting the user
to "press RETURN" again and again till it gets completed.
You're just missing the exec command
set output [exec man cmd-name]
When you do set out [man cmd-name] in an interactive tcl session, the unknown command will intercept the 'man' command and implicitly perform an exec on it. In that scenario, 'man' somehow knows you're interactive and pipes the manpage through your $PAGER.