Pylint and Subprocess.Run returning Exit Status 28 - subprocess

I'm trying to run pylint using subprocess but getting a vague message stating non-zero exit status 28. I can not find any reference to an exit status 28 for either Pylint or subprocess.
I can only assume this is a pylint/windows issue as running some other typical commands e.g. head, works just fine as does running pylint directly in the console.
Running python 3.5 on Windows 10.
Any ideas?
MWE
import subprocess
cmd = 'pylint test_script.py'
subprocComplete = subprocess.run(cmd, shell=True, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(subprocComplete.stdout.decode('utf-8'))
Output
python35 pylint_subprocess_test.py
Traceback (most recent call last):
File "pylint_subprocess_test.py", line 3, in <module>
subprocComplete = subprocess.run(cmd, shell=True, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "C:\Python35\lib\subprocess.py", line 708, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'pylint test_script.py' returned non-zero exit status 28

If the subprocess returns nonzero (which pylint often will), then subprocComplete never gets assigned to. You can catch the error, and the error object holds the output you want.
import subprocess
cmd = 'pylint test_script.py'
try:
subprocComplete = subprocess.run(cmd, shell=True, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(subprocComplete.stdout.decode('utf-8'))
except subprocess.CalledProcessError as err:
print(err.output.decode('utf-8'))

Related

Is there a way to fix this error? I'm trying to understand it but don't really know

ERROR: Command errored out with exit status 1:
command: 'C:\Users\panda\anaconda3\envs\dmodel\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\panda\AppData\Local\Temp\pip-req-build-0lhigfqj\bindings/torch\setup.py'"'"'; file='"'"'C:\Users\panda\AppData\Local\Temp\pip-req-build-0lhigfqj\bindings/torch\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\panda\AppData\Local\Temp\pip-pip-egg-info-h3yznucj'
cwd: C:\Users\panda\AppData\Local\Temp\pip-req-build-0lhigfqj\bindings/torch
Complete output (6 lines):
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\panda\AppData\Local\Temp\pip-req-build-0lhigfqj\bindings/torch\setup.py", line 41, in
raise RuntimeError("Could not locate a supported Microsoft Visual C++ installation")
RuntimeError: Could not locate a supported Microsoft Visual C++ installation
Building PyTorch extension for tiny-cuda-nn version 1.6
----------------------------------------
WARNING: Discarding git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

IPython - Hide traceback when SystemExit is raised

I'm surprised no one asked this issue before, but somehow I couldn't find an answer.
When raising SystemExit in a simple my_script.py.
import sys
sys.exit(2)
No traceback is displayed when running python my_script.py, as explained in python doc:
SystemExit: When it is not handled, the Python interpreter exits; no stack traceback is printed.
However when running with ipython, a traceback is printed:
$ ipython my_script.py
---------------------------------------------------------------------------
SystemExit Traceback (most recent call last)
my_script.py in <module>
2 import sys
3
----> 4 sys.exit(2)
SystemExit: 2
This is an issue when using argparse as parser.parse_args() call sys.exit() if parsing fail. So user see an unhelpful SystemExit: 2 rather than the error message printed above the traceback.
A workaround is to use os._exit but this feels very hacky and might not properly cleanup.
Is there a ipython flag which would silence/hide the traceback when SystemExit is raised, like with standard python interpreter ?
Here's a sample session with xmode 'plain':
In [1]: import argparse
In [2]: parser=argparse.ArgumentParser()
In [3]: parser.add_argument('foo')
Out[3]: _StoreAction(option_strings=[], dest='foo', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [4]: import sys; sys.argv
Out[4]:
['/usr/local/bin/ipython3',
'--pylab',
'qt',
'--nosep',
'--term-title',
'--InteractiveShellApp.pylab_import_all=False',
'--TerminalInteractiveShell.xmode=Plain']
In [5]: parser.parse_args()
usage: ipython3 [-h] foo
ipython3: error: unrecognized arguments: --pylab --nosep --term-title --InteractiveShellApp.pylab_import_all=False --TerminalInteractiveShell.xmode=Plain
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py:3426: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)
In [6]: parser.parse_args([])
usage: ipython3 [-h] foo
ipython3: error: the following arguments are required: foo
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py:3426: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)
This shows the error message produced by the parser, followed by the capture and traceback. This allows me to continue interactively.
With plain interpreter, I get the message followed by an exit.
2310:~/mypy$ python3
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import argparse
>>> parser=argparse.ArgumentParser()
>>> parser.add_argument('foo');
_StoreAction(option_strings=[], dest='foo', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args()
usage: [-h] foo
: error: the following arguments are required: foo
2311:~/mypy$
For me the whole purpose of using ipython is to try multiple things, without it quitting on me. Otherwise I'd run the script without the interactive layer.

ERROR: Command errored out with exit status 1: python setup.py egg_info; update setuptools doesn't work

I'm trying to install a private python module with command: "pip install . -i "private url" "
I had this error:
Command "python setup.py egg_info" failed with error code 1 in
C:\Users\hwi_pi\AppData\Local\Temp\pip-req-build-rnoryb6e\
I uninstall Python 38 and reinstall it.
Pip is updated and setup_tools is also updated.
And now I have this error:
D:\L1\onelib\trx-5g>D:\L1\onelib\venv\Scripts\pip.exe install . -i https://artifactory-espoo1..com/artifactory/api/pypi/ulphy-pypi-prod-virtual/simple
Looking in indexes:
https://artifactory-espoo1..com/artifactory/api/pypi/ulphy-pypi-prod-virtual/simple,
https://artifactory-espoo2.***********.com/artifactory/ap
i/pypi/mn-l1-pz-int-lib-local/simple Processing d:\l1\onelib\trx-5g
ERROR: Command errored out with exit status 1:
command: 'd:\l1\onelib\venv\scripts\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] =
'"'"'C:\Users\hwi_pi\AppData\Local\Temp\pip-req-build-_4yae_lz\se
tup.py'"'"';
file='"'"'C:\Users\hwi_pi\AppData\Local\Temp\pip-req-build-_4yae_lz\setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(file);code=f.read().repla ce('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))'
egg_info --egg-base
'C:\Users\hwi_pi\AppData\Local\Temp\pip-pip-egg-info-zvlq1hsf'
cwd: C:\Users\hwi_pi\AppData\Local\Temp\pip-req-build-_4yae_lz
Complete output (15 lines):
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\hwi_pi\AppData\Local\Temp\pip-req-build-_4yae_lz\setup.py",
line 23, in
version=os.environ.get('CI_COMMIT_TAG', f'0.0.0+{get_local_version_label()}'),
File "C:\Users\hwi_pi\AppData\Local\Temp\pip-req-build-_4yae_lz\setup.py",
line 17, in get_local_version_label
commit_hash = subprocess.check_output(COMMAND.split()).decode('utf-8').strip()
File "C:\Users\hwi_pi\AppData\Local\Programs\Python\Python38\lib\subprocess.py",
line 411, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "C:\Users\hwi_pi\AppData\Local\Programs\Python\Python38\lib\subprocess.py",
line 489, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\hwi_pi\AppData\Local\Programs\Python\Python38\lib\subprocess.py",
line 854, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\hwi_pi\AppData\Local\Programs\Python\Python38\lib\subprocess.py",
line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for
full command output.
Where the logs are located ?
I'm on Windows 10.
Python 3.8.5.
I launched this command via Pycharm and via cmd windows and same result.
I tried to run the command in administration mode; and same result.
I have tried every solution available on stackoverflow and nothing solves my error.
I tried to find C:\Users\hwi_pi\AppData\Local\Temp\pip-req-build-_4yae_lz\setup.py to see the file setup.py but foled pip-req-build-_4yae_lz doesn't exist.
I can access to https://artifactory-espoo1.************.com/artifactory/api/pypi/ulphy-pypi-prod-virtual/simple via firefox; but I can't download it in .zip for example
Someone to help please ?

Airflow: Celery task failure

I have airflow up and running but I have an issue where my task is failing in celery.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 52, in execute_command
subprocess.check_call(command, shell=True)
File "/usr/local/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'airflow run airflow_tutorial_v01 print_hello 2017-06-01T15:00:00 --local -sd /usr/local/airflow/dags/hello_world.py' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 375, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 632, in __protected_call__
return self.run(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 55, in execute_command
raise AirflowException('Celery command failed')
airflow.exceptions.AirflowException: Celery command failed
it is a very basic DAG (taken from the hello world tutorial: https://github.com/apache/incubator-airflow/blob/master/airflow/example_dags/tutorial.py).
Also I do not see any logs of my worker, I got this stack strace from the Flower web interface.
If I run manually on the worker node, the airflow run command mentionned in the stack trace it works.
How can I get more information to debug further?
The only log I get when starting `airflow work` is
root#ip-10-0-4-85:~# /usr/local/lib/python3.5/dist-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
""")
[2018-07-25 17:49:43,430] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python3.5/lib2to3/Grammar.txt
[2018-07-25 17:49:43,469] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python3.5/lib2to3/PatternGrammar.txt
[2018-07-25 17:49:43,594] {__init__.py:45} INFO - Using executor CeleryExecutor
Starting flask
[2018-07-25 17:49:43,665] {_internal.py:88} INFO - * Running on http://0.0.0.0:8793/ (Press CTRL+C to quit)
^C
The config I use is the default one with a postgresql and redis backend for celery.
I see the worked online in Flower.
Thanks.
edit: edited for more informations

Running a python script from Swift

I'm trying to run a python script from swift in my cocoa application.The script does run but there are some errors.When I run that file from terminal the script works ok, no errors.
Here is the code that runs the python file :
let process = Process()
process.launchPath = "/usr/bin/python"
process.currentDirectoryPath = "\(NSHomeDirectory())" + "/tmp"
process.arguments = [path.stringByAppendingPathComponent("pacman.py")]
process.launch()
Here is the error that I am getting back when I'm trying to run the python file from my application with Process() :
Traceback (most recent call last):
File "/Users/fflorica/Library/Developer/Xcode/DerivedData/Taylor-acpgvfjjherixnfchbrbrayxxvsm/Build/Products/Debug/Taylor.app/Contents/Frameworks/TaylorFramework.framework/Resources/pacman.py", line 439, in <module>
g = Game()
File "/Users/fflorica/Library/Developer/Xcode/DerivedData/Taylor-acpgvfjjherixnfchbrbrayxxvsm/Build/Products/Debug/Taylor.app/Contents/Frameworks/TaylorFramework.framework/Resources/pacman.py", line 251, in __init__
self._init_curses()
File "/Users/fflorica/Library/Developer/Xcode/DerivedData/Taylor-acpgvfjjherixnfchbrbrayxxvsm/Build/Products/Debug/Taylor.app/Contents/Frameworks/TaylorFramework.framework/Resources/pacman.py", line 280, in _init_curses
curses.cbreak()
error: cbreak() returned ERR
Traceback (most recent call last):
File "/Users/fflorica/Library/Developer/Xcode/DerivedData/Taylor-acpgvfjjherixnfchbrbrayxxvsm/Build/Products/Debug/Taylor.app/Contents/Frameworks/TaylorFramework.framework/Resources/pacman.py", line 450, in <module>
raw_input()
EOFError
What am I doing wrong?
Edit: In the past I used system() function like that:
system("cd " + path)
system("python " + path + "/pacman.py")
Then, I used NSTask and it worked perfectly fine.Now, NSTask is Process and I get those errors when using Process.There is not much modified in the API, but somehow, it doesn't work.
Edit 1 : After some investigation, I think that the problem may be because Process is starting a new process in the background that is why python is trowing that errors, but I am not sure.