Python subprocess.call() not working with >> operator - subprocess

I'm trying to write a program that automatically sets up python shell scripts for me. Heres the code:
#!/usr/bin/env python3
import click
import subprocess
#click.command()
#click.argument('name')
def foo(name):
subprocess.call("cd ~/bin", shell=True)
subprocess.call(["touch", name])
subprocess.call(["echo", "'#!/usr/bin/env python3'", ">>", name])
subprocess.call(["chmod", "+x", name])
if __name__ == '__main__':
foo()
When it runs this is the output I get:
'#!/usr/bin/env python3' >> foo

A better way to approach this problem is to use the built-in Python libraries for creating files so that you can catch the exceptions easier. I've made a simple outline for you, but you should add some error checking.
import click
import os
import stat
#click.command()
#click.argument('name')
def foo(name):
file_to_create = os.path.expanduser("~/bin/") + name
with open(file_to_create, 'w') as file:
file.write("#!/usr/bin/env python3\n")
file_stats = os.stat(file_to_create)
os.chmod(file_to_create, file_stats.st_mode | stat.S_IEXEC) # equivalent of 'chmod +x'
if __name__ == '__main__':
foo()

Related

locust unrecognized arguments when running as lib

The following codes are from Locust examples - use_as_lib.
import gevent
from locust import HttpUser, task
from locust.env import Environment
from locust.stats import stats_printer, stats_history
from locust.log import setup_logging
setup_logging("INFO", None)
class MyUser(HttpUser):
host = "https://docs.locust.io"
#task
def t(self):
self.client.get("/")
env = Environment(user_classes=[MyUser])
runner = env.create_local_runner()
web_ui = env.create_web_ui("127.0.0.1", 8089)
env.events.init.fire(environment=env, runner=runner, web_ui=web_ui)
gevent.spawn(stats_printer(env.stats))
gevent.spawn(stats_history, env.runner)
runner.start(1, spawn_rate=10)
gevent.spawn_later(60, lambda: runner.quit())
runner.greenlet.join()
web_ui.stop()
If I run it with python use_as_lib.py, everything works fine. But if I run it with python use_as_lib.py -c argument01 -b argument02, it will fail with:
use_as_lib.py: error: unrecognized arguments: -c -b argument02
In my case, the snippet above is part of a big program, which has its own command line arguments.
I checked a bit, seems argument_parser.ui_extra_args_dict() here invoked by env.create_web_ui("127.0.0.1", 8089) will parse all the arguments, which cause this issue.
Any ideas on how to fix it ? Thanks!
You can pass a parsed set of parameters when you create the environment, slightly less hack-y than your suggestion. Something like this:
parser = locust.argument_parser.get_parser()
parsed_options = parser.parse_args("-f yourlocustfile.py --headless <other params>")
env = Environment(user_classes=[MyUser], parsed_options=parsed_options=parsed_options)
Would that work?
Here's the workaround I used. So far it works. But more like a hack.
# ...
tmp = sys.argv
sys.argv = [sys.argv[0]]
env.create_web_ui("127.0.0.1", 8089)
sys.argv = tmp
# ...

Run pytest inside a script (if name...)

I have the next file with tests
import pytest
from httpx import AsyncClient
import sys
import config
from main import app
#pytest.mark.asyncio
async def test_register():
async with AsyncClient(app=app, base_url="http://test") as ac:
response = await ac.post("/register", )
assert response.status_code == 200
I want to run it like
if __name__ == '__main__':
pytest.run() # Or something alike
How I can do it? I need it to assign name 'main' to this module, because the main module (which import above) has a constraint like: if __name__ == '__main__', so without it tests will not be run indeed.
pytest.main() would run pytest in the current working directory and this would include your file depending on the filename (e.g. if the filename starts with test_). To run pytest on the current file only you can run:
if __name__ == "__main__":
pytest.main([__file__])

Paging stdout output in IPython

Is it possible in an (interactive) IPython session to pass the stdout output through a pager, like less? If so, how?
For example, in
In [1]: from some_module import function_that_prints_a_lot
In [2]: function_that_prints_a_lot()
... everything scrolls away ...
I would like to page through the stdout output of function_that_prints_a_lot.
Another example:
In [1]: %run script_that_prints_a_lot.py
I've looked through IPython magic commands but didn't find any solution.
As discussed in chat there is no simple way of doing this. Since the function prints the values, the only thing you can do is Capture output + Then page output. There are few issues on jupyter that you might be interested in
https://github.com/jupyter/notebook/issues/2049
https://github.com/ipython/ipython/issues/6516
Capturing output
Output capturing can be done multiple ways
1. Overload Print Method
import sys
data = ""
def myprint(value, *args, sep=' ', end='\n', file=sys.stdout, flush=False):
global data
current_text = value + " ".join(map(str, args)) + "\n"
data += current_text
original_print = print
print = myprint
def testing():
for i in range(1,1000):
print ("i =", i)
testing()
original_print("The output from testing function is", data)
2. Capture output using StringIO
from cStringIO import StringIO
import sys
class Capturing(list):
def __enter__(self):
self._stdout = sys.stdout
sys.stdout = self._stringio = StringIO()
return self
def __exit__(self, *args):
self.extend(self._stringio.getvalue().splitlines())
del self._stringio # free up some memory
sys.stdout = self._stdout
Usage:
with Capturing() as output:
do_something(my_object)
3. Capture output using redirect_stdout
import io
from contextlib import redirect_stdout
f = io.StringIO()
with redirect_stdout(f):
do_something(my_object)
out = f.getvalue()
4. Capture using %%capture magic command
Paging Output
You can use magin %page
%page -r <variablename>
https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-page
Or you can use Ipython code
from IPython.core import page
page.page(variable)
For more details refer to below
PS: Some helpful threads
How to capture stdout output from a Python function call?
How can I redirect print output of a function in python
https://github.com/ipython/ipython/wiki/Cookbook:-Sending-built-in-help-to-the-pager
overload print python
Using bits and pieces from various sources, but essentially it's from IPython's cookbook and Defining custom magics from IPython official documentation
In [1]: from IPython.core.magic import register_line_magic
In [2]: #register_line_magic
...: def my_pager(line):
...: "my line magic"
...: import io
...: from IPython.core import page
...: from contextlib import redirect_stdout
...: f = io.StringIO()
...: with redirect_stdout(f):
...: eval(line)
...: page.pager_page(f.getvalue())
...: del my_pager # don't pollute my namespace
In [3]: def pippo(): print('\n'.join(str(i)for i in range(80)))
In [4]: %my_pager pippo()
This approach has a serious drawback: if the function call that is the argument to %my_pager returns a value, said value is lost (no, %mypager result=print_and_return() won't work...)

environment variables using subprocess.check_output Python

I'm trying to do some basic module setups on my server using Python. Its a bit difficult as I have no access to the internet.
This is my code
import sys
import os
from subprocess import CalledProcessError, STDOUT, check_output
def run_in_path(command, dir_path, env_var=''):
env_var = os.environ["PATH"] = os.environ["PATH"] + env_var
print(env_var)
try:
p = check_output(command, cwd=dir_path, stderr=STDOUT)
except CalledProcessError as e:
sys.stderr.write(e.output.decode("utf-8"))
sys.stderr.flush()
return e.returncode
else:
return 0
def main():
requests_install = run_in_path('python setup.py build',
'D:\installed_software\python modules\kennethreitz-requests-e95e173')
SQL_install = run_in_path('python setup.py install', # install SQL module pypyodbc
'D:\installed_software\python modules\pypyodbc-1.3.3\pypyodbc-1.3.3')
setup_tools = run_in_path('python setup.py install', # install setup tools
'D:\installed_software\python modules\setuptools-17.1.1')
psycopg2 = run_in_path('easy_install psycopg2-2.6.1.win-amd64-py3.3-pg9.4.4-release', # install setup tools
'D:\installed_software\python modules', ';C:\srv_apps\Python33\Scripts\easy_install.exe')
print('setup complete')
if __name__ == "__main__":
sys.exit(main())
now it gets tricky when i start trying to use easy install. It appears my env variables are not being used by my subprocess.check_output call
File "C:\srv_apps\Python33\lib\subprocess.py", line 1110, in _execute_child
raise WindowsError(*e.args)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I don't want to have to upgrade to 3.4 where easy install is installed by default because my other modules are not supported on 3.4. My main challenge is the subprocess.check_call method does not take environment variables as an input and im wary of trying to use Popen() as I have never really got it to work successfully in the past. Any help would be greatly appreciated.
PATH should contain directories e.g., r'C:\Python33\Scripts', not files such as: r'C:\Python33\Scripts\easy_install.exe'
Don't hardcode utf-8 for an arbitrary command, you could enable text mode using universal_newlines parameter (not tested):
#!/usr/bin/env python3
import locale
import sys
from subprocess import CalledProcessError, STDOUT, check_output
def run(command, *, cwd=None, env=None):
try:
ignored = check_output(command, cwd=cwd, env=env,
stderr=STDOUT,
universal_newlines=True)
except CalledProcessError as e:
sys.stderr.write(e.output)
sys.stderr.flush()
return e.returncode
else:
return 0
Example:
import os
path_var = os.pathsep.join(os.environ.get('PATH', os.defpath), some_dir)
env = dict(os.environ, PATH=path_var)
run("some_command", cwd=some_path, env=env)
run("another_command", cwd=another_path, env=env)

Selenium find_element_by_id returns None

I am using selenium and i have properly installed the selenium module on redhat linux 6.
Below is my script:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
import unittest, time, re
import zlib
class Sele1(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Firefox()
self.driver.implicitly_wait(30)
self.base_url = "https://bugzilla.example.com/"
self.verificationErrors = []
def test_sele1(self):
driver = self.driver
driver.get(self.base_url + "/")
driver.find_element_by_id("Bugzilla_login").clear()
driver.find_element_by_id("Bugzilla_login").send_keys("username")
driver.find_element_by_id("Bugzilla_password").clear()
driver.find_element_by_id("Bugzilla_password").send_keys("password")
driver.find_element_by_id("log_in").click()
driver.find_element_by_id("quicksearch").clear()
driver.find_element_by_id("quicksearch").send_keys("new bugs is bugzilla tool")
driver.find_element_by_xpath("//input[#value='Find Bugs']").click()
def is_element_present(self, how, what):
try: self.driver.find_element(by=how, value=what)
except NoSuchElementException, e: return False
return True
def tearDown(self):
self.driver.quit()
self.assertEqual([], self.verificationErrors)
if __name__ == "__main__":
unittest.main()
When i am running this script it is showing errors:
ERROR: test_sele1 (main.Sele1)
Traceback (most recent call last):
File "sele1.py", line 19, in test_sele1
driver.find_element_by_id("Bugzilla_login").clear()
AttributeError: 'NoneType' object has no attribute 'clear'
Ran 1 test in 2.018s
FAILED (errors=1)
Plateform: Redhat
Python Version: 2.6
Note: while running the same script in windows7, it is running fine but not running in linux with python2.6
Please help me for this...
Thanks in advance!
Simple: Selenium can't find an element called "Bugzilla_login".
There could be hundreds of reasons this may be happening. I'd start with checking whether you're loading the correct page.