For some reason the pytest or pytest --output "foldername" deletes the folder where it was supposed to store output. Did not find any information about this behavior whatsoever
Related
I have an wget command running, but due to the website it regularly downloads "fake data". So, I am now downloading it dozens of times to get the correct content.
I now want the following:
use the wget command
check if in that directory are files <100kB
if yes ->use robocopy to save the >100kB files to another folder; repeat with wget
if no -> stop
I have the wget command and the robocopy command.
I only need now the function to check if an directory has a file <100kB and how to put this together into an cmd/powershell/bat or whatever.
How would the command for that look like? I have especially no idea how to check the files.
I am stuck with this incredibly silly error. I am trying to run pytest on a Raspberry Pi using bluepy.
pi#pi:~/bluepy/bluepy $ pytest test_asdf.py
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/pi/bluepy, inifile:
collected 0 items / 1 errors
==================================== ERRORS ====================================
______________ ERROR collecting bluepy/test_bluetoothutility.py _______________
ImportError while importing test module '/home/pi/bluepy/bluepy/test_asdf.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test_asdf:4: in <module>
from asdf import AsDf
asdf.py:2: in <module>
from bluepy.btle import *
E ImportError: No module named btle
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.65 seconds ============================
I realised that my problem could be that rootdir is showing incorrect path. It should be
/home/pi/bluepy/bluepy
I've been reading pytest docs but I just do not get it how to change the rootdir.
Your problem is nothing to do with Pytest's rootdir.
The rootdir in Pytest has no connection to how test package names are constructed and rootdir is not added to sys.path, as you can see from the problem you were experiencing. (Beware: the directory that is considered rootdir may be added to the path for other reasons, such as it also being the current working directory when you run python -m pytest.)
The problem here, as others have described, is that the top-level bluepy/ is not in sys.path. The easiest way to handle this if you just want to get something running interactively for yourself is as per Cecil Curry's answer: cd to the top-level bluepy and run Pytest as python -m pytest bluepy/test_asdf.py (or just python -m pytest if you want it to discover all test_* files in or under the current directory and run them). But I think you will need to use python -m pytest, not just pytest, in order to make sure that the current working directory is in the path.
If you're looking to set up a test framework that others can easily run without mysterious failures like this, you'll want to set up a test script that sets the current working directory or PYTHONPATH or whatever appropriately. Or use tox. Or just make this a Python package using standard tools that can run the tests for you. (All that goes way beyond the scope of this question.)
By the way, I concur with Cecil's opinion of Mackie Messer's answer; messing around with conftest.py like that is overly difficult and fragile; there are better solutions for almost any circumstance.
Appendix: Use of rootdir
There are only two things, as far as I'm aware, for which rootdir is used:
The .pytest_cache/ directory is stored in the rootdir unless otherwise specified (with the cache_dir configuration option).
If rootdir contains a conftest.py, it will always be loaded, even if no test files are loaded from in or under the rootdir.
The documentation claims that the rootdir also used to generate nodeids, but adding a conftest.py containing
def pytest_runtest_logstart(nodeid, location):
print("logstart nodeid={} location={}".format(nodeid, location))
and running pytest --rootdir=/somewhere/way/outside/the/tree shows that to be incorrect (though node locations are relative to the rootdir).
My first guess would be that you don't have that directory in the python path. You can add it to the python path dynamically. One simple way to do this is in a test configuration file conftest.py, which I believe is always executed before test discovery and test running.
For example, you might have a project setup like:
root
+-- tests
| +-- conftest.py
| +-- tests_asdf.py
+-- bluepy (or main project dir)
| +-- miscellaneous modules
In which case, you could add the root dir to your python path in the conftest.py file like so:
#
# conftest.py
import sys
from os.path import dirname as d
from os.path import abspath, join
root_dir = d(d(abspath(__file__)))
sys.path.append(root_dir)
Let me know if that's helpful.
Actually, py.test is correctly discovering the rootdir for your project to be /home/pi/bluepy. That's good.
Tragically, you are erroneously attempting to run py.test within your project's package subdirectory (i.e., /home/pi/bluepy/bluepy) rather than within your project's rootdir (i.e., /home/pi/bluepy). That's bad.
Let's break this down a little. From within the:
/home/pi/bluepy directory, there is a bluepy.btle submodule. (Good.)
/home/pi/bluepy/bluepy subdirectory, there is no bluepy.btle submodule. (Bad.) Unless you awkwardly attempt to manually inject the parent directory of this subdirectory (i.e., /home/pi/bluepy) onto sys.path as Makie Messer perhaps inadvisably advises, Python has no means of inferring that the package bluepy actually refers to the current directory coincidentally also named bluepy. To avoid ambiguity issues of this sort, Python is typically only run outside rather than inside of a project's package subdirectory.
Since you ran py.test from the latter rather than the former directory, Python is unable to find the bluepy.btle submodule on the current sys.path. For this and similar reasons, py.test should typically only ever be run from your project's top-level rootdir (i.e., /home/pi/bluepy):
pi#pi:~/ $ cd ~/bluepy
pi#pi:~/bluepy $ py.test bluepy/test_asdf.py
Lastly, note that it's typically preferable to defer test discovery to py.test. Rather than explicitly listing all test script filenames on the command line, consider instead letting py.test implicitly find and run all tests containing some substring via the -k option. For example, to run all tests whose function names are prefixed by test_asdf (regardless of the test script they reside in):
pi#pi:~/ $ cd ~/bluepy
pi#pi:~/bluepy $ py.test -k test_asdf .
The suffixing . is optional, but often useful. It instructs py.test to set its rootdir property to the current directory (i.e., /home/pi/bluepy). py.test is usually capable of finding your project's rootdir and setting this property on its own, but it can't hurt to specify it manually. (Especially as you're having... issues.)
For further details on rootdir discovery, see Initialization: determining rootdir and inifile in the official py.test documentation.
I have a script which is running gsutil rsync -r -d -c, and occasionally it will leave the source and target directories out of sync. The last file in in the list (named version.json) is first uploaded, and then immediately deleted.
Has anybody encountered this bug?
Additional information:
versioning is turned off in the target bucket
This occurs when attempting to overwrite the entire contents of the target bucket, which is already present.
In my workflow, I have lots of xxx.smr files in a folder and I need to convert them into other file format xxx_step3.mat by importing some data from xxx_info.xlsx. I learned that GNU make is powerful in keep all the files up-to-date.
In a very simple "explicit" format (without sophisticated wild card usage), Makefile for this process would look like this. To handle multiple xxx.smr files and their descendants, I should be able to do that by modifying this file.
.PHONY: all clean
all: xxx_step3.mat
xxx_step3.mat: xxx_step2.mat xxx_info.xlsx
matlab -r "merge2files('xxx_step2.mat', 'xxx_info.xlsx')"
xxx_step2.mat: xxx_step1.mat
matlab -r "convertmat('xxx_step1.mat')"
xxx_info.xlsx: master.xslx
matlab -r "extractfromMasterxlsx('master.xlsx', 'xxx_info.xlsx')"
xxx_step1.mat: xxx_step0.smr
#echo "\nCreate " $#
# I can't do this step from the command line so I leave message
clean:
rm -f xxx_step1.mat xxx_step2.mat xxx_step3.mat xxx_info.xlsx
However, I realized that, when some of xxx.smr files were found to be surplus and deleted at some point, running GNU make with this Makefile does not delete the obsolete descendant files, including all the intermediate files and the final xxx_step3.mat files, that are dependent on those deleted xxx.smr files.
For example, I start with the three xxx.smr files and run Make.
A.smr, B.smr, C.smr
It will create all the descendants, including the final target files:
A_step3.mat, B_step3.mat, C_step3.mat
Later, say, I find the B.smr contained a fatal error and decided to delete from the folder.
A.smr, C.smr
Running Make at this stage will result in ... no change, because both A_step3.mat and C_step3.mat are newer than its direct prerequisites (and than A.smr and C.smr). However, actually I need to remove all the descendants of B.smr, such as B_step1.mat, B_step2.mat, B_step3.mat, and B_info.xlsx. If those obsolete files are kept, the final target B_step3.mat will be included in the subsequent analyses and affect the results.
I wonder if there is a "smart" way of removing xxx_step1.mat, xxx_step2.mat, xxx_step3.mat, xxx_info.xlsx files, when their corresponding xxx.smr files have been deleted.
Or should I just implement this with MATLAB or Python etc?
Since a Makefile is a collection of shell commands, on your clean: target, you can collect and remove all the files that correspond to your xxx.smr files using a for loop and parameter expansion/substring matching. To find all files that correspond to each xxx.smr file, find all xxx.smr files. Then for each xxx.smr, extract xxx and remove all xxx_step?.* and xxx_info.* files. After each of the step? and info files are removed, then remove xxx.smr. In multi-line form it would look like:
for i in *.smr; do
for j in ${i%.*}; do
rm -f "${j}_step?.*" "${j}_info.*"
done
rm -f "$i"
done
Or, in a single line:
for i in *.smr; do for j in ${i%.*}; do rm -f "${j}_step?.*" "${j}_info.*"; done; rm -f "$i"; done
Note this will remove all xxx_step... and xxx_info... files for each xxx.smr file. Make sure this is what you intend and run on a test directory first. You can tighten the extensions above to just remove xxx_info.xlsx by replacing xxx_info.* with xxx_info.xlsx, etc...
I'm creating a recycle-bin script in SH shell linux in three differant scripts, delete, trash and restore.
The first two scripts are working fine; 'Delete' moves the selected file to the recycle-bin while logging a text file called 'trashinfo' which shows the original path location of the file (to be later used in restore) and 'Trash' which removes everything in the recycle-bin.
The 'restore' script should take the logged path name gained in the delete script and return the file to its original location. I've spent more time than I'd like to remember on this and cant get the restore script to work properly!
Below is the script I've written, as far as I can make out I'm grepping for the filename variable in the text file that holds the pathname, eg 'restore testfile', this is then combined with the basename command, the testfile is then moved into the location thats been grepped and combined with the basename.
Anyone have any pointers on where I'm going wrong?
if [ "$*" != -f ]
then
path=grep "$*" /usr/local/bin/trashinfo
pathname=basename "$path"
mv "$path" "$pathname"
path=$(grep "$*" /usr/local/bin/trashinfo)
pathname=$(basename "$path")