Recommended way to run commands after installing dependencies in the virtualenv - virtualenv

I would like to use tox to run py.test on a project which needs additional setup in addition to installing packages into the virtualenv. After creating the virtualenv and installing dependencies, some commands need to be run.
Specifically I'm talking about setting up a node and npm environment using nodeenv:
nodeenv --prebuilt -p
I see that tox allows me to provide a custom command used for installing dependencies by setting install_command in tox.ini. But I don't think this is what I want because that replaces the command (I assume pip) used to install dependencies.
I thought about using a py.test fixture with session scope to handle setting up nodeenv but that seems hacky to me as I don't want this to happen when py.test is run directly, not via tox.
What is the least insane way of achieving this?

You can do all necessary setup after the creation of the virtualenv and the dependency installation in commands. Yes, it says "the commands to be called for testing." but if you need to do extra work to prepare for testing you can just do it right there.
It works through whatever you throw at it in the order it is given - e.g.:
[testenv:someenv]
deps =
nodeenv
pytest
flexmock
commands =
nodeenv --prebuilt -p
; ... and whatever else you might need to do
py.test path/to/my/tests
If you have commands/scripts or whatever else that produces the right result but it returns a non zero exit status you can ignore that by prepending - (as in - naughty-command).
If you need more steps to happen you can wrap them in a little (Python) script and call that script instead as outlined in https://stackoverflow.com/a/47834447/2626627.
There is also an issue to add the ability to use more than one install command: https://github.com/tox-dev/tox/issues/715 is implemented.

I had the same issue, and as it was important for me to be able to create the environment without invoking the tests (via --notest), I wanted the install to happen in the install phase and not the run phase, so I did something slightly differently. First, I created a create-env script:
#!/usr/bin/env sh
set -e
pip install $#
nodeenv --prebuilt --python-virtualenv --node=8.2.1
Made it executable, Then in tox.ini:
[tox]
skipsdist = True
[testenv]
install_command = ./create-env {opts} {packages}
deps = nodeenv
commands = node --version
This complete example runs and outputs the following:
$ tox
python create: .../.tox/python
python installdeps: nodeenv
python installed: nodeenv==1.3.0
python runtests: PYTHONHASHSEED='1150209523'
python runtests: commands[0] | node --version
v8.2.1
_____________________________________________________________________ summary ______________________________________________________________________
python: commands succeeded
congratulations :)
This approach has the downside that it would only work on Unix.
In tox 715, I propose the possibility of native support for multiple install commands.

Related

How to switch NVM environments within perl

I am writing a perl script, and I want to run a simple shell command to use a certain version of NVM:
Here is my code snippet:
print "\n*** Switching to correct nvm environment for dashboard builds\n";
system("nvm use 8.12.0") == 0 or die $?;
But I am getting the following error:
Can't exec "nvm": No such file or directory
Can someone help?
Update (June 30, 2021):
I also tried adding the command:
my $nvm_version = "8.12.0";
system ("bash", "-lic", "nvm use $nvm_version");
But nothing happens:
I'm not familiar with nwm, but I think I get the gist of what it does. And if so, the attempt is fundamentally flawed. Even if you fixed this to run the proper shell so that nvm could run, I believe all the tool does is change the shell's environment variables, a shell you immediately exit. This means it would have no effect even if if it ran successfully.
Again, it this tool does what I think it does, such tool are meant to be used in interactive shells. In other instances, you simply use the path the to correct executable instead of relying on the PATH.
With that in mind, you can use the following to run the command in bash:
# Non-interactive shell.
system("bash", "-c", "nvm use 8.12.0")
or
# Interactive shell.
# This is improper and fragile as interactive shells
# often create aliases that override basic commands.
system("bash", "-ic", "nvm use 8.12.0")
Just to reiterate, at least one of these will allow the command to run (if it normally works from bash), but I believe it's unlikely this will produce the results you expect.
The nvm command is shell function which is different from a shell command. Also the nvm command is not an exported function so it will not be seen by sub shells. For example, in Bash shell:
$ nvm ls
-> v15.0.1
$ my-test-script.sh
./my-test-script.sh: line 3: nvm: command not found
where my-test-script.sh is:
#! /bin/bash
nvm use 16.4
The error nvm: command not found is because nvm is not exported. I can source the script in the current shell context to make it work:
$ source my-test-script.sh
Now using node v16.4.0 (npm v7.18.1)
$ node --version
v16.4.0
So a Perl script cannot change the node version of the current shell, but it can calculate the version and pass it back to shell, which can set the version. For example:
$ nvm use $(perl -E'$v=15.0; print $v')
Now using node v15.0.1 (npm v7.0.3)

How to run a pytest-bdd test?

I am not understanding how to properly run a simple test(feature file and python file)
with the library pytest-bdd.
From the official documentation, I can't understand what command to issue to run a test.
I tried using pytest command, but I saw the NO test ran.
Do I need to use another library behave to run a feature file?
I figured out trying for 2 days,that ,
for running a pytest-bdd test, there are certain requirements, at least in my view.
put both the feature file and python file in the same directory (maybe this can be changed with configuration files)
the python file name needs to start with test_
the python file needs to contain a method of which name will start with test_
the method starting with test_ , need to be assigned to the #scenario sentence
to run the test, issue pytest command in the same directory(maybe it is also configurable)
After issuing you will only see the method with the name starting with test_ has passed, but all the tests actually ran. To test, you can assert False in any #when or #then annotated method, it will throw errors.
The system contained : pytest-bdd==3.0.2 (copied from pip freeze output)
Features files and python files can be placed in different folders using the bdd_features_base_dir hook provided by pytest-bdd; I think it is better having features files in different folders too.
Here you can see a working example (a simple hello world BDD test):
https://github.com/davidemoro/pytest-play-docker/tree/master/tests
https://github.com/davidemoro/pytest-play-docker/blob/master/tests/pytest.ini (see bdd_features_base_dir in [pytest] section)
https://github.com/davidemoro/pytest-play-docker/tree/master/tests/bdd
If you want to try out pytest-bdd without installation you can use Docker. Create a folder with inside your pytest BDD files and if you want a separate features folder targeted in bdd_features_base_dir and run:
docker run --rm -it -v $(pwd):/src davidemoro/pytest-play:latest
I've found out, that in the python file you don't have to put:
the method starting with test_ , need to be assigned to the #scenario sentence
You can just add: scenarios("") - to allow the tests to be started, which are using steps defined in this specific python file.
Remember to import scenarios!: from pytest_bdd import scenarios
Example:
Code example
Command..
pytest -v path_to_test_file.py
Things to note here..
Check format of feature file as filename.feature
Always __init__ modules, otherwise test-runner will not find test files
Glue right step definitions to test function
Add feature in features module
If you are using python3 execute test with python3
So,
python3 -m pytest -v path_to_test_file.py
Documentation
https://pytest-bdd.readthedocs.io/en/stable/#

Running a test with tox based on a keyword

I am using pytest with tox. I can run some of my tests with a keyword like this:
pytest -k <keyword> path/to/tests
Now it would be really convenient to be able to do this also with tox, as the environments there are clean and different python versions can be tested. However the nearest thing I have found is:
tox -- path/to/tests/test_very_specific_name.py:TestClass.test_func
This is not easy to type, so I rather just run tox without arguments and wait 2 minutes for everything to finish.
Is there a way to run single tests based on keywords with tox? I tried:
tox -- -k <keyword>
This results in a huge list of import errors. It doesn't seem to be able to find any of my local includes. Is this supposed to work?
I figured it out thanks to the comment by phd.
Everything on the command line after -- can be used in tox.ini as {posargs}. I was using that wrong. My tox.ini now has a line like this:
commands = py.test {posargs} <test_folder>
Now it works perfectly with:
tox -- -k <keyword>

Is there a way to prepend a command with a variable assignment like in bash?

In bash one can write
CFLAGS="-O2" rvm install 2.0.0
to run rvm with that specific CFLAGS . Is there anyway to do the same in fish shell?
I know about set -x but that is not exactly the same as the environment variable will be set for the whole session instead of just for that command.
According to the fish FAQ, either use:
env CFLAGS="-O2" rvm install 2.0.0
(which will not work for fish builtins or functions, only external commands), or
begin
set -lx CFLAGS="-O2"
rvm install 2.0.0
end
(which is a little clunky; there are proposals for improvement on GitHub issue #438).
You can use the env command for this:
env FOO=BAR command
Will run command with env variable FOO set to BAR.

Install Perl module with assume yes for given options non-interactively

Normally in linux Debian we do sth like this to install a package non-interactively e.g
sudo apt-get install -y Package_x_z
#[-y --assume-yes]
How we can do the same while installing a perl module e.g
sudo perl -MCPAN -e 'install DBI'
That prompt is (typically) coming from ExtUtils::MakeMaker's prompt() function. Stick export PERL_MM_USE_DEFAULT=1 in your .bashrc (or equivalent for your preferred shell) to stop the prompts. The ExUtils::MakeMaker man page documents it thus:
PERL_MM_USE_DEFAULT
If set to a true value then MakeMaker's prompt function will always return the default
without waiting for user input.
Note that this can come to bite you if you run cpan(1) on a box that's not yet had CPAN repositories configured. It will rattle on and get stuck in a prompt loop at a point where there is no default and you need to make a choice, but have no ability to do so. export PERL_MM_USE_DEFAULT=0 in the shell before running cpan(1) will of course temporarily re-enable input.
To prevent the CPAN client from asking whether to install prerequisites, start it in interactive mode
perl -MCPAN -e shell
and enter the commands:
o conf build_requires_install_policy yes
o conf prerequisites_policy follow
o conf commit
The commit command is optional, but it will update the default configuration, which I suspect is what you want. Without it, you may or may not (depending on whether autocommit is enabled in your CPAN config) need to make this change every time you want to do a prompt-less installation.
These changes will deal with all of the CPAN client's routine questions about whether to install dependencies. For distributions which have questions embedded in their install scripts, you may also want to add
o conf inactivity_timeout 60
to set how long it will wait for a response before automatically going with the default answer to the question. (Set it to 0 to change it back to "wait forever".)
What about just :
$ yes | sudo perl -MCPAN -e 'install DBI'
Ban ! your problem is solved :-)
Appending to an answer here, you can also make these changes in config file located at /usr/share/perl5/CPAN/Config.pm.
'build_requires_install_policy' => q[yes],
'prerequisites_policy' => q[follow],
This helped me to automate installation, since CPAN doesn't have these configuration by default.