Question:
What would be the best way to import functions to tests that don't reside in the same directory?
Example
📁 src
📄 Get-Emoji.ps1
📁 test
📄 Get-Emoji.Tests.ps1
Inb4
Pester documentation[1] suggests test files are placed in the same directory as the code that they test. No examples of alternatives provided.
Pester documentation[2] suggests dot-sourcing to import files. Only with examples from within same directory
Whether breaking out tests from the src is good practice, is to be discussed elsewhere
Using Powershell Core for cross platform support on different os filesystems (forward- vs backward slash)
[1] File placement and naming convention
Pester considers all files named .Tests.ps1 to be test files. This is the default naming convention that is used by almost all projects.
Test files are placed in the same directory as the code that they test. Each file is called as the function it tests. This means that for a function Get-Emoji we would have Get-Emoji.Tests.ps1 and Get-Emoji.ps1 in the same directory. What would be the best way to referencing tests to functions in Pester.
[2] Importing the tested functions
Pester tests are placed in .Tests.ps1 file, for example Get-Emoji.Tests.ps1. The code is placed in Get-Emoji.ps1.
To make the tested code available to the test we need to import the code file. This is done by dot-sourcing the file into the current scope like this:
Example 1
# at the top of Get-Emoji.Tests.ps1
BeforeAll {
. $PSScriptRoot/Get-Emoji.ps1
}
Example 2
# at the top of Get-Emoji.Tests.ps1
BeforeAll {
. $PSCommandPath.Replace('.Tests.ps1','.ps1')
}
I tend to keep my tests together in a single folder that is one or two parent levels away from where the script is (which is usually under a named module directory and then within a folder named either Private or Public). I just dot source my script or module and use .. to reference the parent path with $PSScriptRoot (the current scripts path) as a point of reference. For example:
Script in \SomeModule\Public\get-something.ps1
Tests in \Tests\get-something.tests.ps1
BeforeAll {
. $PSScriptRoot\..\SomeModule\Public\get-something.ps1
}
Use forward slashes if cross platform compatibility is a concern, Windows doesn't mind if path separators are forward or backslashes. You could also run this path through Resolve-Path first if you wanted to be certain a valid full path is used, but I don't generally find that necessary.
Related
I just got an error in Powershell:
because the module nesting limit has been exceeded. Modules can only be nested to 10 levels.
I found this and found something called a "module manifest". I already have a module .psm1 files - why do I also need this?
Note: I dont have 10 levels of modules, I have 10 modules, all loaded in by one import.psm1 file.
The module nesting level getting exceeded is typically the result of accidentally getting into an infinite recursion during module import (irrespective of whether you import via Import-Module or the PSv5+ using module statement).
This can happen whether or not your module has a manifest; the answer to the linked question shows how it can happen with a manifest; here's an example without one: the following foo.psm1 module causes an infinite recursion that results in the error message you saw
# Create sample module (without manifest).
#'
# Accidentally try to import the module itself.
using module .\foo.psm1
function bar { 'hi from module foo' }
'# > foo.psm1
# This fails, because an infinite import loop is entered,
# eventually causing the nesting limit to be exceeded.
Import-Module .\foo.psm1
Why creating a (script) module with a manifest is a good idea:
While module manifests are optional - standalone *.psm1 files can serve as modules by themselves - there are good reasons to use them:
A module manifest is a *.psd1 file that accompanies your *.psm1 file and specifies important metadata, notably the version number, in the form of a hashtable literal; for the best user experience, both files should be placed in a directory of the same name (e.g., Foo.psm1 and its manifest, Foo.psd1, should be placed in a directory named Foo).
By using a manifested module, you enable several important use cases:
You need a manifest to properly support your module's software-development processes, notably version management.
It is also a prerequisite for supporting side-by-side installations of multiple versions of your module.
You need a manifest to automatically load associated resources, such as other modules or auxiliary .NET assemblies, and to define help resources.
You need a manifest in order to integrate with PowerShell's module auto-loading mechanism: If you place your properly manifested module into one of the directories listed in $env:PSModulePath, PowerShell will:
Discover the module and its commands even before the module is imported.
Will import it on demand, the first time you try to call a command from the session.
You need a manifest in order to publish a module to the official online repository for PowerShell modules, the PowerShell Gallery
To quickly outline the steps for creating a module with manifest:
Create a directory named for the base name of your .psm1 file; e.g., Foo
Place the script code as file Foo.psm1 file in that directory.
In the same directory, using the New-ModuleManifest cmdlet, create the manifest .psd1 file with the same base name (e.g., Foo.psd1)
At the very least, update the RootModule entry in the new .psd1 file to point to your .psm1 file (e.g., RootModule = 'Foo.psm1')
To integrate with the auto-loading feature, place your module directory in one of the locations listed in $env:PSModulePath; for the current user, that location is:
Windows PowerShell:
$HOME\Documents\WindowsPowerShell\Modules
PowerShell [Core] v6+:
Windows: $HOME\Documents\PowerShell\Modules
Linux, macOS: $HOME/.local/share/powershell/Modules
To support module discovery and auto-loading efficiently and to explicitly control and signal what a module exports, it is best to explicitly list the individual exported module members in the FunctionsToExport,
CmdletsToExport, VariablesToExport, and AliasesToExport entries of the manifest.
To make module creation easier, the community has provided helper modules:
Plaster is a "template-based file and project generator written in PowerShell", that can also be used to scaffold modules:
The built-in "New PowerShell Manifest Module" template scaffolds a module directory with all necessary files and support for Pester tests.
See this blog post for a walk-through.
Stucco builds on Plaster to provide an "opinionated Plaster template for building high-quality PowerShell modules."
Stucco is an advanced tool that goes beyond mere module creation, by scaffolding an entire project structure that includes psake tasks, scaffolding for CI/CD integration, licensing, and help authoring.
A quick example with Plaster:
# Install Plaster for the current user, if necessary.
Install-Module Plaster -Scope CurrentUser
# Get the template for creating a new script module.
$template = Get-PlasterTemplate | Where TemplatePath -match ScriptModule
# Scaffold a module in subdirectory 'Foo'
# * This will present a series of prompts, most of them with default values.
# * IMPORTANT: Be sure to also choose 'Foo' as the module *name* when prompted,
# so that the module auto-loading feature can discover your module
# (if placed in a dir. in $env:PSModulePath) and also so that you
# you can load it by its *directory* path; e.g., Import-Module ./Foo
Invoke-Plaster -TemplatePath $template.TemplatePath -Destination Foo
# Add a test function to the `.psm1` file.
# Note:
# * This is just for illustrative purposes. In real life, you would
# obviously use an editor to add functions to your module.
# * The function must be placed *before* the `Export-ModuleMember` call in order
# to be exported.
# * As stated, it is additionally recommended to list the exported members
# *explicitly*, one by one, in the *ToExport keys of the *.psd1 file.
(Get-Content -Raw ./Foo/Foo.psm1) -replace '\r?\n\r?\n', "`n`nfunction Get-Foo { 'Hi from module Foo.' }`n" | Set-Content -Encoding utf8 ./Foo/Foo.psm1
# Import the newly created module by its *directory* path.
# IMPORTANT:
# As stated, this assumes that you specified 'Foo' as the module name, i.e.
# that your manifest's file name is 'Foo.psd1', and your script module's
# 'Foo.psm1'.
Import-Module ./Foo -Verbose -Force
'---'
# Call the test function
Get-Foo
'---'
# Invoke the module's tests.
# Note: The scaffolding creates a single test to ensure that the
# module manifest (*.psd1) is valid.
Invoke-Pester ./Foo
You should see output such as the following:
VERBOSE: Loading module from path 'C:\Users\jdoe\Foo\Foo.psd1'.
VERBOSE: Loading module from path 'C:\Users\jdoe\Foo\Foo.psm1'.
VERBOSE: Importing function 'Get-Foo'.
---
Hi from module Foo.
---
____ __
/ __ \___ _____/ /____ _____
/ /_/ / _ \/ ___/ __/ _ \/ ___/
/ ____/ __(__ ) /_/ __/ /
/_/ \___/____/\__/\___/_/
Pester v4.9.0
Executing all tests in './Foo'
Executing script C:\Users\jdoe\Foo\test\Foo.Tests.ps1
Describing Module Manifest Tests
[+] Passes Test-ModuleManifest 128ms
Tests completed in 375ms
Tests Passed: 1, Failed: 0, Skipped: 0, Pending: 0, Inconclusive: 0
Have PowerShell script in resource folder and want to use this script in shared library .groovy script which is under vars folder. Is it there a way to do so??
To run a PS script from the resources dir of your shared library, assuming psScript1.ps1 is in your-shared-library/resources/:
powershell "${libraryResource 'psScript1.ps1'}"
This works great, however, it doesn't let you pass in parameters. I can't seem to find a one-line way to do it.
Building on #christopher's answer, here's a two-line solution:
writeFile file: 'tempFile.ps1', text: "${libraryResource 'psScript1.ps1'}"
powershell './tempFile.ps1 -yourParam $yourParam'
In my testing, I found the ./ to start the temp file path to be required.
You can use libraryResource(file-in-resources) (docs) to load a file from the library resources as a text. That you can write to your workspace with writeFile (docs) and execute.
As the library is checked out somewhere next to your workspace you can maybe execute it directly.
Let's say I had a file File.jl that had a module MyModule containing the functions foo and bar in it. In the same directory as the module-file, I had a script Script.jl, and I wanted to use the functions in MyModule in the script.
How would one go about doing this?
In order to find Modules that are not in the standard LOAD_PATH and be able to import them, you need to update your LOAD_PATH variable for the current folder explicitly
push!( LOAD_PATH, "./" )
then you will be able to import a module appropriately.
Note that, if the file is called File.jl and defines the module MyModule, what you should be importing is import MyModule, not import File. It is generally recommended you use the same name for the file as for the defined module in this kind of scenario, to avoid confusion.
Also note, As #crstnbr noted above, you can also simply 'dump' the file's contents into the current session by simply 'including' it; note however that this simply creates the module on the spot, so any precompilation directives etc will not be honoured.
Somewhat related questions / answers (disclaimer: by me) you might find useful:
https://stackoverflow.com/a/50627721/4183191
https://stackoverflow.com/a/49405645/4183191
You include the file with the module definition and call the functions in your script file:
include(joinpath(#__DIR__,"File.jl"))
MyModule.foo()
MyModule.bar()
# or just foor() and bar() if MyModule exports those functions
The #__DIR__ expands to the directory of the script file, see
help?> #__DIR__
#__DIR__ -> AbstractString
Expand to a string with the absolute path to the directory of the file containing the macrocall. Return the current working directory if run from a REPL or if evaluated by julia -e <expr>.
I am stuck with this incredibly silly error. I am trying to run pytest on a Raspberry Pi using bluepy.
pi#pi:~/bluepy/bluepy $ pytest test_asdf.py
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/pi/bluepy, inifile:
collected 0 items / 1 errors
==================================== ERRORS ====================================
______________ ERROR collecting bluepy/test_bluetoothutility.py _______________
ImportError while importing test module '/home/pi/bluepy/bluepy/test_asdf.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test_asdf:4: in <module>
from asdf import AsDf
asdf.py:2: in <module>
from bluepy.btle import *
E ImportError: No module named btle
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.65 seconds ============================
I realised that my problem could be that rootdir is showing incorrect path. It should be
/home/pi/bluepy/bluepy
I've been reading pytest docs but I just do not get it how to change the rootdir.
Your problem is nothing to do with Pytest's rootdir.
The rootdir in Pytest has no connection to how test package names are constructed and rootdir is not added to sys.path, as you can see from the problem you were experiencing. (Beware: the directory that is considered rootdir may be added to the path for other reasons, such as it also being the current working directory when you run python -m pytest.)
The problem here, as others have described, is that the top-level bluepy/ is not in sys.path. The easiest way to handle this if you just want to get something running interactively for yourself is as per Cecil Curry's answer: cd to the top-level bluepy and run Pytest as python -m pytest bluepy/test_asdf.py (or just python -m pytest if you want it to discover all test_* files in or under the current directory and run them). But I think you will need to use python -m pytest, not just pytest, in order to make sure that the current working directory is in the path.
If you're looking to set up a test framework that others can easily run without mysterious failures like this, you'll want to set up a test script that sets the current working directory or PYTHONPATH or whatever appropriately. Or use tox. Or just make this a Python package using standard tools that can run the tests for you. (All that goes way beyond the scope of this question.)
By the way, I concur with Cecil's opinion of Mackie Messer's answer; messing around with conftest.py like that is overly difficult and fragile; there are better solutions for almost any circumstance.
Appendix: Use of rootdir
There are only two things, as far as I'm aware, for which rootdir is used:
The .pytest_cache/ directory is stored in the rootdir unless otherwise specified (with the cache_dir configuration option).
If rootdir contains a conftest.py, it will always be loaded, even if no test files are loaded from in or under the rootdir.
The documentation claims that the rootdir also used to generate nodeids, but adding a conftest.py containing
def pytest_runtest_logstart(nodeid, location):
print("logstart nodeid={} location={}".format(nodeid, location))
and running pytest --rootdir=/somewhere/way/outside/the/tree shows that to be incorrect (though node locations are relative to the rootdir).
My first guess would be that you don't have that directory in the python path. You can add it to the python path dynamically. One simple way to do this is in a test configuration file conftest.py, which I believe is always executed before test discovery and test running.
For example, you might have a project setup like:
root
+-- tests
| +-- conftest.py
| +-- tests_asdf.py
+-- bluepy (or main project dir)
| +-- miscellaneous modules
In which case, you could add the root dir to your python path in the conftest.py file like so:
#
# conftest.py
import sys
from os.path import dirname as d
from os.path import abspath, join
root_dir = d(d(abspath(__file__)))
sys.path.append(root_dir)
Let me know if that's helpful.
Actually, py.test is correctly discovering the rootdir for your project to be /home/pi/bluepy. That's good.
Tragically, you are erroneously attempting to run py.test within your project's package subdirectory (i.e., /home/pi/bluepy/bluepy) rather than within your project's rootdir (i.e., /home/pi/bluepy). That's bad.
Let's break this down a little. From within the:
/home/pi/bluepy directory, there is a bluepy.btle submodule. (Good.)
/home/pi/bluepy/bluepy subdirectory, there is no bluepy.btle submodule. (Bad.) Unless you awkwardly attempt to manually inject the parent directory of this subdirectory (i.e., /home/pi/bluepy) onto sys.path as Makie Messer perhaps inadvisably advises, Python has no means of inferring that the package bluepy actually refers to the current directory coincidentally also named bluepy. To avoid ambiguity issues of this sort, Python is typically only run outside rather than inside of a project's package subdirectory.
Since you ran py.test from the latter rather than the former directory, Python is unable to find the bluepy.btle submodule on the current sys.path. For this and similar reasons, py.test should typically only ever be run from your project's top-level rootdir (i.e., /home/pi/bluepy):
pi#pi:~/ $ cd ~/bluepy
pi#pi:~/bluepy $ py.test bluepy/test_asdf.py
Lastly, note that it's typically preferable to defer test discovery to py.test. Rather than explicitly listing all test script filenames on the command line, consider instead letting py.test implicitly find and run all tests containing some substring via the -k option. For example, to run all tests whose function names are prefixed by test_asdf (regardless of the test script they reside in):
pi#pi:~/ $ cd ~/bluepy
pi#pi:~/bluepy $ py.test -k test_asdf .
The suffixing . is optional, but often useful. It instructs py.test to set its rootdir property to the current directory (i.e., /home/pi/bluepy). py.test is usually capable of finding your project's rootdir and setting this property on its own, but it can't hurt to specify it manually. (Especially as you're having... issues.)
For further details on rootdir discovery, see Initialization: determining rootdir and inifile in the official py.test documentation.
I'm a newbie in PowerShell and I'm having some problems when defining some utility scripts that are "included" in other files; the problem is regarding paths. Let me explain the issue with a simple example:
Imagine you have some utility script named utility.ps1 located under /tools and you want to invoke it from a build.ps1 placed under /build. Now imagine that the utility.ps1 invokes some other utility script in the same folder called "utility2.ps1". So,
utility.ps1:
[...]
.".\utility2.ps1"
[...]
build.ps1:
[...]
."..\tools\utility.ps1"
[...]
The problem here is that when calling from build to utility.ps1 and then from this last one to utility2.ps1, powershell is trying to load utility2.ps1 from the current dir (build) instead of the tools dir.
My current workaround is pushd . before calling and popd after that but it smells bad, I know; on the other hand if some day I move scripts to some other location I'd need to update all my scripts that use the moved one... a mess.
So, I'm doing something very wrong; what would be the proper one to do this, making the client script unaware and independant of the "utility" script location?
The answer by PetSerAl in the comments is perfect but will work for PowerShell 3+. If for some reason you want your script to be backward compatible as well, then use the below code snippet to find the $ScriptRootPath (which is automatically populated via $PSScriptRoot for PS 3+)
$ScriptRootPath = Split-Path $MyInvocation.MyCommand.Path -Parent
For PS 3+, populate this variable via the global variable:
$ScriptRootPath = $PSScriptRoot
and then use the solution as described by PetSerAl,
i.e. for utility.ps1: . "$ScriptRootPath\utility2.ps1"
for build.ps1: . "$ScriptRootPath\..\tools\utility.ps1"
Also refer this question on SO