How to debug unit test while developping a package in Julia - visual-studio-code

Say I develop a package with a limited set of dependencies (for example, LinearAlgebra).
In the Unit testing part, I might need additional dependencies (for instance, CSV to load a file). I can configure that in the Project.toml all good.
Now from there and in VS Code, how can I debug the Unit tests? I tried running the "runtests.jl" in the debugger; however, it unsurprisingly complains that the CSV package is unavailable.
I could add the CSV package (as a temporary solution), but I would prefer that the debugger run with the configuration for the unit testing; how can I achieve that?
As requested, here is how it can be reproduced (it is not quite minimal, but instead I used a commonly used package as it give confidence the package is not the problem). We will use DataFrames and try to execute the debugger for its unit tests.
Make a local version of DataFrames for the purpose of developing a feature in it. I execute dev DataFrames in a new REPL.
Select the correct environment (in .julia/dev/DataFrames) through the VS-code user interface.
Execute the "proper" unit testing by executing test DataFrames at the pkg prompt. Everything should go smoothly.
Try to execute the tests directly (open the runtests.jl and use the "Run" button in vs-code). I see some errors of the type:
LoadError: ArgumentError: Package CategoricalArrays not found in current path:
- Run `import Pkg; Pkg.add("CategoricalArrays")` to install the CategoricalArrays package.
which is consistent with CategoricalArrays being present in the [extras] section of the Project.toml but not present in the [deps].
Finally, instead of the "Run" command, execute the "Run and Debug". I encounter similar errors here is the first one:
Test Summary: | Pass Total
merge | 19 19
PASSED: index.jl
FAILED: dataframe.jl
LoadError: ArgumentError: Package DataStructures not found in current path:
- Run `import Pkg; Pkg.add("DataStructures")` to install the DataStructures package.
So I can't debug the code after the part requiring the extras packages.
After all that I delete this package with the command free DataFrames at the pkg prompt.
I see the same behavior in my package.

I'm not certain I understand your question, but I think you might be looking for the TestEnv package. It allows you to activate a temporary environment containing the [extras] dependencies. The discourse announcement contains a good description of the use cases.

Your runtest.jl file should contain all necessary imports to run tests.
Hence you are expected to have in your runtests.jl file lines such as:
using YourPackageName
using CSV
# the lines with tests now go here.
This is a standard in Julia package layout. For an example have a look at any mature Julia such as DataFrames.jl (https://github.com/JuliaData/DataFrames.jl/blob/main/test/runtests.jl).

Related

Unable to run/debug robot tests in vscode - robocorp extensions installed

I have installed Robocorp Code as well as Robot Framework Language Server and have configured them. However, I am still having errors when trying to run the tests via the code lens options.
Repo - A webapi repo with a specific folder containing all tests. Lets call it regression.
RF - 4.1.3
Python - 3.8
This is what happens when I click on Run on the code lens for any of the tests -
`PS C:\git\xxxx\regression> C:; cd 'C:\git\xxxx\regression'; &
'C:\Users\xxxx\AppData\Local\Temp\rf-ls-run\run_env_00_smh5defr.bat'
'-u'
'c:\Users\xxxx.vscode\extensions\robocorp.robotframework-lsp-0.47.2\src\robotframework_debug_adapter\run_robot__main__.py'
'--port' '54331' '--no-debug' '--argumentfile'
'C:\git\xxxx\regression\args-local.txt' '--pythonpath'
'c:\git\xxxx\regression\common\lib' '--variable'
'EXECDIR:C:/git/xxxx/regression'
'--prerunmodifier=robotframework_debug_adapter.prerun_modifiers.FilteringTestsSuiteVisitor'
'c:\git\xxxx\regression\api\api_Test.robot'
[ ERROR ] Parsing'--pythonpath' failed: File or directory to execute does not exist.
However, the test starts if I remove the argumentfile parameter but it, of course, fails because its missing arguments from the file.
Do note that the folder specified in pythopath exists and has some python libraries needed for the tests.

Why can't I add a package (module) I created in Julia?

I am having trouble installing a module I created in Julia. I am running the Julia plugin under Visual Studio Code. If I run the file below with Ctrl+F5 I get a message
ERROR: LoadError: ArgumentError: Package Utils not found in current path:
- Run `import Pkg; Pkg.add("Utils")` to install the Utils package.
This is the file:
module demo
using Utils
greet() = print("Hello World!")
end # module
If I follow the advice on the error message I get another error message:
ERROR: LoadError: The following package names could not be resolved:
* Utils (not found in project, manifest or registry)
I also tried inserting this line:
import Pkg; Pkg.add(path="C:/Dropbox/Code/Julia/demo/src/Utils.jl")
and got this message (although the path definitely exists):
ERROR: LoadError: Path `C:/Dropbox/Code/Julia/demo/src/Utils.jl` does not exist.
The files demo.jl and Utils.jl are in C:\Dropbox\Code\Julia\demo\src\ and the demo project has been activated as can be seen in the REPL. The OS is Windows 10 Pro.
Any help will be greatly appreciated. Lots of time wasted trying to make this work.
Module and packages are not the same things. In short, packages are modules plus a set of metadata that make it easy for the package to be found and interact well with other packages. See here for a tutorial to write Julia packages:
https://syl1.gitbook.io/julia-language-a-concise-tutorial/language-core/11-developing-julia-packages
In your case, if you want to load a local module, just type include("fileWhereThereIsTheModule.jl") followed by a using Main.MyModule or using .MyModule. Note the dot... without it, Julia would indeed look for a package and to let it find your Demo or Util module you would have to either change an environmental variable or place your module file in certain predefined folders. Using include followed by the "absolute or relative position" of the module you don't have to do either.

How can I fix tests in Ember testem with errors such as 'could not load', 'failed', 'could not find module' or 'died'?

I managed to get a couple of EAK/grunt based Ember apps upgraded to 1.11 with HTMLBars, and then got them migrated to Ember CLI/Brocolli. The unit tests were setup for karma test runner so I'm looking at how to get those running in the CLI projects now, but I didn't write the tests and really have no experience with unit testing javascript modules.
Searching around the iNet, I can see that others have also used karma becasue of its coverage output and are trying to get it to work with Ember CLI, but that Ember Core isn't supporting it, though they say anyone should be able to get it set up with a custom addon. I'm also trying to use the 'testem' runner to see what sticks with that.
The Ember site does have an 'automating tests with runners' page for v1.10, with sections on 'testem' and 'karma', but it doesn't appear for v1.11 so I can't tell from that site what is or isn't relevant. But it seems like I should be able to work out a solution for the karma test runner, so I added the old devDependencies to the project package.json:
"karma": "^0.12.31",
"karma-chai": "~0.1.0",
"karma-chrome-launcher": "~0.1.2",
"karma-coverage": "~0.2.1",
"karma-firefox-launcher": "~0.1.3",
"karma-junit-reporter": "~0.2.1",
"karma-mocha": "~0.1.3",
"karma-phantomjs-launcher": "~0.1.2",
"karma-sinon-chai": "~0.1.5"
I also dropped the old 'karma.conf.js' (along with a few other karma confs) in the project and updated the paths inside (from 'vendor' to 'bower_components'). I did find a 'ember-cli-karma' node mode and installed it, but it seems to just have a 'package.json'. It has no docs and seems like just a stubbed out starter project with no implementation. I also installed 'karma', 'karma-cli' and 'testem' node modules.
The testem docs say to add you src and test files to 'testem.json', but with out examples I don't know what that means; a list of every src and test file? With what path; relative, absolute? Forward slashes, backslashes? preceded with / or ./ or ../? I just left them out because I think the system just finds the src and tests by convention.
When I run 'karma init' I get:
readline.js:529
this.line = this.line.slice(this.cursor);
^
TypeError: Cannot read property 'slice' of undefined
When I run 'testem' I get:
TEST'EM 'SCRIPTS!
Open the URL below in a browser to connect.
http://localhost:7357/aN;0faN;NaNf
...then the project's '../tests/index.html' loads in a browser, but is not able to 'find' any of the asset files (css, js) so nothing executes or renders correctly. I just see template expressions ({{content-for 'head'}}, etc).
When I run 'ember test' I get:
Building...BuildingBuilding.Building..Building...Built project successfully.
1..0
# tests 0
# pass 0
# fail 0
# ok
No tests were run, please check whether any errors occurred in the page (ember test --server) and ensure that you have a test launcher (e.g. PhantomJS) enabled.
When I run 'ember test --server' I get:
The test index.html loaded in a browser with a test report. When I uncheck 'hide passed tests' the report indicates '29 passed, 28 failed'. It has 11 sections where a particular test may have 3 problems such as 'could not load', 'failed', 'could not find module', 'attempting to register an unknown factory' or 'died'.
With this, I'm obviously running testem and not karma, so may as well work on getting testem working and figure out karma later. If there were more examples and migration troubleshooting docs I might have a systematic way to work through some of these problems.
I ran into "No tests were run,..." problem recently after a node upgrade. I fixed it with a:
npm install -g phantomjs
This provides some additional options as well:
https://github.com/ember-cli/ember-cli/issues/3969
I had the Cannot read property 'slice' of undefined error on MS Windows, running via MSys2. I have solved it by using karma init from an ordinary cmd prompt.

my coffeescript file compiles but mocha gives an error

I have a project that uses "coffee-script": "^1.7.1" in its package.json.
The code has this line in it:
[{id: id, name: name}, ...] = result.rows
This compiles fine using coffeescript version 1.7.1
The problem is that I am trying to use mocha for unit tests and it gives me an error on this line:
Parse error on line xyz: Unexpected '...'
Apparently mocha uses an older coffeescript. Is there a way to make it work without adjusting the source for mocha?
EDIT:
my Gruntfile.coffee:
'use strict'
module.exports = ->
#initConfig
cafemocha:
src: ['test/*.coffee']
options:
reporter: 'spec'
ui: 'bdd'
coffee:
compile:
files:
'lib/mylib.js': ['src/*.coffee']
#loadNpmTasks 'grunt-cafe-mocha'
#loadNpmTasks 'grunt-contrib-coffee'
#registerTask 'default', ['coffee', 'cafemocha']
I added mocha.opts to the test directory:
--require coffee-script/register
--compilers coffee:coffee-script/register
--reporter spec
--ui bdd
but, still, when I run grunt, it gives me the same error. I am new to this environment, and I find it too complicated, please help.
Starting from version 1.7.x CoffeeScript compiler should be explicitly registered (see change log for version 1.7.0).
So, the problem is that CoffeeScript compiler is not registered when you're running your mocha tests, so node.js treats all your .coffee files as .js files.
The best possible solution is to specify --compilers option for your mocha tests:
--compilers coffee:coffee-script/register
If you don't want to include it to every mocha call, you could set it up using mocha.opts file.
Here are some useful links:
issue about it on github
reference in mocha docs
the reason behind this breaking change in CoffeeScript engine
Update
Looks like your issue is much deeper then I thought.
First, grunt-cafe-mocha doesn't respect mocha.opts because it's running tests by requireing mocha as a dependency, instead of calling mocha test runner.
So, it would've been enough to add require('coffee-script/register') to the top of your gruntfile, if not for this old grunt issue.
In short, grunt uses coffee-script 1.3.x, forcing all its tasks to use the same version of coffee. I had the same problem with grunt-contrib-connect, being unable to use latest coffee-script in my express app.
So, the only help I can offer you is a small grunt task I wrote to solve similar problem in one of my projects. It runs mocha in a separate child process, thus completely isolating it from grunt.
N.B. I had a thought about releasing this task to npm, but considered it too minor.

Translate F2PY compile steps into setup.py

I've inherited a Fortran 77 code which implements several subroutines which are run through a program block which requires a significant amount of user-input via an interactive command prompt every time the program is run. Since I'd like to automate running the code, I moved all the subroutines into a module and wrote a wrapper code through F2PY. Everything works fine after a 2-step compilation:
gfortran -c my_module.f90 -o my_module.o -ffixed-form
f2py -c my_module.o -m my_wrapper my_wrapper.f90
This ultimately creates three files: my_module.o, my_wrapper.o, my_module.mod, and my_wrapper.so. The my_wrapper.so is the module which I import into Python to access the legacy Fortran code.
My goal is to include this code to use in a larger package of scientific codes, which already has a setup.py using distutils to build a Cython module. Totally ignoring the Cython code for the moment, how am I supposed to translate the 2-step build into an extension in the setup.py? The closes I've been able to figure out looks like:
from numpy.distutils.core import setup, Extension
wrapper = Extension('my_wrapper', ['my_wrapper.f90', ])
setup(
libraries = [('my_module', dict(sources=['my_module.f90']],
extra_f90_compile_args=["-ffixed-form", ])))],
ext_modules = [wrapper, ]
)
This doesn't work, though. My compiler throws many warnings on the my_module.f90, but it still compiles (it throws no warnings if I use the compiler invocation above). When it tries to compile the wrapper though, it fails to find the my_module.mod, even though it is successfully created.
Any thoughts? I have a feeling I'm missing something trivial, but the documentation just doesn't seem fleshed out enough to indicate what it might be.
It might be a little late, but your problem is that you are not linking in my_module when building my_wrapper:
wrapper = Extension('my_wrapper', sources=['my_wrapper.f90'], libraries=['my_module'])
setup(
libraries = [('my_module', dict(sources=['my_module.f90'],
extra_f90_compile_args=["-ffixed-form"]))],
ext_modules = [wrapper]
)
If your only use of my_module is for my_wrapper, you could simply add it to the sources of my_wrapper:
wrapper = Extension('my_wrapper', sources=['my_wrapper.f90', 'my_module.f90'],
extra_f90_compile_args=["-ffixed-form"])
setup(
ext_modules = [wrapper]
)
Note that this will also export everything in my_module to Python, which you probably don't want.
I am dealing with such a two-layer library structure outside of Python, using cmake as the top level build system. I have it setup so that make python calls distutils to build the Python wrappers. The setup.pys can safely assume that all external libraries are already built and installed. This strategy is advantageous if one wants to have general-purpose libraries that are installed system-wide, and then wrapped for different applications such as Python, Matlab, Octave, IDL,..., which all have different ways to build extensions.