pipenv uses click, and uses click's invoke method in its unit test suite to test itself.
This does not work great when unit testing the pipenv run command, which internally uses execve. In the context of a single threaded test run this means our test runner (pytest) process gets completely replaced by whatever we're exec-ing, and the test suite immediately and silently exits.
(This bug on the pipenv issue tracker talks about this in more detail: https://github.com/pypa/pipenv/issues/4909)
Is there a best practice for how to handle situations like this with click? Should we not be using click's invoke at all, and instead using regular old subprocess? Or perhaps invoke could optionally run the command in a subprocess rather than the current process? Or maybe there's a best practice for cli apps that use click to detect if they're being run via invoke and just be careful not to do anything in the exec family (this is basically what pipenv does right now: if the CI env var is set, then pipenv run triggers a subprocess rather than using execve).
Related
We are using pytest-xdist to run pytest tests with the --forked flag (we are writing integration tests for code that uses Scrapy). However, we noticed that when the tests finish running, some of the created child processes remain alive, which causes our GitLab build to hang.
The Python version we'e using is 3.7.9.
I couldn't find other mentions of the issue online. Is anyone familiar with it? Are there any solutions/fixes/workarounds?
I am running OpenDDS MPC based example stockQuoter. I deleted the run_test.pl, still the project builds and runs properly. why do we need this Perl script?
You don't really need it and you're free to start the programs directly. All examples and tests in OpenDDS have a file called run_test.pl for the purposes of testing. Among other functions, they define what programs get called with what arguments for a certain test scenario and are responsible for killing the programs if they get stuck.
I have a gradle build, which runs a few tests on our application. Currently the tests that store assets in mongoDB fail if the developer forgets to run mongod first. So I want any build that uses mongoDB to fail with a message the user that clearly tells him to start mongoDB. Ideally, later we would start mongoDB from gradle.
I already found this nice article about how to see if mongoDB is running under Linux, which is quite simple. I am sure something similar can be done under Windows using tasklist /FI "IMAGENAME eq mongod", etc. But I need to know how to use this correctly in gradle.
Is there a cross platform way to check if a service or normal process is running in gradle?
The suggestion provided by Orid to use the Gradle Mongo Plugin should work if you set the necessary Gradle tasks to be dependent on a startManagedMongoDb task.
While that may seem to be the easiest way, it may be breaking with how MongoDb will be used in non-development environments or on a continuous integration build-server, where the MongoDb service will already be running.
A very simple solution would be to add the MongoDb checking functionality to the top of a customized gradlew.bat (and the gradlew bash script if it will be run on a *nix operating system).
Another simple solution that wouldn’t require changing the gradlew.bat script would be to create your own MongoDb checking script that then called gradlew.bat, passing on command line arguments. I’m not sure if there is an equivalent to the bash $# for all positional arguments in windows, but looping through the arguments with SHIFT %1 can be used to generate the gradlew.bat command line.
I've been working on a small set of command line programs in Scala. While
developing I used SBT, and tested the program with run within the console. At
this point the programs had a fast startup time (when re-run after initial compilation); nearly instant, even
with additional dependencies.
Now that I'm trying to actually utilize them on my system outside of sbt, the speeds have noticeable lag. I'm looking for ways to
reduce this, since the nature of these utilities requires little to no delay.
The best speeds I've achieved so far has been through utilizing Drip. I include all dependencies in a lib directory by utilizing Pack and then run by executing a shell script like this:
#!/bin/sh
SCRIPT=$(readlink -f "$0")
SCRIPT_PATH=$(dirname "$SCRIPT")
PROG_HOME=`cd "$SCRIPT_PATH/../" && pwd`
CLASSPATH_SUFFIX=""
# Path separator used in EXTRA_CLASSPATH
PSEP=":"
exec drip \
-cp "${PROG_HOME}/lib/*${CLASSPATH_SUFFIX}" \ # Add lib directory to classpath
TagWorkspace "$#" # TagWorkspace is the main class
This is still noticeably slower then invoking run from within SBT.
I'm curious as to why SBT is able to startup the application so much faster, and if there is someway for me to levarage its strategy, or SBT itself, even if that means keeping a long living process around to actually run a command through.
Unless you have forking turned on for your run task, this is likely due to VM startup time. When you run from inside an active SBT session, you have an already initialized VM pointing at your classes - all SBT needs to do is create a new ClassLoader and point it at your build output directory. This bypasses all of the other (not insignificant) stuff that happens when you fire up a new VM.
Have you tried using the client VM to start your utility from the command line? Sadly, this isn't an option with 64-bit Java, since Oracle apparently doesn't want to support it, but if you're using a 32-bit VM, try adding the -client argument to the list that you give the VM from the command line.
If you are using a 64-bit VM, some googling will find you some unofficial forks of OpenJDK that have the client VM re-enabled. It's really just a #define in the JVM build itself - it works fine once it's been compiled in.
The only slowness I have is launching SBT. Running a hello-word Scala app with java (no Drip) version 1.8 on a 7381 bogomips CPU takes only 0.2 seconds.
If you're not in that magnitude, I suspect your application startup requires loading thousands of classes, and creating instances of them.
My build script runs on linux and invokes things like gcc, shell scripts, etc.
Part of the solution is written in mono and could be compiled easily on linux.
But I want to obfuscate the code. Not manually, but as part of the build process.
Therefore I need to invoke Dotfuscator and Dotfuscator so far only runs on windows.
Is there a good solutions to invoke command line based workers/build scripts remotely from linux on a windows machine? I don't just want to run a command remotely, but also pass files along.
Like a windows service that is accessed using simple curl-uploads of a tar file, creates a temp folder for each concurrently connected client (or blocks concurrent calls) and unpacks the file, invokes something on these files and packages the result again as tar file to give it back to the caller? And clears the temp file even in case of failures?
Maybe someone knows a good solution that saves me from writing this myself!
It should not be so uncommon that a build process spans multiple platforms, yet common build server answers I found mainly talk about only one build script.
Also think about running e.g. the nsis setup builder from a linux driven build script, if part of your solution has a tiny windows component