I have a gradle build, which runs a few tests on our application. Currently the tests that store assets in mongoDB fail if the developer forgets to run mongod first. So I want any build that uses mongoDB to fail with a message the user that clearly tells him to start mongoDB. Ideally, later we would start mongoDB from gradle.
I already found this nice article about how to see if mongoDB is running under Linux, which is quite simple. I am sure something similar can be done under Windows using tasklist /FI "IMAGENAME eq mongod", etc. But I need to know how to use this correctly in gradle.
Is there a cross platform way to check if a service or normal process is running in gradle?
The suggestion provided by Orid to use the Gradle Mongo Plugin should work if you set the necessary Gradle tasks to be dependent on a startManagedMongoDb task.
While that may seem to be the easiest way, it may be breaking with how MongoDb will be used in non-development environments or on a continuous integration build-server, where the MongoDb service will already be running.
A very simple solution would be to add the MongoDb checking functionality to the top of a customized gradlew.bat (and the gradlew bash script if it will be run on a *nix operating system).
Another simple solution that wouldn’t require changing the gradlew.bat script would be to create your own MongoDb checking script that then called gradlew.bat, passing on command line arguments. I’m not sure if there is an equivalent to the bash $# for all positional arguments in windows, but looping through the arguments with SHIFT %1 can be used to generate the gradlew.bat command line.
Related
I was wondering if there is a way to get Bazel to list, output, display, etc., all of the commands that can be executed from a command line that are run during a build after a clean. I do not care if the output is to the screen, in a file, etc. I will massage it into a usable form if necessary.
I have captured the screen output during a run of Bazel which gives me an idea of what is being done, however it does not give me a command I can execute on the command line. The command would have to include all of the command options and not display variables.
If this is not possible, since Bazel is open source, where in the code is/are the lines that represent the commands to be run so that I can modify Bazel to output the executable commands.
I am aware of the query command within Bazel, and used it generate the dependency diagram. If this could be done as a query command it would be even better.
TLDR;
My goal is to build TensorFlow using Bazel on Windows. Yes I know of all of the problems and reasons NOT to do it and have successfully installed TensorFlow on Windows via a Virtual Machine or Docker. I did take a shot at building Bazel on Windows starting with Cygwin, but that started to get out of hand as I am use to installing with packages and Cygwin doesn't play nice with packages, so then I started trying to build Bazel by hand and that was turning into a quagmire. So I am now trying to just build TensorFlow by hand on Windows by duplicating what Bazel would do to build TensorFlow on Linux.
You are correct, you can use the -s (--subcommands) option:
bazel build -s //foo
See https://docs.bazel.build/versions/master/user-manual.html#flag--subcommands.
For your use case, you'd probably want to redirect the output to a file and then global replace any library/binary paths to the Windows equivalents.
You might want to track https://github.com/bazelbuild/bazel/issues/276 (Windows support), although it'll probably be a while.
(Disclaimer: This solution does not print the commands that currently get executed but the commands that would get or got executed.)
I'd use aquery (action graph query) (forget about "graph"):
bazel aquery //foo
Advantages:
It's very fast, because it prints the actions without executing the build.
It's a query. It does not have side effects.
You don't have to do a bazel clean before in order to find out the build steps for a library that has already been built.
It prints information about the specific build step that you request. It does not print all the build commands required for the dependencies.
I've been working on a small set of command line programs in Scala. While
developing I used SBT, and tested the program with run within the console. At
this point the programs had a fast startup time (when re-run after initial compilation); nearly instant, even
with additional dependencies.
Now that I'm trying to actually utilize them on my system outside of sbt, the speeds have noticeable lag. I'm looking for ways to
reduce this, since the nature of these utilities requires little to no delay.
The best speeds I've achieved so far has been through utilizing Drip. I include all dependencies in a lib directory by utilizing Pack and then run by executing a shell script like this:
#!/bin/sh
SCRIPT=$(readlink -f "$0")
SCRIPT_PATH=$(dirname "$SCRIPT")
PROG_HOME=`cd "$SCRIPT_PATH/../" && pwd`
CLASSPATH_SUFFIX=""
# Path separator used in EXTRA_CLASSPATH
PSEP=":"
exec drip \
-cp "${PROG_HOME}/lib/*${CLASSPATH_SUFFIX}" \ # Add lib directory to classpath
TagWorkspace "$#" # TagWorkspace is the main class
This is still noticeably slower then invoking run from within SBT.
I'm curious as to why SBT is able to startup the application so much faster, and if there is someway for me to levarage its strategy, or SBT itself, even if that means keeping a long living process around to actually run a command through.
Unless you have forking turned on for your run task, this is likely due to VM startup time. When you run from inside an active SBT session, you have an already initialized VM pointing at your classes - all SBT needs to do is create a new ClassLoader and point it at your build output directory. This bypasses all of the other (not insignificant) stuff that happens when you fire up a new VM.
Have you tried using the client VM to start your utility from the command line? Sadly, this isn't an option with 64-bit Java, since Oracle apparently doesn't want to support it, but if you're using a 32-bit VM, try adding the -client argument to the list that you give the VM from the command line.
If you are using a 64-bit VM, some googling will find you some unofficial forks of OpenJDK that have the client VM re-enabled. It's really just a #define in the JVM build itself - it works fine once it's been compiled in.
The only slowness I have is launching SBT. Running a hello-word Scala app with java (no Drip) version 1.8 on a 7381 bogomips CPU takes only 0.2 seconds.
If you're not in that magnitude, I suspect your application startup requires loading thousands of classes, and creating instances of them.
My build script runs on linux and invokes things like gcc, shell scripts, etc.
Part of the solution is written in mono and could be compiled easily on linux.
But I want to obfuscate the code. Not manually, but as part of the build process.
Therefore I need to invoke Dotfuscator and Dotfuscator so far only runs on windows.
Is there a good solutions to invoke command line based workers/build scripts remotely from linux on a windows machine? I don't just want to run a command remotely, but also pass files along.
Like a windows service that is accessed using simple curl-uploads of a tar file, creates a temp folder for each concurrently connected client (or blocks concurrent calls) and unpacks the file, invokes something on these files and packages the result again as tar file to give it back to the caller? And clears the temp file even in case of failures?
Maybe someone knows a good solution that saves me from writing this myself!
It should not be so uncommon that a build process spans multiple platforms, yet common build server answers I found mainly talk about only one build script.
Also think about running e.g. the nsis setup builder from a linux driven build script, if part of your solution has a tiny windows component
I have downloaded a project which uses sbt as its build system and I want to build it. You'd think it would be as simple as typing "sbt" or something, but no.
I thought I'd add a question for this because it can take literally hours to figure this out on your own. I'm not joking.
tl;dr:
sbt compile
If you want to run it:
sbt run
To see what other targets are available:
sbt tasks
To get some (other) help, but mostly targeted at commands typed from the sbt console (ie, running sbt without parameters):
sbt help
This all assumes sbt version >= 0.10.0. To see what version of sbt is in use, do:
grep sbt.version project/build.properties
If there's no such file, and there's a file with extension ".sbt" on the base directory (not the project directory), then it's >= 0.10.0. Of course, if the grep works, it should tell you the version.
First, you'll want to use sbt-extras, because that automatically downloads and uses the right version of sbt. Trying to use the wrong version of sbt (newer or older than what the project you're trying to build says it requires) won't necessarily work, and may cause strange errors.
Run it:
~/path/to/sbt-extras/sbt
Wait for it to start up and download everything. If you need to use an authenticated proxy, you'll need to edit the script to specify the username and password for the proxy.
Check the version of Scala that sbt thinks it needs to build against (at the end of the output, if everything worked). If this is OK, fine, you don't need to do anything. If it isn't, you can temporarily specify a version explicitly with ++, e.g.:
++2.8.1
(If you want to make this permanent, you can edit the build definition files, but as that involves making a change to files under version control, that might not be what you want to do.)
Now, if you are using an older version of sbt, don't skip the next step! You could get strange errors if you do.
update
Now you can build and test what you've built:
test
If you get an error "Filename too long", this is not an sbt-specific problem, it's a scala problem, which most frequently affects Ubuntu users (technically, for Unbuntu users it's generally related to home directories encrypted with encfs). If you are using Scala >= 2.9, edit the build to use the scalac command-line option that allows you to specify a maximum filename length. Otherwise, if you are on Linux, you can redirect the build to /dev/shm or /tmp, by running these commands in a shell prompt (don't background sbt with CTRL+Z on Unix, because it may appear to stop working properly):
rm -rf target
ln -s /dev/shm target
(you may have to execute these commands in project/build instead or as well.)
Actually, it's probably better, and may even be more secure, to create a subdirectory of /dev/shm or /tmp and use that instead.
The compilation result should appear in target. You might then want to run it, if it's something you can run:
run
If everything looks OK, you can optionally publish the result locally so that the result can then be picked up automatically by other sbt builds:
publish-local
I don't think I could explain it better that the Getting Started Guide could. Please read the first 6 parts of it, which shouldn't too long time, to get it up and running.
I am using the Ruby Daemon-kit to setup a services that does various background operations for my Rails application.
It works fine when I call in on the commandline:
./bin/bgservice
How do I go about creating a daemon initd starter script for it, so that it will autostart on reboot?
There's a few approaches:
You could write /etc/init.d/ scripts that could be placed into the /etc/rc?.d/ directories (or wherever they live on your target distributions). Some details on this mechanism can be found in the Debian policy guidelines and openSUSE initscript tutorial. There's an annoying number of distribution-specific idiosyncrasies in initscripts, so don't feel about writing a simple one and asking distributions to contribute 'better' ones tailored for their environment. (For example, any Debian-derived distribution will provide the immensely useful start-stop-daemon(8) helper, but it is sorely missing from other distributions.)
You could write upstart job specifications for the distributions that support upstart (which I think is Ubuntu, Google ChromeOS, Fedora, .. more?). upstart documentation is still pretty weak, but there are some details and plenty of examples in /etc/init/ on Ubuntu, probably the same location in other distributions that use upstart. Getting the dependencies correct may be some work across all distributions, but upstart job specifications look far simpler to write and maintain than initscripts.
You could add lines to /etc/inittab on distributions that still support the standard SysV-init inittab(5) file. This would only be useful if your program doesn't do the usual daemon fork(2)/setsid(2)/fork(2) incantation, as init uses the pid it gets from fork(2) to determine if your program needs to be restarted.
Modern Vixie cron(8) supports a #reboot specifier in crontab(5) files. This can be used both by the system crontab as well as user crontabs, which might be nice if you just want to run the program as your usual login account.
As the author of daemon-kit I've avoided making any init-style scripts due to coping with the various distributions and they're migrations from old init-V style to newer upstart/insserv, saving myself a nightmare.
How I recommend to do this is to use the god config-generator, and ensure god is started on boot (by runit or some other means), and god starts the daemon up initially and keeps it running.
At best I'll expand daemon-kit to be able to generate runit scripts for boot...
HTH.