I am currently in a process of setting up Jenkins-CI for my Scala/Akka project.
I managed to create build that is integrated with BitBucket and executes builds when new pull reuqest is created/old pull request is updated.
As a testing framework we are using Specs2 which is also integrated with Jenkins by using JUnit post-build action. Now I am wondering how to properly execute e2e tests in my build.
Basically in git repository we have 2 projects, lets call them main-project and rest-tests. rest-tests contains e2e tests that are written using REST-assured library. To execute them I need to start main-project application (which uses Spray library to set up HTTP server) and then execute test task in sbt project of rest-test.
My idea was to execute main-project startup script (generated by sbt-native-packager) whith something like this:
$WORKSPACE/main-project/target/universal/stage/bin/main-project & echo $! > /tmp/main-project.pid
then execute test task of rest-tests project and finally kill process with PID that is saved in /tmp/main-project.pid file.
The last step should be implemented using https://wiki.jenkins-ci.org/display/JENKINS/Post+build+task because if some rest-testswill fail the next steps of build will not be executed (or at least that is what I am thinking) and I could end with my instance of application running after the build is finished.
This is first time when I am setting up CI system and my solution seems to be a little hacky (at least to me). I am wondering if there is a better/more idiomatic way of solving my problem of running e2e tests which require another application running.
Related
Is it possible to force one or more tests in a test suite to run separately and serially in dart/flutter?
Note: I'm not looking to run a single test via CLI filters. I want to be able to run my full test suite as part of my CI flow.
Example
The test runner ava is able to do this in javascript via the serial modifier:
test.serial('passes serially', t => {
t.pass();
});
https://github.com/avajs/ava/blob/main/docs/01-writing-tests.md#running-tests-serially
Context
I'm using a third-party library to communicate and authenticate with my backend server. This library makes use of a singleton to make accessing the current user "easier". However, it makes integration testing in parallel impossible since the testing process can only mimic a single user at a time, which in turn makes tests interfere with each other.
If you run "flutter test" in your CI all your tests in the project will be run in serial. No need to do something extra for that.
I have a TeamCity build configuration that builds a C# project, runs some unit tests, and then does some extra things. My question is: Can I get information about my unit test run stored into build configuration variables (i.e. how many tests were run, how many were successful, how many failed, how many were skipped) so that I can then check these variables in a PowerShell script in later build steps and perform different actions depending on how many tests have passed?
AFAIK the best way is to ask these information directly to teamcity server using its REST API (pay attention, maybe the build locator could be a little be tricly to be found, if the build is still running).
By other hand, you can parse your NUnit test result file (or files if you run more than one NUnit test runner step in your build) inside your build agent machine.
I have a batch file which uses nunitconsole runner to run my tests.
It's a combination of nunit framework and specflow.
When I run the batch file on the Octopus tentacle it works as expected.
However, if I create a step in the Octopus server to trigger this batch file, none of the UI related things happen.
Any setting or prerequisite is required for this?
Some time ago I had a similar situation, this is because Octopus server runs as a service, due to this, you can not have anything that requires a window or interaction to run, however a workaround could be, execute a scheduled task through Octopus to execute your batch, so you will execute in a non background way.
I am finishing a Continuous Integration system with Jenkins and Gradle for a REST service. It will build the App and dependent sub-libraries, build a Docker, start main docker and secondary ones (database, ...) all in Gradle.
As it is a REST service I have a separate project that executes the REST tests completely from outside my project just as it is a REST client, and works ok...
Once my project is built and everything running I need to execute the build in the other project (which is just for tests) as a subproject, and wether it passes or not the tests I want to continue the main script as Dockers need to be stopped and deleted. What is the best approach for this?
Thanks
You just need to create a task with type: GradleBuild in parameter
Example:
task buildAnotherProjectTask(type: GradleBuild) {
buildFile = '../pathToBuildFileInTheOtherProject/build.gradle'
tasks = ['build'] // You can run any task like that
tasks = ['test']
}
and to run it u can use the following command
gradle buildAnotherProjectTask
This is worked with me when i tried it.
Hope my answer will help :)
It would be great to improve test driven development productivity by automatically firing of tests whenever there is a code change.
This is what I'm hoping for.
Whenever a Scala file is saved, SBT (or a shell script) should execute the ScalaTest or Specs2 specifications.
If the tests complete, the system should play a sound indicating success or failure
I'm using Scala IDE to do development and SBT to run my test specs at the moment, so just autimating the steps above would save a person from switching to the console, running the test specs and waiting for the result.
Any ideas automatically firing of the tests and playing a 'succeed' or 'fail' sound would be great.
Why not create a custom SBT task that depends on the test task. You could add the code for playing the sound to your build definition.
See here how to run a custom task after another task.
To automatically re-run tests, simply prefix the newly define task with a ~ in the SBT shell before running it.