I used to be developer long ago but for last 10 years working on system ops. I am planning to move into devops and trying to sharpen my saw. However, when it comes to jenkins and specially static code analysis, code coverage, automated test and code review, I get so much confused.
Lets start from automated test ( for simplicity take unit test). I understand that we write a separate class file for unit test. But how does that test is carried out? Will jenkins create a jvm where the newly build artifact is deployed and the tests are run against it? or will the test be run against code ( I do not think but still want to clarify)?
I downloaded one example application with maven and codertura from github and build the project. When the build was completed, it publishes code coverage report.
I have not done any post build, for deploying the artifact. So, I am not sure how it works, and what did it do and how?
Thanks
J
Here is a common flow that you can follow to achieve your requirement.
Work with code --> Push to gerrit for review --> Jenkins gerrit trigger plugin get triggered --> The corresponding job will checkout the code you committed and do the compile, package, unit test, deploy to artifactory --> Execute the sonar build to analysis the code quality, static analysis, code coverage...
Br,
Tim
Related
We have a web application in an Azure DevOps repo and there's a branch policy on the master branch that kicks off a build when a pull request is created. This validates that it compiles and performs code quality checks and the like.
We also have some integration tests (using Mocha and Selenium) that live in another repo. I would like to run the integration tests when a PR against master is created.
As far as I know I cannot have the same build pull from two different repos (without using extensions and it seems cleaner to me to have two separe builds anyway). So I thought I would have another build just to run the integration tests. The build that pulls from the webapp repo would have a final step where it would deploy to an integration tests environment and then the second build would get the latest version of the integration tests and run them against the integration tests environment. I created a Build Completion trigger on the integration tests build that is triggered by the completion of the webapp build.
The problem is that when I queue the webapp build manually, it will launch the integration tests build when done. But when the webapp build is queued by an incoming PR, the integration tests build does not get triggered.
Is this a bug in Azure DevOps or am I going about this wrong?
Also in my side builds from PR doesn't trigger another builds (with Build Completion trigger), I don't know if it's a bug or it's by design.
Anyway, there is a workaround - the final step in the first build will trigger the second build. how? with Trigger Build task.
You just need to change the branch because it will be a merge branch from the PR that doesn't exist in the tests repository:
You can also do it without install extensions with PowerShell task and the Rest API.
I'm having a hard time figuring out how to correctly deploy to different environments with TeamCity (in terms of cross BuildConfiguration dependencies) and hope to get some input as to how to configure my SubProjects/BuildConfigurations properly. Lets start based on a concrete example: I made this test "TeamcityConfigurationTests" to better learn how TeamCity handled dependencies, and the current state shows the result i am looking for:
I have 3 subProjects, Dev, Test and Prod - and all associated tasks for those "environments" as seperate build configurations within that subProject. This is to more clearly visualize what is going on, and if anything breaks, to be able to see immediately what is broken (separate Build, UnitTest and DeployToDev BuildConfigurations, rather than 3 different steps in one single Build Configuration).
Ideally, i only want to build my application once in the Dev.Build step, and let the Dev.UnitTest and Dev.DeployToDev steps grab that artifact and run tests and deploy. That i got going for me, by having snapshot and artifact dependencies. But i am having trouble getting the correct artifact when i want to deploy from Dev -> Test or Test -> Prod.
My issue is to correctly reference the latest successfully DEV deployed artifact when running Test.DeployToTest - and the same for getting the latest successfully TEST deployed artifact when running Prod.DeployToProd. (Essentially i want to promote the artifact to the next environment).
Now, my issue is, if I in the Test.DeployToTest have a SnapshotDependency to Dev.DeployToDev and an artifact dependency to Dev.Build, and the VCS source has changed since Deploy to Dev has run, it triggeres running all the DEV steps again. This is not the worst part, the same happens when i run Prod.DeployToProd if the VCS source changed since the initial build on dev (because of all the snapshot dependencies). Meaning, that rather than promoting Test -> Prod, I Build and deploy whatever is currently on VCS to Dev, Test AND Prod.
How am i supposed to set this up correctly?
The only other option i am aware of, is letting Dev.DeployToDev also publish the same artifact, and only have an (LatestSuccessful) ArtifactDependency in Test.DeployToTest. I would also have to publish the artifact again in Test.DeployToTest, for letting Prod.DeployToProd only have a (LatestSuccessfull) artifact dependency to Test.DeployToTest. (This would be to get rid of the SnapshotDependencies causing previous environments to run build/deploy again in case of VCS changes). But then i am publishing the artifact 3 times, rather than just the one time when the application is originally built in DEV - which i would like to avoid. Also, i have cases where no artifact is needed for deploying to Test and Prod, so there is no artifact to depend on (essentially i only need the BuildNumber from the "Dependent" environment i want to promote from).
I hope for some input. Thank you
Regards
Frederik
For anyone wondering, i made a JetBrains support ticket, and got the following response:
Basically, there are options to resolve your case:
Option 1: use "Promote" action form the build's Actions top-right menu
(or change the type of the Deploy* configurations to deployment and
use the action from the block on the build results. This is the
preferred way: before deploying you select the build to deploy and
"promote" it to the next environment. There is also an experimental
hidden feature to hide the "Run" button: add
"teamcity.ui.runButton.caption" configuration parameter in the build
configurations to empty value.
Option 2: do not use snapshot dependency, use only artifact dependency
on the latest successful build. However, when you run the build you
cannot be sure that the last successful build you see will be
deployed: while the build is standing int he queue, another
Dev.DeployToDev can finish and then be deployed as the last
successful.
We went with option 1
Currently all our Regression tests are configured in a Jenkins job.We want once the regression tests are completed,it should trigger a plan in our Bamboo server and also record the tests results in Bamboo using TestNG parser.Is it possible?
Ps : I have already seen Bamboo rest-api but can not seem to find a solution.Any suggestions will be highly appreciated.Thanks
It hasn't been updated for a while and I'm not using it currently, thus can't confirm it is still working as desired (the download statistics suggest it being used though), but given there's not much to it, you should be able to achieve the first part of your use case with the Bamboo Notifier, which allows you to Trigger a Bamboo build upon successful completion of a Jenkins job.
The second part should be covered by the Bamboo TestNG Parser task, though you'll need to push your existing test result files to Bamboo by some means of course, possibly by Using the SCP task in Bamboo.
My company recently started getting up to date on the usage of TFS, source control, and branching strategies. Our current branching strategy is the basic 'Dev > Main > Release' method, which works well for our small team. However, the issue lies with our automated tests. All of our integration tests and UI tests are written in C# and executed in a nightly build process. In effort to keep source clean and well kept, where exactly should we place the automated test code?
You could place the automated tests inside of each branch. Your automated tests can be merged and treated just like regular code since it will be changing for new development.
The other option could be for you to place it where your projects build-types and build files reside.
It is important to make sure it is checked into source control.
In a CI environment, what exactly is considered a broken build?
There are several answers I can imagine (any combination of compiles, tests pass, metrics are in range, documentation exists etc.) , but I am not sure which of these are cannoncial.
For example, just today it happened to me that I actually checked in all code changes but forgot to commit the Visual Studio project file, thus breaking the unit tests. (even though I literally triple checked my commit, as it's a public OSS project on google code).
I was easily able to fix this in under a minute after my first commit, but should I consider myself a buildbreaker now?
How do you configure your CI environment: Is every revision built or only the newest version after each complete build, or do you use time based checking for new revs?
Ideally, you have
automated script that is scheduled to run each night to build the application from source code.
Scripts to copy the binaries to a directory/set of directories from which another script can be run to deploy the application if it is running in your environment, or used to create deliverables for a customer.
Automated suite of tests that run and verify all components pass all tests.
Automated script that verifies that the build has been built correctly.
Automated script/monitoring system that sends out an alert if the verification script fails.
When an alert is generated by the above process, then that is considered "breaking the build".
But since procedures/processes can vary from company to company, there may be alternate definitions. In some places it may be breaking the unit tests. Others it may be checking in source code that results in the code being unabl to compile.
Breaking the build is commiting any changes that make it impossible (or possible but not smart to do) to deploy the project.
Fixing a broken build does not repair the broken build, but makes it possible to create a new non-broken build.
I configure my CI-server to create a minimal build on every commit, and create a maximum build every period of time. The period depends on the number of people working on the project (more people is more commits) and the build duration (You can run your unit test suite every time, but the 30min. acceptance test suite once or twice a day).
Breaking a build is preventing any user that relies on the same standard tools and set of code than your CI environnment to get a compiling and running system.
If your coworkers cannot compile the system when they're up to date, because some config is missing, the build is broken, isn't it ?
If you coworkers cannot be confident that unit tests pass, because one of them is flaky, the build is broken, isn'it ?
If you have automated performance tests, and you're project have to be optimized, I would go as far as too say that if you code doesn't run fast enough, you've broken the build (but that is arguable)
I would not be so strongly minded about code coverage, or other metrics for example.
Breaking the build can happen. CI is just there to make sure it doesn't happen too much on the day you're supposed to ship ;)
for us, we use the term "breaking the build" any time the test suite fails after a new commit.
so, in your instance, yes you would have broken the build (according to our company at least)
As long as you broke the build because of a simple human error (like forgetting to commit something) and as long as this is an exception and not the rule, I would say it's OK. As long as you take care to fix this quickly :)
On the other hand, if somebody breaks builds on a regular basis by not executing the complete build locally on his/her machine before committing, then this is a sign of an ill-disciplined team member who doesn't really care for other team members and for the development process.
My experience is that one effective way to make people be more careful is to set up your CI server to send an email when (and only when) the status of the build changes, with the "culprit" on the To: list and the rest of the team on the Cc:. I guess you can could call it a "shame factor" ;)
A broken build is anything that doesn't pass the automated test suite.
You're a build breaker. ;) That's not really a big deal as long as you fix it fast. The whole point of a CI environment is to catch bugs, not make people afraid to commit.
My company builds the tip of each branch that is either in production or next up for production. We do this on every commit and each day at 4am. We're using Mercurial, so commit here means pushing changes to the integration repo.