When deploying a Scala application, we use SBT on Jenkins. Currently our build action is specified as clean assembly (using Assembly plugin to produce fat JARs). Our build currently takes between 2-3 minutes, which is sensible, but as the project will become larger and deployments for frequent, it might become a bottleneck.
I remember when doing C++ deployment with Visual Studio, clean (Rebuild All) was necessary, otherwise builds were sometimes (say 0.1%) broken (most likely because build missed some changed dependency in headers).
Is this a concern with SBT? Is clean considered a necessary practice to get reliable builds?
My experience is that sometimes SBT gets mixed-up, the most common thing I've seen is that it can't find classes that are part of the project (and not compiled this time around). I haven't had the inclination to really debug it since doing a clean fixes it every time, but for a CI server I would go for clean every time.
From Gradle's documentation:
The scripts generated by this task are intended to be committed to
your version control system. This task also generates a small
gradle-wrapper.jar bootstrap JAR file and properties file which should
also be committed to your VCS. The scripts delegates to this JAR.
From: What should NOT be under source control?
I think generated files should not be in the VCS.
When are gradlew and gradle/gradle-wrapper.jar needed?
Why not store a gradle version in the build.gradle file?
Because the whole point of the gradle wrapper is to be able, without having ever installed gradle, and without even knowing how it works, where to download it from, which version, to clone the project from the VCS, to execute the gradlew script it contains, and to build the project without any additional step.
If all you had was a gradle version number in a build.gradle file, you would need a README explaining everyone that gradle version X must be downloaded from URL Y and installed, and you would have to do it every time the version is incremented.
Because the whole point of the Gradle wrapper is to be able, without having ever installed Gradle
Same argument goes for the JDK, do you want to commit that also? Do you also commit all your dependency libraries?
The dependencies should be upgraded continuously as new versions are released to get security and other bug fixes, and because if you get to far behind it can be a very time consuming task to get up to date again.
If the Gradle wrapper is incremented for every new release, and it is committed, the repo will grow very large. The problem is obvious when working with distributed VCS where a clone will download all versions of everything.
and without even knowing how it works
Create a build script that downloads the wrapper and uses it to build. Everyone does not need to know how the script works, they need to agree that the project is build by executing it.
where to download it from, which version
task wrapper(type: Wrapper) {
gradleVersion = 'X.X'
}
for Gradle version >= 5:
wrapper {
gradleVersion = 'X.X'
}
and then
gradle wrapper
to download the correct version.
to clone the project from the VCS, to execute the gradlew script it contains, and to build the project without any additional step.
Solved by the steps above. Downloading the Gradle wrapper is not different from downloading any other dependency. The script could be smart enough to check for any current Gradle wrapper and only download it if there is a new version.
If the developer has never used Gradle before and maybe doesn't know the project is built with Gradle, then it is more obvious to run a build.sh compared to running gradlew build.
If all you had was a gradle version number in a build.gradle file, you would need a README explaining everyone that gradle version X must be downloaded from URL Y an installed,
No, you would not need a README. You could have one, but we are developers and we should automate as much as possible. Creating a script is better.
and you would have to do it every time the version is incremented.
If the developers agree that the correct process is to:
Clone repo
Run build script
Then there upgrading to latest Gradle wrapper is no problem. If the version is incremented since last run, the script could download the new version.
I would like to recommend a simple approach.
In your project's README, document that an installation step is required, namely:
gradle wrapper --gradle-version 3.3
This works with Gradle 2.4 or higher. This creates a wrapper without requiring a dedicated task to be added to "build.gradle".
With this option, ignore (do not check in) these files/folders for version control:
./gradle
gradlew
gradlew.bat
The key benefit is that you don't have to check-in a downloaded file to source control. It costs one extra step on installation. I think it is worth it.
According to Gradle docs, adding gradle-wrapper.jar to VCS is expected as making Gradle Wrapper available to developers is part of the Gradle approach:
To make the Wrapper files available to other developers and execution environments you’ll need to check them into version control. All Wrapper files including the JAR file are very small in size. Adding the JAR file to version control is expected. Some organizations do not allow projects to submit binary files to version control. At the moment there are no alternative options to the approach.
What is the "project"?
Maybe there is a technical definition of this idiom that excludes build scripts. But if we accept this definition, then we must say your "project" is not all the things that you need to versioned!
But if we say "your project" is everything you have done. Then we can say you must include it and only it into VCS.
This is very theoretical and maybe not practical in case of our development works. So we change it to "your project is every file (or folder) you need to editing them directly".
"directly" means "not indirectly" and "indirectly" means by editing another file and then an effect will be reflected into this file.
So we reach the same that OP said (and is said here):
I think Generated files should not be in the VCS.
Yes. Because you haven't created them. So they are not part of "your project" according to the second definition.
What is the result about these files:
build.gradle: Yes. We need to edit it. Our works should be versioned.
Note: There is no difference where you edit it. Whether in your text editor environment or in Project Structure GUI environment. Anyway you doing it directly!
gradle-wrapper.properties: Yes. We need to at least determine Gradle version in this file.
gradle-wrapper.jar and gradlew[.bat]: I haven't created or edited them in any of my development works, till this moment! So the answer is "No". If you have done so, the answer is "Yes" about you at that work (and about the same file you edited).
The important note about the latest case is the user who clones your repo, needs to execute this command on repo's <root-directory> to auto-generate wrapper files:
> gradle wrapper --gradle-version=$v --distribution-type=$distType
$v and $distType are determined from gradle-wrapper.properties:
distributionUrl=https\://services.gradle.org/distributions/gradle-{$v}-{$distType}.zip
See https://gradle.org/install/ for more information.
gradle executable is bin/gradle[.bat] in local distribution. It's not required that local distribution be same as that determined in the repo. After wrapper files created then gradlew[.bat] can download determined Gradle distribution automatically (if not exists locally). Then he/she probably must regenerate wrapper files using new gradle executable (in downloaded distribution) using above instructions.
Note: In the above instructions, supposed the user has at least one Gradle distribution locally (e.g. ~/.gradle/wrapper/dists/gradle-4.10-bin/bg6py687nqv2mbe6e1hdtk57h/gradle-4.10). It covers almost all real cases. But what happens if the user hasn't any distribution already?
He/She can download it manually using the URL in .properties file. But if he/she doesn't locate it in the path that the wrapper expected, the wrapper will download it again! The expected path is completely predictable but is out of the subject (see here for the most complex part).
There are also some easier (but dirty) ways. For example, he/she can copy wrapper files (except .properties file) from any other local/remote repository to his/her repository and then run gradlew on his/her repository. It will automatically download the suitable distribution.
Old question, fresh answer. If you don't upgrade gradle often (most of us don't), it's better to commit it to VCS. And the main reason for me is to increase the build speed on the CI server. Nowadays, most of the projects are getting built and installed by CI servers, different server instance every time.
If you don't commit it, CI server will download a jar for every build and it significantly increases a build time. There are other ways to handle this problem, but I find this one the easiest to maintain.
I have some data resources that I would like eclipse to not copy every time it builds. I put them as part of the build path so they get copied, but I don't want that to happen every time as it's time consuming.
Any idea on a better strategy?
I don't even want them to be deleted when clean is invoked. In Visual Studio one can mark a resource file as "copy once" is there such a thing in Eclipse.
Thank You
Having eclipse not copy some file that you have modified even after a clean will be a nightmare: you'll have to remember to copy it manually each time it's modified.
And Eclipse only copies files which have been modified when building incrementally.
If it's so slow, it probably means you have too many such files, and they should perhaps be put in a jar in the build path.
As you ask for a strategy, then ...
Eclipse is not a build tool, it's IDE. So you better not try to setup some build logic based on it. Use eclipse for coding and for performing specific tasks during build use build tools like maven or ant.
Using TeamCity 6.5, I am trying to figure out how to setup a manual deployment for a specific build run if it's possible.
What I would like to be able to do is to take an already built and tested TeamCity run (only the artifacts needs to be deployed - this is not a web application or site) and call an MSBuild step to publish the artifacts to somewhere else.
You can do what you want by setting up Artifact Dependency between the configurations where you want to do the manual deployment and the one where you have the built artifacts.
Once you have setup the Artifacts dependency, click on the Run custom build ellipsis near the "Run" button for the configuration. Here you will have the Artifacts dependencies part where it will say the configuration that this configuration you are running is dependent on and will also have a dropdown list from which you can choose the particular version of the other configuration from which to get the artifacts. Click run from here to run your custom build.
See here for more details: http://confluence.jetbrains.net/display/TCD65/Triggering+a+Custom+Build
You might be thinking about this a bit backwards. What you probably want is a build configuration that takes the previously known successful build (in TC terms it has a snapshot dependency) and then runs a different build targeted at dropping the artifacts somewhere. Pretty easily done by switching the output directories in MSBuild.
The most "integrated" way I could think to do it would be to add a dependency to your deployment configuration that depends on the latest pinned build for the dependent configuration. Then you just unpin any newer builds in the dependent configuration and pin the one you want and run the deploy...This is a bit kludgy and might not work very well if you depend on pinned builds for anything else in the dependent configuration.
The other built in way to do with would be to add an artifact dependency using a specific build number. The drawback of this method is that any time you want to deploy a different build, you will need to be able to edit the artifact dependency build number by hand and then hit run.
I have a large codebase added into the Eclipse project and have added one External Tool providing the path where the class for that java file is to be kept and the Classpath. The build folder is somewhere else.
Now when I need to compile only one file, Eclipse starts building whole of the codebase(>100 MB of Java files), it takes my system down and I have to wait for the whole compilation to go through.
Can only one java file be compiled without building the whole code?
Any pointers would be helpful.
I think you would benefit greatly with a build tool, such as ant or maven. You can have a target to compile only one class without building the whole code, or any other related task.
Have you built the project at least once ?
Eclipse builds incrementally using the saved state from the previous build. So if you build your project once, the subsequent builds would only pick up the changes made "after" the previous build.