How to cherry pick ZIP contents with Gradle Distribution plugin? - plugins

My Gradle build currently produces the following directory structure under a build dir in my project root:
myapp/
src/
build.gradle
build/
docs/
groovydoc/* (all Groovydocs)
libs/
myapp-SNAPSHOT.jar
myapp-SNAPSHOT-sources.jar
reports/
codenarc/
main.html
test-results/* (JUnit test results)
I would like to add the distribution plugin (or anything that accomplishes my goals, really) to have Gradle produce a ZIP file with the following directory structure:
myapp-SNAPSHOT-buildreport.zip/
tests/
(JUnit tests from build/test-results above)
reports/
main.html (CodeNarc report from build/reports/codenarc above)
api/
(Groovydocs from build/docs above)
source/
myapp-SNAPSHOT-sources.jar (from build/libs above)
bin/
myapp-SNAPSHOT.jar (from build/libs above)
After reading the plugin's documentation, I can't tell how to configure it to suit these needs. Its obvious that I need to run gradle distZip, but as to how to actually configure it to produce the desired directory structure, it doesn't seem to provide any documentation/examples. Any ideas?
Note: The JAR's version is obviously SNAPSHOT, and is passed into the Gradle build with a -Pversion=SNAPSHOT command-line argument.

The Gradle Distribution plugin automatically has defaults (the problem is that the docs do not tell us the defaults, but the Gradle project default structure is what is assumed) so if your Gradle project is fairly straightforward and already using src/main/groovy or src/main/java, you typically just need to...
Use the CopySpec reversing pattern of giving your into{} (makes a dir) containing the contents of from{} , rather than the reverse, like so:
apply plugin: 'groovy'
apply plugin: 'eclipse'
apply plugin: 'application'
distributions {
main {
baseName= 'vz_sde_dm'
contents {
into('bin/config') {
from 'config'
}
into('lib/samples') {
from 'samples'
}
}
}
}
Notice I did not need to define a from{} for my contents{}, but only into{}s that's because I am already using the default Gradle Groovy project layout and only added 2 extra folders (config & samples) under my project in Eclipse but needed those 2 folders to land into a slightly different hierarchy for my Distribution compared to my regular build folder layout.

I would probably not use the distribution plugin and instead just create a new custom Zip task. It would look something like this:
task buildreportZip(type: Zip, dependsOn: build) {
classifier = 'buildreport'
from('build/test-results') {
into 'tests'
}
from('build/reports/codenarc') {
into 'reports'
}
from('build/docs') {
into 'api'
}
from(sourcesJar) { // or whatever you source jar task name is
into 'source'
}
from(jar) {
into 'bin'
}
}

I was trying to make a custom layout also, and had real trouble figuring out how to exclude the project output from build/libs from the yourProject.zip/yourProject/lib directory (and excluding things in general) and putting it instead into yourProject.zip/yourProject.
After quite a few hours across multiple days of searching and poking around in the API I finally found something that worked using actual configurations of the Distribution and underlying CopySpec (documented here and here, respectively for Gradle 5.6.1, you can just replace 5.6.1 with current in the URL to get the most recent API docs, 5.6.1 just happens to be the version I'm using):
distributions {
main {
baseName = appName
contents {
filesMatching("**/${appName}.jar", {
if (it.getPath().contains('/lib/')) {
it.setPath(it.getPath().replace('lib/', ''))
}
})
into('config') {
exclude(['server.crt', 'spotbugs-exclusion-filters.xml'])
from 'src/main/resources'
}
}
}
}
For exclusions the only thing that worked was matching on a glob pattern and specifying the correct action (to copy it to the root dist directory instead of root/lib) with a Closure through the filesMatching method of the main distribution's content CopySpec. You can see also how destination for configs is changed from the root to the root/config directory. Thanks so Thad's answer for helping to guide me to the correct build configuration, also.

Related

Gradle DSL - Eclipse Equivalent for IDEA Module Property

Good localtime,
I am in the process of updating legacy (4.8.1) Gradle build files for a big-McLarge-huge, multimodule project. We utilize an intellij.gradle file which has the following line (marked by comment):
idea {
module {
inheritOutputDirs = true // <-- HOW DO I DO THIS
downloadJavadoc = true
downloadSources = true
}
workspace.iws.withXml { provider ->
def node = provider.asNode()
def dynamicClasspath = node.component.find { it."#name" == "dynamic.classpath" }
if (dynamicClasspath != null) {
dynamicClasspath."#value" = "true"
}
}
From the 4.8.1 DSL docs:
If true, output directories for this module will be located below the
output directory for the project; otherwise, they will be set to the
directories specified by IdeaModule.getOutputDir() and
IdeaModule.getTestOutputDir().
Any ideas on what the Eclipse DSL equivalent of inheritOutputDirs? Should this be handled using the eclipseClasspath API? Right now everything is building fine, but the Eclipse Java builder is is flagging things.
References:
https://docs.gradle.org/4.8.1/dsl/org.gradle.plugins.ide.idea.model.IdeaModule.html
https://docs.gradle.org/4.8.1/dsl/org.gradle.plugins.ide.eclipse.model.EclipseClasspath.html
Usually this would have been picked up through sourceSets but I can't see what your project looks like...
If your subproject uses Gradle to generate sources into /build/cxf/generated-sources directory, then you can tell Eclipse via Gradle DSL to include that as a source folder like this:
plugins { id 'eclipse' }
eclipse.classpath.file.whenMerged {
// this is the brute-force approach; there is likely a better way to add a source folder
entries << new org.gradle.plugins.ide.eclipse.model.SourceFolder('build/cxf/generated-sources', null)
}
Once this is run (via gradle eclipseClasspath) you should see a build/cxf/generated-sources folder under your project node in the Package Explorer or Project Explorer. Sort of like this:
NOTE: This is untested because I don not have a sample project to work with.
There is more discussion here: How to add gradle generated source folder to Eclipse project?

Running Vert.x (w/ES4X) via Eclipse

This question is a follow-up to:
Running Vuetify on Vert.x (w/ES4X)
I would like to be able to run ES4X via Eclipse (instead of NPM). I'm not exactly sure if it's possible or how to wire it in.
Let's say I have the following build.gradle.file
plugins {
id 'java'
id 'application'
id 'com.johnrengleman.shadow' version "5.0.0"
}
sourceCompatibility='1.8'
mainClassName='io.vertx.core.Launcher'
repositories {
mavenCentral()
}
dependencies {
implementation 'io.vertx:vertx-core:3.7.1'
implementation 'io.vertx:vertx-web:3.7.1'
implementation 'io.vertx:vertx-lang-js:3.7.1'
// implementation 'io.reactiverse:es4x:0.8.0'
// implementation 'io.reactiverse:es4x-pm:0.8.0'
}
processResources {
from '/src/main/js'
}
shadowJar {
classifier = 'fat'
manifest {
attributes 'Main-Verticle' : 'index.js'
}
mergeServiceFiles {
include 'META-INF/services/io.vertx.core.spi.VerticleFactory'
}
}
and my src/main/js/index.js looks like the one from the other referenced post:
import { Router, StaticHandler } from '#vertx/web';
const app = Router.router(vertx);
app.get().handler(StaticHandler.create("dist"));
vertx.createHttpServer().requestHandler(app).listen(8080);
If I create an executable jar via shadowJar, I get javax.script.ScriptExceptions due to Nashorn choking on the index.js contents (as expected).
If I uncomment the es4x implementation in the gradle build, I get ClassNotFound exceptions for org.graalvm.polyglot.io.FileSystem
So how would I correctly modify this app to take advantage of ES4X? I guess the equivalent of what the 'es4x init' would do?
Say that you have your es4x application and a package.json, when you execute:
npm install
You will get inside node_modules a few extra folders:
.bin
.lib
In the .bin directory there is a es4x-launcher.jar file you can use to start your application from Eclipse. It will refer to the dependencies which are unpacked to the .lib dir. In order to make things work fine the same JVM you have when running the npm install command should be used in eclipse. Otherwise you might end up missing dependencies. This is the case when running graalvm which will not require graaljs dependencies or when running on jdk8 which will not require jvmci dependencies.

Intellij IDEA - Output path ...\project\target\idea-classes intersects with a source root. Only files that were created by build will be cleaned

Using Intellij IDEA with Scala plugin.
When doing a Build -> Rebuild Project I get the following make warnings:
Output path ProjectRootFolder\project\target\idea-test-classes intersects with a source root. Only files that were created by build will be cleaned.
Output path ProjectRootFolder\project\target\idea-classes intersects with a source root. Only files that were created by build will be cleaned.
The project was generated with SBT gen-idea plugin.
The two output paths mentioned in the warnings are set as output path and test output path for the build module of the project under Project Structure -> Modules -> ProjectName-build -> Paths -> Use module compile output path.
Looking at the Sources tab for both the ProjectName module and ProjectName-build modules I saw that there is no place where ProjectRootFolder\project\target was marked as Source.
It seems the warnings were caused by the fact that the project and . folders were marked as Sources in the ProjectName-build module.
Since the SBT build module is not needed when using IDEA to build the project, one solution would be to generate the IDEA project without that module:
sbt gen-idea no-sbt-build-module
More details here: https://github.com/mpeltonen/sbt-idea/issues/180
UPDATE
Removing the build module is actually problematic since the build.scala file will show a lot of warnings because the required libraries would be missing.
A solution would be to unmark . and project from being Sources of the build module, which is also troublesome since it would need to be done after each gen-idea.
A better solution would be to use sbt to build the project instead of make. To achieve that remove the make before launch step in the IDEA run configuration and add a SBT products step instead.
I get the same warning and it didn't cause any problems so far.
Judging by this code it seems that they simply delete only files generated by IDE, otherwise they would want to delete everything in target directory. They are playing safe by checking if there could be any source files there:
// check that output and source roots are not overlapping
final List<File> filesToDelete = new ArrayList<File>();
for (Map.Entry<File, Collection<BuildTarget<?>>> entry : rootsToDelete.entrySet()) {
context.checkCanceled();
boolean okToDelete = true;
final File outputRoot = entry.getKey();
if (JpsPathUtil.isUnder(allSourceRoots, outputRoot)) {
okToDelete = false;
}
else {
final Set<File> _outRoot = Collections.singleton(outputRoot);
for (File srcRoot : allSourceRoots) {
if (JpsPathUtil.isUnder(_outRoot, srcRoot)) {
okToDelete = false;
break;
}
}
}
if (okToDelete) {
// do not delete output root itself to avoid lots of unnecessary "roots_changed" events in IDEA
final File[] children = outputRoot.listFiles();
if (children != null) {
filesToDelete.addAll(Arrays.asList(children));
}
else if (outputRoot.isFile()) {
filesToDelete.add(outputRoot);
}
}
else {
context.processMessage(new CompilerMessage(
"", BuildMessage.Kind.WARNING, "Output path " + outputRoot.getPath() + " intersects with a source root. Only files that were created by build will be cleaned.")
);
// clean only those files we are aware of
for (BuildTarget<?> target : entry.getValue()) {
clearOutputFiles(context, target);
}
}
}

Migrating Eclipse Android Junit Tests into Android Studio Gradle

We are trying to migrate our Eclipse projects using ant builds into Android Studio using gradle. So far all is good except for our JUnit tests that use external json files. We have a ton of these where the the external file is located in the same directory as the java file. Is there a way to keep the java code and the json file in the same location and just modify the build scripts in gradle?
Current build directory is something like this \com\pack\krf\ contains MyFileTest.java and data.json. Source code to load json is this:
getClass().getResourceAsStream("data.json");
I do not want to change 300+ java files that have this same structure. Plus I do not want to group into the resource directory in gradle since some of these files will have the same name.
Here's the solution one of my teammates came up with. Add this to your build.gradle file after the android declaration. The print path is for debugging.
task alterResourcePath << {
sourceSets.testDebug.resources.srcDir('src/test/java')
sourceSets.testDebug.resources.srcDirs.each {
File f->
if(f.exists()){
println f.path
}
}
}
test.dependsOn alterResourcePath
By default gradle (like Maven) puts the resources under src/main/resources, so you'll have to tell Gradle to look for them in src/<something>/java instead:
android {
sourceSets {
instrumentTest {
resources {
srcDirs = ['src/instrumentTest/java']
}
}
}
}
if you are running robolectric test you do the same, but the robolectric plugins out there uses standard java source sets. So you won't do this inside the Android extension:
sourceSets {
test {
resources {
srcDirs = ['src/test/java']
}
}
}
If you use other sourceSets (per flavor or per build Type) you'll need to update those as well.

Custom Gradle Plugin ID not found

I'm writing a Gradle plugin and I'm failing to get the apply plugin: command to work in the Gradle script that uses the plugin. I'm using Gradle 1.1.
I've build the plugin with clean build and I'm attempting to add it to the Gradle build via a flat repo for now. That seems to be working but Gradle isn't picking up that there is a plugin with the ID test-plugin. The project name in the plugin's settings.gradle is test-plugin and the properties file in META-INF/gradle-plugins is also test-plugin.properties. I'm not sure where else I can specify the plugin ID.
The build.gradle file in the project that is using the test-plugin:
repositories {
flatDir name: 'libs', dirs: "../build/libs"
}
dependencies {
compile 'test:test-plugin:0.1'
}
apply plugin: 'test-plugin'
Error from Gradle:
What went wrong:
A problem occurred evaluating root project 'tmp'.
Plugin with id 'test-plugin' not found.
The plugin Jar has to be added as a build script dependency:
buildscript {
repositories { flatDir name: 'libs', dirs: "../build/libs" }
dependencies { classpath 'test:test-plugin:0.1' }
}
apply plugin: "test-plugin"
If you want to implement a plugin into your buildscript, then you have two options.
Option 1
apply plugin: YourCustomPluginClassName
Option 2
plugins {
id 'your.custom.plugin.id'
}
apply plugin: is used when specifying your plugin by its class name (ex. apply plugin: JavaPlugin)
plugins { } is used when specifying your plugin by its id (ex. plugins { id 'java' })
See Gradle Plugins by tutorialspoint for reference
If you choose Option 1, the your custom plugin will need to be brought into your build script by 1 of 3 ways.
You can code it directly within your Gradle build script.
You can put it under buildSrc (ex. buildSrc/src/main/groovy/MyCustomPlugin).
You can import your custom plugin as a jar in your buildscript method.
See Gradle Goodness by Mr. Haki for information about the buildscript method.
If you choose Option 2, then you need to create a plugin id. Create the following file buildSrc/src/main/resources/META-INF/gradle-plugins/[desired.plugin.id].properties.
Copy and paste implementation-class=package.namespace.YourCustomPluginClassName into your newly created .properties file. Replace package.namespace.YourCustomPluginClassName with the fully-qualified class name of your desired plugin class.
See Custom Plugins by Gradle for more info.
I also had the same problem with a custom plugin id not being found. In my case, I simply forgot to add the 'src/main/resources/META-INF/gradle-plugins' properties file. The name of the properties file must match the name of the plugin id with a '.properties' extension.
The file must contain a the line:
implementation-class=(your fully qualified plugin classpath)
That's the complete mechanism on how plugin id's get resolved to class names.
In addition the plugin needs to be added as a dependency as pointed out in the previous answer. The android documentation states that you should use a name associated with your unique domain name. I.e.: the name 'test-plugin' is not really in good form, but an id like 'com.foo.gradle.test-plugin' would be better.
Ensure that your top-level build.gradle uses the correct classpath to refer to the path to the built *.jar file.
Some plugins, like maven-publish, will build and save the jar to a specific location in mavenLocal, but the path may not be clear to see.
You should look at the file path of the jar file, and ensure it matches your classpath, but the mapping is not immediately obvious:
buildscript {
dependencies {
// This *MUST* match the local file path of the jar file in mavenLocal, which is:
// ~/.m2/repository/com/company/product/plugin/product-gradle-plugin/1.0/product-gradle-plugin-1.0.jar
classpath 'com.company.product.plugin:product-gradle-plugin:1.0'
}
}
Be careful not to use the wrong classpath, which can refer to a directory instead of the actual jar file; like this:
buildscript {
dependencies {
// This is wrong, because it refers to the mavenLocal FOLDER "product-gradle-plugin",
// instead of the jar FILE "product-gradle-plugin-1.0.jar"
// However, a gradle sync will still resolve it as a valid classpath!!
classpath 'com.company.product:product-gradle-plugin:1.0'
}
}
More info:
https://discuss.gradle.org/t/what-is-the-preferred-gradle-approach-to-locally-install-an-artifact-equivalent-to-mavens-install/5592
https://docs.gradle.org/current/userguide/publishing_maven.html
https://blog.codefx.org/tools/snapshots-gradle-maven-publish-plugin/
https://docs.gradle.org/current/userguide/custom_plugins.html#sec:custom_plugins_standalone_project
Adding to what #Bonifacio2 wrote this is a path META-INF/gradle-plugins and shows in IntelliJ as META-INF.gradle-plugins. At all costs don't make the stupid mistake I did creating this directly as a directory META-INF.gradle-plugins because you are based on another sample, and never works. Another tip is copying also from another intelliJ project as this is what is added: gradle-plugins.
hmm perhaps try;
configure <org.jsonschema2pojo.gradle.JsonSchemaExtension> {
this.sourceFiles = files("${project.rootDir}/schemas")
}