to make it short: I am unable to shrink my more complex scala programs with Proguard. I think I am overlooking something, but currently I have no Idea what it could be.
My Current setup relys upon a build.gradle-file to shrink my compiled fat-jar.
import proguard.gradle.ProGuardTask
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'net.sf.proguard:proguard-gradle:5.2.1'
}
}
plugins {
id 'application'
id 'java'
id 'scala'
}
mainClassName = 'app.Main'
sourceCompatibility = 1.8
repositories {
...
}
dependencies {
...
}
jar {
manifest {
attributes(
'Class-Path': configurations.compileClasspath.files.collect {"$it.name"}.join(' '),
'Main-Class': 'app.Main'
)
}
from {
configurations.compileClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
}
task proguarding(type: ProGuardTask, dependsOn: jar) {
configuration 'proguard-rules.pro'
libraryjars files(configurations.compileClasspath.collect { it })
injars "$buildDir/libs/Main.jar"
outjars "$buildDir/libs/Shrunk.jar"
}
These are my proguard rules inside the proguard-rules.pro file
-keep class testing.** { *; }
-dontoptimize
-dontobfuscate
#-dontpreverify
#-dontnote
-ignorewarnings
-forceprocessing
-keepclassmembers enum * {
public static **[] values();
public static ** valueOf(java.lang.String);
}
-dontwarn scala.**
-keep class !scala*.** { *; }
-dontwarn **$$anonfun$*
-dontskipnonpubliclibraryclasses
-dontskipnonpubliclibraryclassmembers
-keep class ... # imagine like 5-thousend different keep-rules I tried
-keepclassmembers class * {
** MODULE$;
}
All the Programs/ Libs I tried to shrink so far didn't work after the shrinking.
Except for the most basic scala program:
package main
import java.io._
object Main extends App {
val pw = new PrintWriter(new File("hello.txt" ))
pw.write("Hello, world")
pw.close()
}
This one I could shrink from 5.5mb to 432kb, but everything else won't work and instead throws me run time errors like NullPointerExceptions or something about the stackmap.
Exception in thread "main" java.lang.VerifyError: Expecting a stackmap frame at branch target 14
Exception Details:
Location:
generator/Main$.parameters()Lscalafx/application/JFXApp$Parameters; #4: ifne
Reason:
Expected stackmap frame at this location.
Bytecode:
0x0000000: 2ab4 0010 9a00 0a2a b700 1ca7 0007 2ab4
0x0000010: 0011 b0
at generator.Main.main(Main.scala)
I found out, that these problems arose through the dontpreverify-option, but If I don't use "-dontpreverify", it won't even compile.
Without that option ProGuard is not able to find superclasses (just like this guy)
So I tried every keep-option I could find.
None of the solutions I found helped in any way. If anybody knows what's wrong, please help I am at the end of my nerves.
If somebody knows another shrinker for jar files that is actually easy to use, please tell me.
Btw, The GUI was even more confusing than the gradle plugin so, I ditched that approach
The mistake I made, was not to include the java runtime jar from the jdk
Here is the gradle with the proguarding task
import proguard.gradle.ProGuardTask
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'net.sf.proguard:proguard-gradle:6.0.3'
}
}
plugins { ... }
version '1.0'
mainClassName = 'app.Main'
sourceCompatibility = 1.8
repositories { ... }
dependencies { ... }
jar {
manifest {
attributes(
'Class-Path': configurations.compileClasspath.files.collect {"$it.name"}.join(' '),
'Main-Class': 'app.Main'
)
}
from {
configurations.compileClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
}
task proguarding(type: ProGuardTask, dependsOn: jar) {
configuration 'proguard-rules.pro'
libraryjars files(configurations.compileClasspath.collect { it })
injars "$buildDir\\libs\\" + project.name + "-" + version + ".jar"
outjars "$buildDir\\libs\\" + project.name + "-" + version + "_shrunk" + ".jar"
}
And here are the pro-guard rules inside proguard-rules.pro
-keep class app.** { *; }
#-dontoptimize
-dontobfuscate
#-dontnote
#-dontusemixedcaseclassnames
-ignorewarnings
-forceprocessing
-libraryjars C:/Program Files/Java/jdk1.8.0_181/jre/lib/rt.jar
-libraryjars C:/Program Files/Java/jdk1.8.0_181/jre/lib/ext/jfxrt.jar
Now I don't get any warnings and everything compiles just fine.
Related
I have created a gradle/scala project in intellij.
This is my project structure
build
gradle
- wrapper
META-INF
out
src
- main
-resources
-scala
- test
-resources
-scala
build.gradle
gradlew
gradlew.bat
settings.gradle
These are the contents of my build.gradle file
plugins {
id 'scala'
}
group 'org.example'
version '1.0-SNAPSHOT'
repositories {
mavenLocal()
maven {
url = uri('<artifactory location>')
}
}
jar {
manifest {
attributes(
'Class-Path': configurations.compile.collect { it.getName() }.join(' '),
'Main-Class': 'org.example.Hello'
)
}
}
sourceCompatibility = '12'
dependencies {
implementation 'org.scala-lang:scala-library:2.12.14'
testImplementation 'org.scalatest:scalatest_2.12:3.0.5'
testRuntimeOnly 'org.scala-lang.modules:scala-xml_2.12:1.1.1'
}
test {
useJUnitPlatform()
}
Main Class Content
package org.example
object Hello extends App {
println("hello!")
}
When I try to build the jar and execute it , it gives an error
Could not find or load main class org.example.Hello . What might be the mistake here?
I was able to resolve the issue by adding the following in the settings.gradle
rootProject.name = 'gradle-scala'
include('gradle-scala')
Basically , the include tag is used to specify the directory where the code resides .
I see out directory, can you check the gradle settings
ensure below are ticked
I'm new to Gradle and built a Jar build using Gradle. When I tried executing that jar using spark-submit, I'm getting a Class Not Found error. I'm able to execute the same application in IntelliJ IDEA, but getting error when executing as Independent JAR.
Below are the contents of build.gradle:
/*
* This file was generated by the Gradle 'init' task.
*/
apply plugin: 'scala'
apply plugin: 'idea'
apply plugin: 'org.sonarqube'
// Applying Sonar Qube details for gradle
apply from: "${rootProject.rootDir}/gradle/sonar.gradle"
repositories {
mavenLocal()
mavenCentral()
maven {
url = uri('https://repo1.maven.org/maven2')
}
}
dependencies {
implementation 'com.google.code.gson:gson:2.8.2'
implementation 'io.netty:netty-all:4.1.42.Final'
implementation 'com.github.jengelman.gradle.plugins:shadow:4.0.1'
testImplementation "org.scalatest:scalatest_2.11:$scalaTestVersion"
runtime "com.apple.jvm.commons:commons-metrics:0.13.1"
}
buildscript {
dependencies {
classpath 'org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.6'
}
}
sourceSets {
main {
scala {
srcDirs = ['src/main/scala']
}
}
test {
scala.srcDirs = ['src/test/scala']
}
}
sonarqube {
properties {
property "sonar.projectName", "App-name"
property "sonar.projectKey", "App-name"
property "sonar.sourceEncoding", "UTF-8"
property "sonar.tests", "src/test"
property "sonar.sources", "src/main"
property "sonar.scala.coverage.reportPaths", "$buildDir/reports/scoverage/scoverage.xml"
property "sonar.coverage.jacoco.xmlReportPaths", "$buildDir/jacoco/jacoco.xml"
property "sonar.test.exclusions", "src/test/**"
}
}
configurations {
jars
}
// Force Jacksom-module version to handle below error.
//com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.8.8 requires Jackson Databind version >= 2.8.0 and < 2.9.0
configurations.all {
resolutionStrategy {
force "com.fasterxml.jackson.module:jackson-module-scala_2.11:2.9.5"
}
}
jar {
manifest {
attributes(
'Class-Path': configurations.compile.files.collect {"$it.name"}.join(' '),
'Main-Class': 'com.github.MainClass'
)}
from {
files(sourceSets.main.output.classesDirs)
configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}{
exclude "META-INF/*.SF"
exclude "META-INF/*.DSA"
exclude "META-INF/*.RSA"
}
zip64 true
archiveClassifier = 'jar-with-dependencies'
}
/*task fatJar(type: Jar) {
//manifest.from jar.manifest
manifest {
attributes 'Main-Class': 'com.github.MainClass'
}
zip64 true
archiveClassifier = 'jar-with-dependencies'
from files(sourceSets.main.output.classesDirs)
from {
configurations.runtime.asFileTree.files.collect {zipTree(it) }
configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
} {
exclude "META-INF/*.SF"
exclude "META-INF/*.DSA"
exclude "META-INF/*.RSA"
}
with jar
}
artifacts {
archives fatJar
}*/
I tried building the Jar with both ./gradlew clean build and ./gradlew clean fatJar
and executing below command:
spark-submit --master local --deploy-mode client --class com.github.MainClass /local_path/.out/libs/App-name-1.0-SNAPSHOT-jar-with-dependencies.jar
Error:
java.lang.ClassNotFoundException: com.github.MainClass
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:810)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
2020-12-02 13:02:49 INFO ShutdownHookManager:54 - Shutdown hook called
2020-12-02 13:02:49 INFO ShutdownHookManager:54 - Deleting directory /private/var/folders/sq/npjk1mkn7lgfm57mf9g_3rrh0000gn/T/spark-8ab912b5-23ef-4182-9c70-947f2cd2831a
I add compile 'org.apache.commons:commons-vfs2-project:2.3' to my build.gradle file, but I found there is ~\.gradle\caches\modules-2\files-2.1\org.apache.commons\commons-vfs2-project\2.3\860b837e62ab6d3282e7ff96f63ea21aebcdba40\commons-vfs2-project-2.3.pom, how can I download jar file?
Here is one of the solution.
Below is the build.gradle for a sample project
I have included 2 plugins and written script to include additional scripts while building jar.
There are some eclipse generated scripts which I have not removed.
apply plugin: 'java'
/**Following plugins are needed */
apply plugin: 'maven-publish'
apply plugin: 'maven'
/**Just to include jar versioning*/
version = '1.0.0'
repositories {
jcenter()
}
dependencies {
compile 'org.slf4j:slf4j-api:1.7.25'
testCompile 'junit:junit:4.12'
}
apply plugin: 'eclipse'
eclipse {
classpath {
downloadSources = true // default: true
downloadJavadoc = true // default: false
}
}
def eclipseSourceFolders=[
'src/main/java',
'src/main/resources',
'src/test/java',
'src/test/resources'
];
tasks.eclipse.dependsOn << {
for (String sourceFolder: eclipseSourceFolders){
def resourceDir = new File(project.projectDir, sourceFolder)
if( !resourceDir.exists() && ! resourceDir.mkdirs() ) {
logger.info("Not able to create %1",resourceDir);
}
}
}
/**
Add Following scripts to placs pom.xml within jar and .pom file within libs folder*/
publishing {
publications {
mavenJava(MavenPublication) {
from components.java
}
}
}
/** While creating jar generate and place pom.xml also place .pom file in libs folder */
jar {
into("META-INF/maven/$project.group/$project.name") {
from { generatePomFileForMavenJavaPublication }
rename ".*", "pom.xml"
}
doLast {
pom {
project {
}
}.writeTo("$buildDir/libs/$project.name-${version}.pom")
}
}
Does somebody understands what is a problem? Why my IDE doesn't see classes from dependencies? Idea version: 17.2.2
The root build.gradle:
subprojects.each {
apply plugin: 'idea'
}
The root setting.gradle:
include 'client'
include 'api'
rootProject.name = 'app-1-akka-reactjs'
My build.gradle of api project:
apply plugin: 'play'
buildscript {
repositories {
jcenter()
}
dependencies {
classpath 'org.iatoki:gradle-play-idea:0.4.1'
}
}
apply plugin: 'org.iatoki.play-idea'
repositories {
jcenter()
ivy {
url "https://repo.typesafe.com/typesafe/ivy-releases/"
layout "pattern", {
ivy "[organisation]/[module]/[revision]/ivys/ivy.xml"
artifact "[organisation]/[module]/[revision]/jars/[artifact].[ext]"
}
}
}
model {
components {
play {
platform play: '2.5.8', scala: '2.11', java: '1.8'
injectedRoutesGenerator = true
}
}
}
dependencies {
play 'com.typesafe.play:play-slick_2.11:2.1.0'
play 'com.typesafe.play:play-slick-evolutions_2.11:2.1.0'
play 'org.postgresql:postgresql:9.4-1200-jdbc41'
}
The build.gradle of client is empty for now.
To generate idea's files I've used:
gradle cleanIdea idea
I faced a similar issue. I could resolve it by moving/copying the settings.xml file from $MVN_HOME/conf to ~/.m2 folder. As MVN_HOME was not detected by Idea it could not resolve the settings.xml from MVN_HOME.
I've created a Gradle plugin below:
class CommandServiceProjectPlugin implements Plugin<Project> {
public void apply(Project project) {
project.buildscript{
repositories {
maven: {
url: 'http://localhost:8081/artifactory/zailab-virtual-repo'
credentials: {
username = "admin"
password = "password"
}
}
}
/*Spring Boot Gradle plugin */
dependencies {
classpath: 'org.springframework.boot:spring-boot-gradle-plugin:1.1.6.RELEASE'
}
}
project.apply plugin: 'spring-boot'
project.apply plugin: 'java'
project.apply plugin: 'eclipse'
project.repositories {
maven: {
url: 'http://localhost:8081/artifactory/zailab-virtual-repo'
}
}
project.dependencies {
/*Spring Boot dependencies */
compile: 'org.springframework.boot:spring-boot-starter-test'
compile: 'org.springframework.boot:spring-boot-starter-aop'
compile: 'org.springframework.boot:spring-boot-starter-data-mongodb'
compile: 'org.springframework.boot:spring-boot-starter-integration'
compile: 'org.springframework.boot:spring-boot-starter-amqp'
/*Axon dependencies */
compile: 'org.axonframework:axon-core:2.3.1'
compile: 'org.axonframework:axon-mongo:2.3.1'
}
}
}
I then apply the plugin within another project as below, but it seems the buildscript definitions override/conflict as the 'spring-boot' plugin cannot be found. Am I attempting the impossible or is there perhaps another way to achieve what I am trying to do?
buildscript {
repositories {
maven {
url 'http://localhost:8081/artifactory/zailab-virtual-repo'
credentials {
username = "admin"
password = "password"
}
}
}
dependencies {
classpath(group: 'com.zailab', name: 'zailab-command-service-build', version: '1.0.0- SNAPSHOT')
}
}
apply plugin: 'com.zailab.command.service.project'
Thanks,
Roscoe
As far as I know, it's not possible to add build script dependencies programmatically from a plugin.
Reason for this is build script life cycle - invocation of plugins' apply method happens after the project's classpath configuration had already been resolved.
You should either configure the buildscript in project's build script, or package classpath dependencies with the plugin.