I'm using ScalaPB (version 0.11.1) and plugin sbt-protoc (version 1.0.3) to try to compile an old project with ProtocolBuffers in Scala 2.12. Reading the documentation, I want to set the file property preserve_unknown_fields to false. But my question is, where? Where do I need to set this flag? On the .proto file?
I've also tried to include the flag as a package-scoped option by creating a package.proto file next to my other .proto file, with the following content (as it is specified here):
import "scalapb/scalapb.proto";
package eur.astrata.eu.bigdata.tpms.protobuf;
option (scalapb.options) = {
preserve_unknown_fields: false
};
But when trying to compile, I get the following error:
[libprotobuf WARNING T:\src\github\protobuf\src\google\protobuf\compiler\parser.cc:648] No syntax specified for the proto file: package.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
scalapb/scalapb.proto: File not found.
package.proto:1:1: Import "scalapb/scalapb.proto" was not found or had errors.
I've also tried with syntax = "proto3"; at the beginning but it doesn't work.
Any help would be greatly appreciated.
From the docs:
If you are using sbt-protoc and importing protos like
scalapb/scalapb.proto, or common protocol buffers like
google/protobuf/wrappers.proto:
Add the following to your build.sbt:
libraryDependencies += "com.thesamet.scalapb" %% "scalapb-runtime" % scalapb.compiler.Version.scalapbVersion % "protobuf"
This tells sbt-protoc to extract protos from this jar (and all its dependencies,
which includes Google's common protos), and make them available in the
include path that is passed to protoc.
It is important to add that by setting preserve_unknown_fields to false you are turning off a protobuf feature that could prevent data loss when different parts of a distributed system are not running the same version of the schema.
Environment:
OpenJDK 64-Bit Server VM Zulu12.2+3-CA (build 12.0.1+12, mixed mode, sharing)
Scala 2.12.7
Windows 10 Professional, X86_64
IntelliJ IDEA 2019.1.3 (Ultimate Edition)
I checked out the scalafx-hello-world from GitHub, built and ran it in IntelliJ and it worked all fine. Here quickly the significant application implementation:
package hello
import scalafx.application.JFXApp
import scalafx.application.JFXApp.PrimaryStage
import scalafx.geometry.Insets
import scalafx.scene.Scene
import scalafx.scene.effect.DropShadow
import scalafx.scene.layout.HBox
import scalafx.scene.paint.Color._
import scalafx.scene.paint._
import scalafx.scene.text.Text
object ScalaFXHelloWorld extends JFXApp {
stage = new PrimaryStage {
// initStyle(StageStyle.Unified)
title = "ScalaFX Hello World"
scene = new Scene {
fill = Color.rgb(38, 38, 38)
content = new HBox {
padding = Insets(50, 80, 50, 80)
children = Seq(
new Text {
text = "Scala"
style = "-fx-font: normal bold 100pt sans-serif"
fill = new LinearGradient(
endX = 0,
stops = Stops(Red, DarkRed))
},
new Text {
text = "FX"
style = "-fx-font: italic bold 100pt sans-serif"
fill = new LinearGradient(
endX = 0,
stops = Stops(White, DarkGray)
)
effect = new DropShadow {
color = DarkGray
radius = 15
spread = 0.25
}
}
)
}
}
}
}
EDIT: My build.sbt:
// Name of the project
name := "ScalaFX Hello World"
// Project version
version := "11-R16"
// Version of Scala used by the project
scalaVersion := "2.12.7"
// Add dependency on ScalaFX library
libraryDependencies += "org.scalafx" %% "scalafx" % "11-R16"
resolvers += Resolver.sonatypeRepo("snapshots")
scalacOptions ++= Seq("-unchecked", "-deprecation", "-Xcheckinit", "-encoding", "utf8", "-feature")
// Fork a new JVM for 'run' and 'test:run', to avoid JavaFX double initialization problems
fork := true
// Determine OS version of JavaFX binaries
lazy val osName = System.getProperty("os.name") match {
case n if n.startsWith("Linux") => "linux"
case n if n.startsWith("Mac") => "mac"
case n if n.startsWith("Windows") => "win"
case _ => throw new Exception("Unknown platform!")
}
// Add JavaFX dependencies
lazy val javaFXModules = Seq("base", "controls", "fxml", "graphics", "media", "swing", "web")
libraryDependencies ++= javaFXModules.map( m=>
"org.openjfx" % s"javafx-$m" % "11" classifier osName
)
After that, I changed the implementation to:
package hello
import javafx.application.Application
import javafx.scene.Scene
import javafx.scene.control.Label
import javafx.stage.Stage
class ScalaFXHelloWorld extends Application {
override def start(stage: Stage): Unit = {
stage.setTitle("Does it work?")
stage.setScene(new Scene(
new Label("It works!")
))
stage.show()
}
}
object ScalaFXHelloWorld {
def main(args: Array[String]): Unit = {
Application.launch(classOf[ScalaFXHelloWorld], args: _*)
}
}
Here I get the following error:
Exception in Application start method
java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:464)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:363)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1051)
Caused by: java.lang.RuntimeException: Exception in Application start method
at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:900)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:195)
at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.lang.IllegalAccessError: superclass access check failed: class com.sun.javafx.scene.control.ControlHelper (in unnamed module #0x40ac0fa0) cannot access class com.sun.javafx.scene.layout.RegionHelper (in module javafx.graphics) because module javafx.graphics does not export com.sun.javafx.scene.layout to unnamed module #0x40ac0fa0
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:802)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:700)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:623)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
at javafx.scene.control.Control.<clinit>(Control.java:86)
at hello.ScalaFXHelloWorld.start(ScalaFXHelloWorld.scala:39)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:846)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:455)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:428)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:389)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:427)
at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174)
... 1 more
Exception running application hello.ScalaFXHelloWorld
Now my question is: What does ScalaFX that the module problem does not occur?
I've not been able to exactly reproduce your problem, but I have been able to get a project that uses JavaFX-only (that is, it doesn't make use of ScalaFX) to build and run.
Here's what I'm using (everything else is specified in the build file):
Zulu OpenJDK 11
SBT 1.2.8
(I did try using Zulu OpenJDK 12 to build and run the project, and that worked too. However, it's probably best if you use the version of OpenJFX that matches the JDK.)
When I tried your original sources and build.sbt, I encountered the following error when executing an sbt run command from the command line:
D:\src\javafx11>sbt run
[info] Loading global plugins from {my home directory}\.sbt\1.0\plugins
[info] Loading project definition from D:\src\javafx11\project
[info] Loading settings for project javafx11 from build.sbt ...
[info] Set current project to JavaFX 11 Hello World (in build file:/D:/src/javafx11/)
[info] Running (fork) hello.ScalaFXHelloWorld
[error] Error: JavaFX runtime components are missing, and are required to run this application
[error] Nonzero exit code returned from runner: 1
[error] (Compile / run) Nonzero exit code returned from runner: 1
[error] Total time: 1 s, completed Aug 11, 2019, 3:17:07 PM
as I mentioned in my original comments to your question.
I thought that was odd because the code compiled, which meant that the compiler was able to find the JavaFX runtime just fine.
I then tried running the program without forking, by commenting out the fork := true in the build file. Guess what? The program ran without error!
I may be missing something, regarding using SBT with JDK versions 9+, but this indicated that SBT was somehow not running the forked process correctly. I could force the forked process to run correctly by adding the following to the end of the build file:
val fs = File.separator
val fxRoot = s"${sys.props("user.home")}${fs}.ivy2${fs}cache${fs}org.openjfx${fs}javafx-"
val fxPaths = javaFXModules.map {m =>
s"$fxRoot$m${fs}jars${fs}javafx-$m-11-$osName.jar"
}
javaOptions ++= Seq(
"--module-path", fxPaths.mkString(";"),
"--add-modules", "ALL-MODULE-PATH"
)
This works by adding the downloaded ivy-managed JavaFX jar files to Java's module path. However, this is not a good solution for running standalone applications. It may be possible for the sbt-native-packager to provide the necessary environment for the completed application to run, but I haven't tried that.
I've posted the complete solution on GitHub
Let me know whether this helps. In the meantime, I'll look into SBT's support for JDK 9+ modules to see whether there is a simpler solution...
UPDATE:
I have raised an issue (#4941) with the SBT team to look into this in more detail.
UPDATE 2
I patched an issue that stopped the solution from working on Linux. Perform a git pull to update the sources.
UPDATE 3
I should also mention that it's best to have IntelliJ run the application using SBT, which keeps things simple and ensures that the application's environment is properly configured.
To do this, got to the IntelliJ Run menu, and select the Edit Configurations... option. Click the + button at the top left corner of the dialog, select sbt Task" from the list under **Add New Configuration, then configure as follows:
This will compile and build the application first, if required.
Note: The _VM parameters are for running SBT, and do not relate to how SBT runs your forked application.
(You can also add SBT run configurations to test your code, as well.)
Adding to Jonathan Crosmer's answer:
The reason that naming the class and the object differently works is because the Java launcher actually has special behaviour in place if the main class extends javafx.application.Application. If you have the Java sources available, the relevant code can be found in JAVA_HOME/lib/src.zip/java.base/sun/launcher/LauncherHelper.java. In particular there are two methods which are of interest:
public static Class<?> checkAndLoadMain(boolean, int ,String)
//In nested class FXHelper
private static void setFXLaunchParameters(String, int)
The first methods has a check in place that looks if the main class extends javafx.application.Application. If it does, this method replaces the main class with the nested class FXHelper, which has its own public static void main(String[] args).
The second method, which is directly called by the first method, attempts to load the JavaFX runtime. However, the way it does this is by first loading the module javafx.graphics via java.lang.ModuleLayer.boot().findModule(JAVAFX_GRAPHICS_MODULE_NAME).
If this call fails, Java will complain about not having found the JavaFX runtime and then immediatly exit via System.exit(1).
Going back to SBT and Scala, some other details are in play. First, if both the main object and the class extending javafx.application.Application have the same name, the Scala compiler will generate a class file which both extends Application and has a public static void main(...). That means that the special behaviour described above will be triggered and the Java launcher will try to load the JavaFX runtime as a module. Since SBT currently has no notion about modules, the JavaFX runtime will not be on the module path and the call to findModule(...) will fail.
On the other hand, if the main object has a different name from the main class, the Scala compiler will place public static void main(...) in a class which does not extend Application, which in turn means that the main() method will execute normally.
Before we go on, we should note that while SBT did not put the JavaFX runtime on the module path, it DID in fact put it on the classpath. That means that the JavaFX classes are visible to JVM, they just can not be loaded as a module. After all
A modular JAR file is like an ordinary JAR file in all possible ways, except that it also includes a module-info.class file in its root directory.
(from The State of the Module System)
However, if a method happens to call, let's say Application.launch(...), Java will happily load javafx.application.Application from the classpath. Application.launch(...) will similarly have access to the rest of JavaFX and everything works out.
That is also the reason why running a JavaFX app without forking works. In that case SBT will always invoke public static void main(...) directly, which means that no special behaviours from the java launcher are triggered and the JavaFX runtime will be found on the classpath.
Here is a snippet to see the above behaviour in action:
Main.scala:
object Main {
def main(args: Array[String]): Unit = {
/*
Try to load the JavaFX runtime as a module. This is what happens if the main class extends
javafx.application.Application.
*/
val foundModule = ModuleLayer.boot().findModule("javafx.graphics").isPresent
println("ModuleLayer.boot().findModule(\"javafx.graphics\").isPresent = " + foundModule) // false
/*
Try to load javafx.application.Application directly, bypassing the module system. This is what happens if you
call Application.launch(...)
*/
var foundClass = false
try{
Class.forName("javafx.application.Application")
foundClass = true
}catch {
case e: ClassNotFoundException => foundClass = false
}
println("Class.forName(\"javafx.application.Application\") = " + foundClass) //true
}
}
build.sbt:
name := "JavaFXLoadTest"
version := "0.1"
scalaVersion := "2.13.2"
libraryDependencies += "org.openjfx" % "javafx-controls" % "14"
fork := true
I ran across this same exact problem and found a disturbingly odd and easy solution. tldr; make the main class have a different name from the JavaFX Application class. First an example:
import javafx.application.Application
import javafx.event.ActionEvent
import javafx.event.EventHandler
import javafx.scene.Scene
import javafx.scene.control.Button
import javafx.scene.layout.StackPane
import javafx.stage.Stage
object HelloWorld {
def main(args: Array[String]): Unit = {
Application.launch(classOf[HelloWorld], args: _*)
}
}
// Note: Application class name must be different than main class name to avoid JavaFX path initialization problems! Try renaming HelloWorld -> HelloWorld2
class HelloWorld extends Application {
override def start(primaryStage: Stage): Unit = {
primaryStage.setTitle("Hello World!")
val btn = new Button
btn.setText("Say 'Hello World'")
btn.setOnAction(new EventHandler[ActionEvent]() {
override def handle(event: ActionEvent): Unit = {
System.out.println("Hello World!")
}
})
val root = new StackPane
root.getChildren.add(btn)
primaryStage.setScene(new Scene(root, 300, 250))
primaryStage.show()
}
}
The code as written above throws the exception from the original question. If I rename the class HelloWorld to HelloWorld2 (keeping object HelloWorld, and changing the launch call to classOf[HelloWorld2]), it runs fine. I suspect this is the "magic" that makes ScalaFX work as well, because it is wrapping the JavaFX Application in its own JFXApp type, creating a hidden Application class.
Why does it work? I'm not completely sure, but when running each piece of code in IntelliJ using a standard Run config (right-click HelloWorld and "run HelloWorld.main()") , then in the output clicking "/home/jonathan/.jdks/openjdk-14.0.1/bin/java ..." to expand it shows a command that includes "--add-modules javafx.base,javafx.graphics ", among other things. In the second version, with the renamed HelloWorld2 app, the command does not include this. I can't figure out how IntelliJ has decided to make the command different, but I can only speculate it has something to do with inferring it is a JavaFX app and trying to be helpful by automatically adding "--add-modules "...? In any case, the modules list doesn't include all the modules needed, so for example creating a button requires "javafx.controls", and you get the error. But when the main class doesn't match the Application name, whatever magic inference it does gets turned off, and the standard classpath from the build.sbt just works.
Fun follow up: if I run the application from the sbt shell using sbt run, then the pattern is the same (HelloWorld fails, but renaming the application class fixes it), but the error message is the more-straightforward-but-still-unhelpful "Error: JavaFX runtime components are missing, and are required to run this application". So maybe not entirely an IntelliJ problem, but something to do with JavaFX and Jigsaw? Anyway it's a mystery, but at least we have an easy solution.
Is it somehow possible to use an external library inside the build.sbt file?
E.g. I want to write something like this:
import scala.io.Source
import io.circe._ // not possible
version := myTask
lazy val myTask: String = {
val filename = "version.txt"
Source.fromFile(filename).getLines.mkString(", ")
// do some json parsing using the circe library
// ...
}
One of the things I actually like about sbt is that the build project is (in most ways) just another project (which is also potentially configured by a meta-build project configured by a meta-meta-build project, etc.). This means you can just drop the following line into a project/build.sbt file:
libraryDependencies += "io.circe" %% "circe-jawn" % "0.11.1"
You could also add this to plugins.sbt if you wanted, or any other .sbt file in the projects directory, since the filenames (excluding the extension) have no meaning beyond human convention, but I'd suggest following convention and going with build.sbt.
Note though that sbt implicitly imports sbt.io in .sbt files, so the circe import in your build.sbt (at the root level—i.e. the build config, not the build build config) will need to look like this:
import _root_.io.circe.jawn.decode
scalaVersion := decode[String]("\"2.12.8\"").right.get
(For anyone who hasn't seen it before, the _root_ here just means "start the package hierarchy here instead of assuming io is the imported one".)
For example, suppose I have a file project/CodeGeneration.scala that generates "managed" source-code, and suppose that object (CodeGeneration) needs to leverage a third-party library -- say jsoup...
import org.jsoup._
object CodeGeneration {
def generateCode = /* Generate code using jsoup... */
}
Simply adding a line for jsoup to your libraryDependencies in build.sbt doesn't do the trick; it leads to a compilation error complaining about the missing jsoup object/namespace.
So, (how) can one access this dependency from "meta" code -- code that generates other code?
It seems the solution is to leverage sbt's "recursive" nature, and put an additional build.sbt file in the project directory. So, for example, project/build.sbt might look like this:
libraryDependencies += "org.jsoup" % "jsoup" % "1.11.2"
There's more detail in sbt's official documentation.
I have two different copies of scala version 2.9.2 from the Scala-Lang website.
No. 1 is binary code: scala-2.9.2.tgz
No. 2 is source code: scala-sources-2.9.2.tgz
I built scala from the source code. I have tried using both in an application and the binary works but the compiled source code does not.
My use case: I am compiling a tiny scala file that I wrote (see below for an example), then creating a jar, and then running "main" from within that jar on the command line. To make that jar, I must include "scala-library.jar" (from this SO answer) on the classpath. I need to be able to use the scala-sources that I build so that I can modify the source code later on.
When I use the compiled source code, I get an error regarding serialVersionUID, whereas the binary code does not cause this issue.
Exact error message (when using the scala that I build from source):
java.io.InvalidClassException: scala.collection.mutable.WrappedArray$ofRef; local class incompatible: stream classdesc serialVersionUID = 8184381945838716286, local class serialVersionUID = 6238838760334617323
Example code that I am running for deserializing a file:
object test {
def main(args: Array[String]) {
var infile = args(0)
val thing1 : MyClass = withObjectInput(infile) {
f => f.readObject().asInstanceOf[MyClass]
}
}
}
If MyClass doesn't explicitly supply a serialVersionUID, the the compiler generates one. This will be different for different versions of javac used when compiling it.
So this error will be caused by a mismatch between the version of javac that you used for compiling scala-library and the version that was used for the downloaded binary.