I am trying to instrument a java app with prometheus exporter using sbt-native-packager:
This is what I have:
plugin.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.5.1")
build.sbt
import com.typesafe.sbt.packager.docker.DockerChmodType
dockerChmodType := DockerChmodType.UserGroupWriteExecute
enablePlugins(JavaServerAppPackaging, AshScriptPlugin, DockerPlugin)
settings(moduleName := "my-test")
settings(mainClass in Compile := Some("org.name.demo.stream.MyTestClass"))
dockerBaseImage := "openjdk:8-jre-alpine"
daemonUser in Docker := "test"
As per the prometheus exporter docs, I would like to instrument the exporter as an java agent:
java -javaagent:./jmx_prometheus_javaagent-0.12.0.jar=8080:config.yaml -jar my-test_2.12-0.1.jar
Is it possible for me to do this instrumentation via sbt-native-packager? Appreciate inputs.
Use https://github.com/sbt/sbt-javaagent
In plugins.sbt
addSbtPlugin("com.lightbend.sbt" % "sbt-javaagent" % "0.1.6")
then in build.sbt
val MyAppSettings = Seq(
name := "My",
dockerBaseImage := "adoptopenjdk:11-jre-hotspot",
mainClass := Some("com.MyClass"),
packageName in Docker := "mycontainer",
libraryDependencies ++= myDependencies,
javaAgents += JavaAgent("io.prometheus.jmx" % "jmx_prometheus_javaagent" % "0.16.1", arguments = "33002:/opt/docker/jmx-exporter.yaml"),
mappings in Universal ++= Seq(
file("jmx-exporter.yaml") -> "jmx-exporter.yaml"
)
)
lazy val myProject = (project in file("my-project"))
.settings(MyAppSettings)
.enablePlugins(JavaAppPackaging, JavaAgent)
Related
I am trying to publish a multi project to sonatype using sbt-release, sbt-pgp and sbt-sonatype plugins.
while running: sbt publishLocalSigned I see the .asc file published.
but when running sbt release to sonatype I can only see the .md5 and sha1 files.
here is my release.sbt:
import ReleaseTransformations._
// publishing
publishMavenStyle in ThisBuild := true
credentials in ThisBuild += Credentials(Path.userHome / ".ivy2" / ".credentials_sonatype")
publishTo in ThisBuild := {
val nexus = "https://oss.sonatype.org/"
if (version.value.trim.endsWith("SNAPSHOT"))
Some("snapshots" at nexus + "content/repositories/snapshots")
else
Some("releases" at nexus + "service/local/staging/deploy/maven2")
}
publishArtifact in Test := false
pomIncludeRepository in ThisBuild := { _ => false }
pomExtra in ThisBuild := {
<url>my.url</url>
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<scm>
<connection>scm:git:git#github.com:url.git</connection>
<developerConnection>scm:git:git#github.com:url.git</developerConnection>
<url>url</url>
</scm>
<developers>
<developer>
<id>dev</id>
<name>dev</name>
<email>dev#gmail.com</email>
</developer>
</developers>
}
// use maven style tag name
releaseTagName in ThisBuild := s"${name.value}-${(version in ThisBuild).value}"
// sign artifacts
releasePublishArtifactsAction in ThisBuild := PgpKeys.publishSigned.value
// don't push changes (so they can be verified first)
releaseProcess in ThisBuild := Seq(
checkSnapshotDependencies,
inquireVersions,
runTest,
setReleaseVersion,
commitReleaseVersion,
tagRelease,
publishArtifacts,
setNextVersion,
commitNextVersion,
pushChanges,
releaseStepCommand("sonatypeRelease")
)
my plugin.sbt:
addSbtPlugin("org.scoverage" % "sbt-scoverage" % "1.5.1")
addSbtPlugin("org.scoverage" % "sbt-coveralls" % "1.2.7")
addSbtPlugin("org.xerial.sbt" % "sbt-sonatype" % "2.3")
addSbtPlugin("com.jsuereth" % "sbt-pgp" % "1.1.0")
addSbtPlugin("com.github.gseitz" % "sbt-release" % "1.0.11")
I am sure my gpg keys are properly set, otherwise I wont be able to run the publishLocalSigned but it seems like I miss something to my sbt-gpg plugin to kick in on release.
sbt version is 1.2.8 and I tried to downgrade to 0.13.17 and got the same behaviour.
I have to mention I was mainly following the jackson-module-scala approach:
https://github.com/FasterXML/jackson-module-scala/blob/master/release.sbt
I'm working on a JPA + Playframework 2.5.x project which I package it as Docker image.
I want to forcibly add all the files except one file (META-INF/persistence.xml) under conf folder to docker output
Currently I have done the following:
mappings in Docker += file("conf/base/application.conf") -> "opt/docker/conf/base/application.conf"
mappings in Docker += file("conf/base/default-client.conf") -> "opt/docker/conf/base/default-client.conf"
mappings in Docker += file("conf/prod/application.conf") -> "opt/docker/conf/prod/application.conf"
mappings in Docker += file("conf/prod/logback.xml") -> "opt/docker/conf/prod/logback.xml"
mappings in Docker += file("conf/stage/application.conf") -> "opt/docker/conf/stage/application.conf"
mappings in Docker += file("conf/stage/logback.xml") -> "opt/docker/conf/stage/logback.xml"
mappings in Docker += file("conf/local/application.conf") -> "opt/docker/conf/local/application.conf"
mappings in Docker += file("conf/local/logback.xml") -> "opt/docker/conf/local/logback.xml"
mappings in Docker += file("conf/routes") -> "opt/docker/conf/routes"
mappings in Docker += file("conf/ValidationMessages.properties") -> "opt/docker/conf/ValidationMessages.properties"
I am sure this is not the best way to achieve this. Can anybody suggest better option to customize the Docker output the way I need it ?
My build.sbt file:
PlayKeys.externalizeResources := false
name := """wp-pw-ng"""
version := "1.0.0-SNAPSHOT"
lazy val `wp-pw-ng` = (project in file(".")).enablePlugins(PlayJava, JavaAppPackaging)
val playVer = "2.5.9"
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
javaJpa,
"org.hibernate" % "hibernate-core" % "5.2.3.Final" exclude("dom4j", "dom4j"),
"mysql" % "mysql-connector-java" % "6.0.5",
cache,
javaWs,
filters,
"dom4j" % "dom4j" % "1.6",
"com.google.guava" % "guava" % "20.0-rc1",
"com.amazonaws" % "aws-java-sdk" % "1.11.34"
)
val docDeps = Seq(
"io.swagger" %% "swagger-play2" % "1.5.3.2"
)
libraryDependencies ++= docDeps
fork in Test := true
javaOptions in Test ++= Seq(
"-Xms512M",
"-Xmx1536M",
"-Xss1M",
"-XX:MaxPermSize=384M"
)
fork in run := false
unmanagedResourceDirectories in Compile <+= (sourceDirectory in Compile) (_ / "resources")
mappings in Docker += file("conf/base/application.conf") -> "opt/docker/conf/base/application.conf"
mappings in Docker += file("conf/base/default-client.conf") -> "opt/docker/conf/base/default-client.conf"
mappings in Docker += file("conf/base/pw.conf") -> "opt/docker/conf/base/paywall.conf"
mappings in Docker += file("conf/prod/application.conf") -> "opt/docker/conf/prod/application.conf"
mappings in Docker += file("conf/prod/logback.xml") -> "opt/docker/conf/prod/logback.xml"
mappings in Docker += file("conf/stage/application.conf") -> "opt/docker/conf/stage/application.conf"
mappings in Docker += file("conf/stage/logback.xml") -> "opt/docker/conf/stage/logback.xml"
mappings in Docker += file("conf/local/application.conf") -> "opt/docker/conf/local/application.conf"
mappings in Docker += file("conf/local/logback.xml") -> "opt/docker/conf/local/logback.xml"
mappings in Docker += file("conf/routes") -> "opt/docker/conf/routes"
mappings in Docker += file("conf/ValidationMessages.properties") -> "opt/docker/conf/ValidationMessages.properties"
//************************************************
// Custom Docker Build,
// use command 'activator docker:publishLocal'
// to publish image locally to computer.
//************************************************
import com.typesafe.sbt.packager.docker._
val playUser = "play"
val grp = "idud"
dockerRepository := Some("quay.io/we")
version in Docker := "latest"
val buildEnv: String = System.getProperty("build.env")
dockerCommands := Seq(
Cmd("FROM", "anapsix/alpine-java:8_jdk_unlimited"),
Cmd("RUN", "apk", "-Uuv add", "--no-cache", "su-exec", "groff", "less", "python", "py-pip", "&& pip install awscli ", "&& apk --purge -v del py-pip ", " && rm /var/cache/apk/* "),
Cmd("RUN", s"addgroup $grp"),
Cmd("RUN", s"adduser -s /bin/bash -D -G $grp $playUser"),
Cmd("RUN", "echo", s"'$playUser ALL=(ALL) NOPASSWD:ALL'", ">> /etc/sudoers"),
Cmd("WORKDIR", "/opt/docker"),
Cmd("ADD", "opt /opt"),
Cmd("RUN", "chown", "-R", s"$playUser:$grp", "."),
Cmd("USER", s"$playUser"),
Cmd("ENTRYPOINT", "[\"bin/wp-pw-ng\", \"-Dconfig.file=conf/" + buildEnv + "/application.conf\", \"-Dhttp.port=7000\" , \"-Dpidfile.path=/dev/null\" ,\"-Dlogger.file=conf/" + buildEnv + "/logback.xml\"]"),
Cmd("EXPOSE", "9877")
)
Additional Info
I had to resort to this work-around due to issue in build stage mode issue with Playframework, Issue 4590 and I had to use PlayKeys.externalizeResources := false flag in my build.sbt file. This would remove all files from conf folder in Docker.
The playExternalizeResources:= false setting just prevents sbtfrom adding the playExternalizedResources to your mappings in Universal, which are the mappings available for all target package formats including docker.
Your build sbt can be lightenend with the MappingsHelper ( ScalaDocs )
import NativePackagerHelper._
mappings in Universal ++= contentOf("conf")
You can also filter the mappings to remove the unwanted files, e.g
import NativePackagerHelper._
mappings in Universal ++= contentOf("conf").filter(_.2.contains("persistence.xml")
Hope that helps,
Muki
Simply add the files that you want to be mapped inside a universal/conf directory. As a result of this, when you're creating the docker image the files get added to the /opt/docker/conf directory.
Here is a build.sbt file from https://github.com/jducoeur/bootstrap-datepicker-scalajs :
import SonatypeKeys._
sonatypeSettings
lazy val root = project.in(file(".")).
enablePlugins(ScalaJSPlugin)
name := "Scala.js facade for bootstrap-datepicker"
normalizedName := "bootstrap-datepicker-facade"
version := "0.3"
organization := "org.querki"
scalaVersion := "2.11.6"
crossScalaVersions := Seq("2.10.4", "2.11.5")
libraryDependencies ++= Seq(
"org.querki" %%% "querki-jsext" % "0.5",
"org.scala-js" %%% "scalajs-dom" % "0.8.0",
"org.querki" %%% "jquery-facade" % "0.6"
)
jsDependencies += "org.webjars" % "bootstrap" % "3.3.4" / "bootstrap.js" minified "bootstrap.min.js" dependsOn "jquery.js"
jsDependencies += "org.webjars" % "bootstrap-datepicker" % "1.4.0" / "bootstrap-datepicker.js" minified "bootstrap-datepicker.min.js" dependsOn "bootstrap.js"
jsDependencies in Test += RuntimeDOM
homepage := Some(url("http://www.querki.net/"))
licenses += ("MIT License", url("http://www.opensource.org/licenses/mit-license.php"))
scmInfo := Some(ScmInfo(
url("https://github.com/jducoeur/bootstrap-datepicker-scalajs"),
"scm:git:git#github.com:jducoeur/bootstrap-datepicker-scalajs.git",
Some("scm:git:git#github.com:jducoeur/bootstrap-datepicker-scalajs.git")))
publishMavenStyle := true
publishTo := {
val nexus = "https://oss.sonatype.org/"
if (isSnapshot.value)
Some("snapshots" at nexus + "content/repositories/snapshots")
else
Some("releases" at nexus + "service/local/staging/deploy/maven2")
}
pomExtra := (
<developers>
<developer>
<id>jducoeur</id>
<name>Mark Waks</name>
<url>https://github.com/jducoeur/</url>
</developer>
</developers>
)
pomIncludeRepository := { _ => false }
It declared two js dependencies: bootstrap and bootstrap-datepicker. Since bootstrap-datepicker already depends on bootstrap, can we not just declare bootstrap-datepicker alone?
There are different dependencies here. One is dependencies between WebJars, which just makes sure the relevant packages are downloaded.
Then there are dependencies between JS packages, which is used to make sure they are loaded in correct order into the jsdeps.js file. This dependency does not automatically include the other library into the final output.
So you need to define all JS libs you need in your application and use dependsOn to make sure they are in the correct order.
I'm using an build.sbt which has cross compile settings and basically is an adapted version of "Play with scala-js example" and having some trouble getting a clean setup for my tests. Specifically, when running my server tests, my client tests also get executed (which is something that I want to avoid).
I followed the instructions from Cannot get uTest to see my tests
and added
libraryDependencies += "com.lihaoyi" %%% "utest" % "0.3.0"
My tests for some reason didn't get executed until I added
testFrameworks += new TestFramework("utest.runner.Framework")
to every project definition. Also not adding
"com.lihaoyi" %% "utest" % "0.3.1" % "test"
to the server side triggers a series of
not found: object utest [error] import utest._
-style errors.
To my impression I shouldn't have to add these additional settings at all if having a clean setup. Here's my sbt file:
import sbt.Project.projectToRef
lazy val clients = Seq(client)
lazy val scalaV = "2.11.7"
lazy val server = (project in file("server")).settings(
scalaVersion := scalaV,
scalaJSProjects := clients,
pipelineStages := Seq(scalaJSProd/*, gzip*/),
resolvers += "scalaz-bintray" at "https://dl.bintray.com/scalaz/releases",
libraryDependencies ++= Seq(
"com.vmunier" %% "play-scalajs-scripts" % "0.3.0",
"be.doeraene" %% "scalajs-pickling-play-json" % "0.4.0"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).enablePlugins(PlayScala).
aggregate(clients.map(projectToRef): _*).
dependsOn(sharedJvm)
lazy val client = (project in file("client")).settings(
scalaVersion := scalaV,
persistLauncher := true,
persistLauncher in Test := false,
libraryDependencies ++= Seq(
"org.scala-js" %%% "scalajs-dom" % "0.8.0"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).enablePlugins(ScalaJSPlugin, ScalaJSPlay).
dependsOn(sharedJs)
lazy val shared = (crossProject.crossType(CrossType.Pure) in file("shared")).
settings(scalaVersion := scalaV,
libraryDependencies ++= Seq(
"com.lihaoyi" %%% "utest" % "0.3.1",
"be.doeraene" %%% "scalajs-pickling-core" % "0.4.0",
"com.lihaoyi" %%% "pprint" % "0.3.6"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).
jsConfigure(_ enablePlugins ScalaJSPlay)
lazy val sharedJvm = shared.jvm
lazy val sharedJs = shared.js
// loads the Play project at sbt startup
onLoad in Global := (Command.process("project server", _: State)) compose (onLoad in Global).value
And here a summary of my problems:
when I run client/test, only client tests are executed
when running play-with-scalajs-example/test, client + shared tests are executed
and strangely when running server/test my server AND client tests are executed
How could I modify my project setup to
find my server tests when running server/test
running all tests when running play-with-scalajs-example/test
and additionally including the shared tests when running server/test or client test?
And on another node, is there a way to disable scalatest? It leads to a rather unreadable testoutput:
[info] 1/2 TestSimpleServerSuite.absolutely simple test on the server side Success
[info] 2/2 TestSimpleServerSuite Success
[info] utest
[info] -----------------------------------Results-----------------------------------
[info]
[info]
[info] Tests: 0
[info] Passed: 0
[info] Failed: 0
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[info] 1/2 TestSimpleClientSuite.absolutely simple test on the client side Success
[info] 2/2 TestSimpleClientSuite Success
[info] 1/2 SimpleClient.TestSimpleClientSuite.absolutely simple test on the client side Success
[info] 2/2 SimpleClient.TestSimpleClientSuite Success
[info] ScalaCheck
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] ScalaTest
[info] Run completed in 1 second, 751 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[info] utest
[info] -----------------------------------Results-----------------------------------
[info] SimpleClient.TestSimpleClientSuite Success
[info] absolutely simple test on the client side Success
[info] TestSimpleClientSuite Success
[info] absolutely simple test on the client side Success
[info]
[info] Tests: 4
[info] Passed: 4
[info] Failed: 0
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] Total time: 11 s, completed 16.10.2015 03:25:59
Thanks a bunch and kind regards
sjrd's comment about removing aggregate pointed me to right path.
Removing it stopped
server/test
from executing both server and client tests. As sjrd pointed out, the aggregate function used also runs every task that is run on server on the client.
Aggregation means that running a task on the aggregate project will
also run it on the aggregated projects.
(See: sbt doc)
As I also wanted to run shared tests on both client and server project when running test, I modified the aggregate function for the server project definition and added an additional aggregate to the client project definition. :
Server def.:
lazy val server = (project in file("server")).settings(
scalaVersion := scalaV,
scalaJSProjects := clients,
pipelineStages := Seq(scalaJSProd/*, gzip*/),
resolvers += "scalaz-bintray" at "https://dl.bintray.com/scalaz/releases",
libraryDependencies ++= Seq(
"com.vmunier" %% "play-scalajs-scripts" % "0.3.0",
"be.doeraene" %% "scalajs-pickling-play-json" % "0.4.0",
"com.lihaoyi" %% "utest" % "0.3.1" % "test"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).enablePlugins(PlayScala).
/*
* Executes shared tests compiled to JVM with server/test
*/
aggregate(projectToRef(sharedJvm)). // Former: aggregate(clients.map(projectToRef): _*). before
dependsOn(sharedJvm)
Client def.:
lazy val client = (project in file("client")).settings(
scalaVersion := scalaV,
persistLauncher := true,
persistLauncher in Test := false,
libraryDependencies ++= Seq(
"org.scala-js" %%% "scalajs-dom" % "0.8.0"
),
testFrameworks += new TestFramework("utest.runner.Framework")
).enablePlugins(ScalaJSPlugin, ScalaJSPlay).
/*
* Executes shared tests compiled to JS with client/test
*/
aggregate(projectToRef(sharedJs)).
dependsOn(sharedJs)
When running tests with play-with-scalajs-example/test now runs all tests, including the shared tests both compiled seperately for JS and JVM.
As for having to explicitly include the utest dependencies, it seems that client and server projects don't derive from crossProject and are thus disconnected - will look into finding a better way of handling that.
When upgrading to Play 2.2 I get the error
Exception: There is no cache plugin registered. Make sure at least one CachePlugin implementation is enabled
I understand that this means that there could be more than one cache on the classpath or none at all. I have tried removing cache from the dependencies and also excluding any play imports from the other dependencies but nothing has altered the error. If I remove cache and exclude("com.typesafe.play", "*") from all dependencies I correctly get java.lang.NoClassDefFoundError: play/api/cache/Cache
But as soon as I then add cache back back in I get the same error of no cache plugin registered.
My cache dependencies but running play dependencies | grep cache
| com.typesafe.play:play-cache_2.10:2.2.2-RC2 | rm-play:rm-play_2.10:1.0-SNAPSHOT | As play-cache_2.10.jar|
| net.sf.ehcache:ehcache-core:2.6.6| com.typesafe.play:play-cache_2.10:2.2.2-RC2| As ehcache-core.jar |
| com.typesafe.play:play_2.10:2.2.2-RC2| com.typesafe.play:play-cache_2.10:2.2.2-RC2| As play_2.10.jar| net.sf.ehcache:ehcache-core:2.6.6|
My Build.scala
import sbt._
import Keys._
import play.Project._
object ApplicationBuild extends Build {
val appName = "rm-play"
val appVersion = "1.0-SNAPSHOT"
val appDependencies = Seq(
javaCore,
javaJdbc,
javaEbean,
cache,
//Group id
"com.google.guava" % "guava" % "14.0.1" ,
"com.dbdeploy" % "maven-dbdeploy-plugin" % "3.0M3",
"postgresql" % "postgresql" % "9.1-901.jdbc4",
"joda-time" % "joda-time" % "2.3",
"com.amazonaws" % "aws-java-sdk"% "1.6.11",
"ws.securesocial" %% "securesocial" % "master-SNAPSHOT" exclude("com.typesafe.play", "*")
)
val main = play.Project(appName, appVersion, appDependencies).settings(
resolvers += Resolver.sonatypeRepo("snapshots"),
resolvers += Resolver.sonatypeRepo("releases")
)
}
plugin.sbt
// Comment to get more information during initialization
logLevel := Level.Warn
// The Typesafe repository
resolvers += "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/"
// Use the Play sbt plugin for Play projects
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.2.2-RC2")
Ok so my problem was that in my application.conf file I had some cache properties left over from 2.0, commenting them out helped me get the above error.
#memcachedplugin=enabled
#memcached.namespace=srm
#ehcacheplugin=enabled
#memcached.host="127.0.0.1:11211"