I'm setting up my very first play app in a mixed build environment. My company uses maven for everything (so far) and I'm trying to get my play app to interact nicely with the rest of my artifacts.
Is there any way to get ivy/sbt/play to deal with SNAPSHOTs in a similar way to maven - namely, either update them from the remote repository always (for example, on a build worker) or use the local .m2 repository until the dependency 'expires' and then refresh it from the server.
I have declared a SNAPSHOT dependency in my Build.scala for an artifact, and I'd like local updates to this dependency to be visible to my play project. On the maven side, I do the following
mvn clean install
which (of course) builds and installs my external artifact to my local maven repository (at ~/.m2/repository). I'd like these changes to be immediately visible to my play project, but I can't figure out how to tell sbt/play to not cache SNAPSHOTs. No matter what I do, this dependency is never refreshed in play - I have to go into the actual play ivy cache and delete the dependency by hand for any changes to be picked up. Ideally, I'd like sbt/ivy to just resolve the path to my local maven repo and not cache it internally. I've got the following in my Build.scala
val main = PlayProject(appName, appVersion, appDependencies, mainLang = SCALA).settings(
resolvers += "Local Maven Repository" at "file://" + Path.userHome.absolutePath + "/.m2/repository",
testOptions in Test := Nil
)
When I run a build in play, it properly uses this repo, but then caches the results in the ivy cache. Is there an incantation I can tell Ivy/sbt to not do this? Perhaps something in ivysettings.xml?
#kheraud -> clean /reload/ update -> will not work
sbt caches it localy and do not check again for new snapshot in local maven
#dprat -> I have been looking for solution in web and haven't found anything more :(
I gave up - just delete your local package in ivy cache and do play update
you can simplify it and make a script
rm -rf ~/.ivy2/cache/your.package.foo
play update compile
Elsewhere I've seen this ascribed to an SBT defect https://groups.google.com/forum/?fromgroups=#!topic/play-framework/O7_cAdUWQII
One solution seems to be to use Nexus. You will have to deploy from maven to nexus. You will have to use the nexus path instead of mvn. You will have to install and run nexus!
To install nexus go to sonatype and download. Watch file permissions (read the instructions) but it is simple. You will need to put the credentials in ~/.m2/settings.xml. Default is admin, admin123.
<settings>
<servers>
<server>
<id>snapshots</id>
<username>admin</username>
<password>admin123</password>
</server>
</servers>
</settings>
The maven deploy is given to you by nexus, e.g.:
<distributionManagement>
<repository>
<id>releases</id>
<url>http://0.0.0.0:8081/nexus/content/repositories/releases</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<url>http://0.0.0.0:8081/nexus/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>
Then mvn deploy will put your resource there.
Then in the play sbt use
resolvers += "Local Nexus Repository" at "http://0.0.0.0:8081/nexus/content/repositories/snapshots"
You still need to stop play, use play update, and restart play.
You can use:
play reload // Reload the current application build file
play update // Update application dependencies
before building your application. I don't know if you can configure sbt to not cache the SNAPSHOT dependencies, but you can script your building process to force reloading dependencies.
I'm not sure how this works, but "another guy told me" - yes, that's the extent of my references for this - that cleaning out the "repository" folder in the play installation might help.
I have a little "refresh.sh" script that does this:
rm -rf /opt/play/repository/cache/com.mycompany
play clean
play update
play run
It seems to work for me. Where "/opt/play" is where you have your play installation and "com.mycompany" is what you need to refresh.
I'm not saying this is right, but it might be worth of shot if nothing else works.
As of sbt version 0.13.6 (Aug 2014), one can use build settings flag updateOptions in Build.scala/build.sbt, to control SNAPSHOT resolution.
updateOptions := updateOptions.value.withLatestSnapshots(false/true)
Documentation about this new feature is here
Corresponding pull request on github for details.
Related
I have a scala library that I just converted from gradle to sbt.
By default it works by publishing to Sonatype upon release. However I also want to publish it to Bintray. The problem is that Bintray sbt plugin is overwriting the original publish to Sonatype.
I know I can sync to Sonatype and Maven central repository via Bintray. However I still like the way Sonatype handle the validation and check before I really can release it to Maven central.
How do I publish to both Sonatype and Bintray from my release server (not relying on Bintray to sync for me)?
I ran into the same problem and found a setup that works.
sbt-bintray supports a JVM property flag sbt.sbtbintray, When it is set to false, sbt-bintray will not overwrite the publishTo setting (and a few others). So to publish to both sonatype, just run the sbt publish once with the flag set to true and once to false.
However, I also use the sbt-ci-release plugin, which also overrides the publishTo setting (after bintray), but does not offer a flag to disable this. To workaround this, copy what sbt-bintray does into your own build:
publishTo := {
val old = publishTo.value
val p = (publishTo in bintray).value
if (BintrayPlugin.isEnabledViaProp) p
else old
}
Also see the build:
https://github.com/JetBrains/intellij-compiler-indices/tree/master/sbt-idea-compiler-indices
I have an SBT project that depends on two snapshot dependencies. Every time I build it, it goes off to the remote repository to fetch the dependencies. This is true even if I set offline := true.
When I look at how it is trying to resolve the local dependencies, the build is saying it is looking in "local", i.e., ~/.ivy2/local/... -- which is a nonexistent directory.
The jars are in ~/.ivy2/cache/... and this is where SBT downloads them when it pulls the dependencies from the remote server.
I have searched my .sbt and .scala build files and the string "local" does not appear in them in connection with a repository or cache.
SBT is at version 0.13.11 building against scala 2.11.8.
Why is SBT doing this, and how can I get it to see the cached jars?
If you want to prevent SBT from trying to download from official repositories you could simply create a file project/offline-repositories:
[repositories]
mirror-central: file:////nexus/central
mirror-maven-central-org: file:////nexus/maven-central-org
...
(/nexus/central and /nexus/maven-central-org should contain a (partial) mirror of what you need offline)
Then call sbt with the sbt.repository.config property configured:
-Dsbt.override.build.repos=true \
-Dsbt.repository.config=./project/offline-repositories
For Reference:
http://www.scala-sbt.org/0.13/docs/Proxy-Repositories.html
How to prevent SBT from trying to download from official repositories?
EDIT
If you want to use your ~/.m2 cache:
[repositories]
mirror-central: file:////home/XXXXX/.m2/repository
mirror-maven-central-org: file:////home/XXXXX/.m2/repository
...
This apparently is because in my Ivy cache I had a file named ~/.ivy2/cache/com.xxx/xxx-utils/ivy-2.3.2-SNAPSHOT.xml.original , which the build was trying and failing to parse. I'm not sure where this file came from; conceivably it was put there manually ages ago.
I'm not an expert with sbt so probably my question is a bit noob, but I've notice than when I create a project and download its dependencies with sbt, if I open the project with intellij, all the dependencies are redownloaded again, the same happen in the inverse orden intellij->sbt and also activator..
my (poor) knowledge about sbt is than this use ivy and the dependencies are downloaded in ~/.ivy2/ folder...that is where sbt is downloading my deps, but seems than intellij is using other folder.
personally I don't use so much activator, but I would like configure sbt and intellij for use the same ivy path...
2)recently I publish finagle-postgre to my local ivy using sbt +publishLocal, I can check in my ivy folder
/home/yo/.ivy2/local/com.twitter/finagle-postgres_2.11/0.1.0-SNAPSHOT
but unfortunately intellij is unable to resolve this dependency, I try adding this line to my build
resolvers += Resolver.file("Local", file( Path.userHome.absolutePath + "/.ivy2/local"))(Resolver.ivyStylePatterns)
but seems not works
3) the path where is downloaded the dependencies is related to which sbt-launch.jar file is used? How can I know what sbt-lauch.jar file is using sbt right now...
thanks guys!
If we're talking about IntelliJ appearing to download artifacts after they've already been downloaded by SBT/Activator, then it turns out that it's probably just that IntelliJ is downloading the sources - it's not redownloading the binary artifacts, just the source artifacts that accompany them.
This isn't readily apparent when you're looking at the Refreshing SBT project task in the Background Tasks popup, because the full download path is truncated, so you see something like this:
[info] downloading https://repo1.maven.org/maven2/org/apache/httpcompo...
..it's natural to assume that this is the same binary artifact you already saw SBT download on the console, but you can see the full story if you check the full log (go Help -> Show Log in files and open sbt.last.log in the file browser).
You'll see that the only artifacts getting downloaded end with -sources.jar:
$ grep repo1.maven.org /home/roberto/.IntelliJIdea2016.3/system/log/sbt.last.log
[info] downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.3.6/httpclient-4.3.6-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/com/googlecode/javaewah/JavaEWAH/0.7.9/JavaEWAH-0.7.9-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/org/pegdown/pegdown/1.2.1/pegdown-1.2.1-sources.jar ...
[info] downloading https://repo1.maven.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3-sources.jar ...
```
If you don't have the Sources checkbox checked when you're doing Import project, these source downloads won't happen.
tested using IntelliJ 2016.3.5 and Scala plugin v2016.3.9
First, the activator is just a launcher for SBT itself, so there should be no difference in behaviour.
Second, IntelliJ also uses the files in ~/.ivy2 by default if you have not told it otherwise (by setting SBT_OPTS environment variable for example, but that depends on your IntelliJ version).
A difference might result if you're using different scala versions (e.g. 2.10.x vs. 2.11.x) when you do not have set the scalaVersion in your project explicitly. Then, each tool would download the corresponding libraries for the appropriate scala version it has configured by default.
Another thing is that IntelliJ will download source and javadoc jars for each dependency if you have enabled that in your settings which might look like it downloads the dependencies again.
Note, I'm wildly guessing here because you have not included any output of the programs you're using, so it's hard to say what the real problem is.
How I can delete my project from local repository? I previously published it using publish-local SBT command.
I want to clean all compiled and cached stuff because I don't see any changes in my project after recompiling it and redeploying on server.
If you want to retry sbt publishLocal, add your module version x.x.x-SNAPSHOT.
Run publishLocal again and note where it writes the published jars to. For me, it was ~/.ivy2/local. Removing this directory cleared the locally published repository.
There is a sbt command clean. if you need you can add additional folders to clean task in your build file
cleanFiles <+= baseDirectory { base => base / "temp" }
I've configured SBT (0.11.0) to pull in a GitHub project as a dependency, as per my answer on this question here.
It works fine except that I can't seem to get SBT to re-compile my Git dependency when it gets updated. In other words: if I make an update to the dependency, push to Git and reload my project's SBT and run package, then SBT does not recompile the external Git dependency when compiling my project.
I've tried creating a new branch in my Git dependency (say, forcenew) and updating the branch in my SBT project configuration to use this:
lazy val depProject = RootProject(uri("git://github.com/me/dep-project.git#forcenew"))
But even this doesn't force a refresh. I'm a bit stumped - I can't even find where SBT puts the Git project to compile it (it doesn't seem to be in ~/.sbt/ or ~/.ivy2/)...
Any help greatly appreciated!
From: https://github.com/sbt/sbt/issues/335
this should be fixed in 0.12.0, just call "sbt update"
It was fixed in 0.12.0 so sbt update is enough, but got back in 13.0 -- for now, you have to wipe dependency from ~/.sbt/staging/ manually
You likely want to clear out ~/.sbt/staging/
A quick hack you can add to your build.sbt:
def removegit = Command.command("removegit"){state =>
val home = sys.env("HOME")
val k = ("rm -rf "+ home + "/.sbt/0.13/staging/").!
state
}
commands ++= Seq(removegit)
And then sbt removegit will wipe that directory. This doesn't do anything smart like checking commits, which would be a great upgrade... The repos are being stored in ~/.sbt/0.13/staging/ on my machine, you may need to adjust that.