My plan is to take an existing project and slowly transform it to something else. To this aim I took the brutal approach and simply copied the folder of the project as it was to a new folder.
When I ran sbt in the new folder, it resolved some dependencies and recompiled all the sources. This is kind of expected, because the project registry may have contained reference to the absolute path where the project resided earlier and sbt may have detected that it changed.
However when I went back to the old project folder, to which I made no modification and ran sbt in that folder, it recompiled all the sources as well in the original project.
This is a puzzle for me. Why did this happen? Is there a magic link between projects residing in different folders that I'm missing?
The simplest explantion might be that compilation was not up-to-date when you copied the original project. If that was the case, copying the project would have updated the file modified dates and triggered a recompilation for the new folder. Similarly, when going back to the old folder again, it would still be out-of-date, and sbt would recompile all sources again.
Related
Newcomer to the Intellij IDE here, with no Java background. I've looked at Build Definition to get a brief idea on how should I organize my scala files, but their example doesn't cover the full structure of an SBT-based project shown attached.
Can you advise what each folder should be used for (e.g. where my source files should go, etc.) and also point me to sources where I can go read up more.
Thanks so much
It is described pretty well here:
http://www.scala-sbt.org/0.13.5/docs/Getting-Started/Directories.html
But to sum up.
.idea:
This contains the project files for your idea project, and has nothing directly to do with sbt itself. However idea (if auto refresh is enabled) updates its own project, each time the sbt build files change.
project:
This contains the sbt project files, except for the main build file (files ending in .sbt). Sbt build is itself based on scala, and if you need to have some scala code included in your build (e.g., code-generation/meta-programming, pre-compiler macros), then you can place scala source files in this directory. The code of these files can be used in your build system, and is not part of your project itself. To really understand how a build is made, then you will need to understand the difference in how sbt files and scala files for the build should be placed. When you run sbt, then it will search for .sbt files in the directory your are standing in, when these are found, it will search for scala files in the project directory. These files together are the source of the build system, but because these are source files, they need to be built before they can be used. To build this build system, sbt uses sbt. So a build system to build the build system is needed. It therefore looks for sbt files inside the project directory, and scala files for this build inside project/project and build these files to get a build system, that can build the build system (that can build your project). Actually it can continue recursive down to any project/project/project... directory, until it finds a project folder containing no scala files, and therefore needs no building before use.
The target folder inside project, is the target folder for the sbt build of your build definition. See below what a target folder is.
Normally you would not need to be concerned about this; just remember that build.sbt in your root directory is the build script for your project. project/plugins.sbt defines plugins activated for your build system, and project/build.properties contains special sbt properties. Currently the only sbt property I now of, is what version of sbt should be used.
src:
This is where your place the source files of your project. You should place any java sources in src/main/java, scala sources in src/main/scala. Resources are placed in src/main/resources.
The src/main/scala_2.11 folder is typically used, if you have some code that it not binary compatible with different versions of scala. In such cases you would be able to configure sbt to use different source files when building for different versions of scala. You probably do not need this, so I would advise to just delete the src/main/scala_2.11 folder.
Test sources are placed inside src/test/java and source/test/scala, and test resources are placed in src/test/resources.
target
This folder is the target folder for sbt. All compiled files, generated packages and so on are placed somewhere inside this dir.
Most things in this dir are not so interesting, as most of it is just internal sbt things. However if your build a jar file by calling sbt package, then it will be placed inside target/scala-x (where x is the scala version). There are also a lot of different plugins, that can package your application in different ways, and they will normally also place the package files somewhere inside the target dir.
My problem is best described with a screenshot:
As you can see, there is a org.virtuslab:unicorn-core_2.11 dependency loaded in my SBT/Scala project. The jar file with corresponding sources is available as well. For some reason, Intellij Idea decided to ignore the available source file Table.scala and uses the .class file instead. Clearly, all other files are okay.
Obviously, without the information contained in the .scala file, Intellij is not able to properly analyze my code and I'm getting Cannot resolve symbol BaseIdRepository errors (and similar) all over my project.
I manually checked the source package and the Table.scala file is there and seems alright. Plus this worked flawlessly until two days ago. I don't know why it broke down. I tried to revert my project using git to an earlier version when this was still working, but no luck.
I tried to both downgrade and upgrade the library, I tried to remove it and add it again, I tried create a completely new project based on existing sources, but once again, no luck.
Just for the record, I'm able to compile and run my project. The only thing that might be related is this warning that I get when compiling the project:
[warn] /home/tobik/scala/payola-viz/src/app/model/appgen/repository/UsersRepository.scala:7: Could not find any member to link for "BaseIdRepository".
I got an update for the Scala plugin which seems to fix this issue.
In case you experience similar problems, this temporary solution worked for me: I added the unicorn sources directly to my code base and manually removed it from the project libraries. I got rid of the errors and it still compiles as the library remains as a dependency inside built.sbt. To avoid polluting my project, I added the included sources to .gitignore. The workaround broke after a while but all I had to do was repeat the procedure again.
TL;DR: How can I fully reconcile all dependencies in IntelliJ when SBT file changes
I have a SBT project setup in IntelliJ. For the most part if works like a charm, but it is quite a pain for me, to change the version of the dependencies and compilers.
What I hope to do, is to update my SBT file, and click something like clean project in IntelliJ, so it gets up to date.
The best menu item I have found is Rebuild project. It reads new dependencies, but keeps the old ones around, so External Libraries contains multiple versions of the Scala runtime and 3rd party libraries.
To get around this I can close IntelliJ and delete all files in .idea/libraries. When I restart IntelliJ it will re-resolve the dependencies. It works, but manually deleting metadata files indicates that I am doing the wrong thing.
To refresh IntelliJ after changes made in *.sbt files:
Open the SBT Project pane (Menu View / Tool WIndows / SBT)
Right click on your SBT project
Select Refresh external project
When you save the SBT file, IntelliJ IDEA normally refreshes the project. But if it doesn't, you can do it manually.
There is a Refresh All SBT Projects button in the SBT Panel:
On the occasion that IDEA is still confused, not finding packages you just added:
From the File menu:
It's a pretty big hammer, requiring a restart of the app, and multi-minute rebuild of the IntelliSense index. But it's occasionally the only thing that will get IDEA to recognize your new SBT packages.
try following steps:
close IntelliJ
backup whole project folder
delete .idea folder
delete target folder
delete project/target folder
reopen IntelliJ and import as a sbt project
Note this is only for SBT projects
Another heavy hammer but seems to work.
Close IntelliJ
Delete .idea folder
Import app from existing sources
Has happened to me before, and yes, is a PITA.
What I usually do is to manually remove the dependencies from the project, all of them, then let sbt re-import what is actually needed. Don't do it on the UI though, since even having multiple selection, will ask for confirmation for EVERY dependency! (you will be clicking "Yes, I'm sure" for an hour), instead, go and delete directly from the .iml project file on your .idea/modules directory.
I was using different versions of sbt and scala across different proejcts and IntelJi got confused somehow. I am only able to resolve the issue by removing the sbt and ivy2 cache folders:
rm -rf ~/.ivy2
rm -rf ~/.sbt
Agree, that's a PITA. Nothing helped me (I tried "Invalidate/Restart", removing files from ".idea/libraries" and restarting).
The only way to overcome the problem for me was to manually remove project-related folders from the IntelliJIdea cache. I removed these folders from the following places (I use OSX and IntelliJ-Idea 14):
/Users/{UserName}/Library/Caches/IntelliJIdea14/compiler/{ProjectName}
/Users/{UserName}/Library/Caches/IntelliJIdea14/compile-server/{ProjectName}
This helped me, hope it will save time for someone.
If you are using the activator through a terminal/shell/command prompt use the following command:
activator clean
Or just "clean" on the sbt shell through intellij
It should help reset things and next time you run or compile, it should go through it from scratch.
IDEA 2016.2.4
None of these solutions worked for me.
**** BEFORE YOU TRY THE ANY OF ABOVE. MAKE A BACKUP OF THE SBT / IDEA PROJECT IN QUESTION *****
If I updated the 'build.sbt' then the external library dependencies on the project view did not update. Moreover, the Project Structure still showed the old dependencies. No matter what I tried IDEA would not refresh.
The only way I could get a partial restore was to copy the '.idea' folder from another machine. SBT plugin implementation is ****ed! I removed the '.idea/libraries' folder beforehand and then IDEA did not restore this folder. I tried many times importing the project again.
Very bad
:(
ADDENDUM: I managed to get my project working. I think SBT with IDEA is no longer at fault entirely. It could be just possible that my Apache Ivy 2 cache might have been corrupted. So I executed the following commands:
$ rm -rf ~/.ivy2/cache/org.scalatest/scalatest_2.11/*
$ rm -rf ~/.ivy2/cache/org.scalactic/scalactic_2.11/*
Restarted IDEA with invalidate cache under the file drop down menu item. This seemed to trigger some internal action in IDEA and SBT. I also tried manually adding add JARs throught the IDE, but it got massively confused. Moral of the story is to work on two different laptops machines daily and keep them updated.
Also double check SBT from the command line works. I was using 0.13.12
$ sbt about
$ sbt clean
$ sbt test
Also recheck with the SBT dependencies plugin too.
What worked for me was to:
1) Close IntelliJ
2) delete .idea/libraries internals
3) Start IntelliJ
4) right-clicked pom.xml Maven > Reimport
After that there were no old versions and duplicate libraries in the External Libraries.
I had a package structure which I changed, and this lead to the corresponding folders in /bin/ being removed, so now I get a ClassNotFoundException.
More specifically, I had the ususal /src/ and /bin/, and inside /src/ I had /main and /test, and this was presumably mirrored in /bin/. I shuffled the files around a bit and ended up with /main/model/ and /test/model/.
I have tried to manually create the corresponding folders (mirroring the packages in /src/), but this does not seem to help. I managed to compile the classes in one of the packages, but not in the test package, since it uses ScalaTest* and I didn't manage to compile the classes with the scalatest jar file (or something was wrong with the classpath).
So my question is:
How do I fix this within this peoject? (I don't want to simply create a new project and copy the source files over)
I'm guessing that I should learn more about build systems to be more resilient to such annoyances in the future? If so, what should I read up on, specifically to become better at troubleshooting and having more fine grained control over the build in the context of IDEs in the future (making builds independent of IDEs is not a priority for me at this point)?
An answer to one of these questions would be sufficient.
*All the source files are .scala files, if that might matter.
Udate
I did a clean of the project (project -> clean). This seemed to fix the problem: I was able to run the test classes from within Eclipse. All the binary files were in there, too. I made a new package, main.controller, with one class, and when I tried to run it, it said that it couldn't find the class. I tried to run the tests again, but those gave me a ClassNotFoundException, too. When I looked in /bin/ it turned out that all the folders and files were gone. I've tried to clean the project again but to no avail. I don't understand how I was able to clean the project, but now it can't fix it?
Update 2
To test if this was reproducible with a Java project, I made a Java project with two packages; main and test. I had the main class in main, which used a class from test (so there were dependencies across the packages). It ran succesfully. Then I added packages so that I had main.model and test.model and moved the corresponding files there. It also ran. Then I tried to delete all the files and folders in /bin/, and then the main class would not run. But if I did a clean of the project, then it cleaned it succesfully and the Main class was able to run. Then I made a Git repository for it, placing it outside of /workspace/ (in my git folder) and tried to do the same there. Eclipse was able to clean the project succesfully everytime.
So I don't understand why it can't manage to clean my Scala project.
I had errors in the "Problems" tab (Window -> Show View -> Problems). Now that I've made them go away, my /bin/ is correct and I can run both my Main class and my tests. This Question helped find out what the problem was:
Scala project won't compile in Eclipse; "Could not find the main class."
I have a maven project on eclipse with jrebel plugin installed. Hot-deploy used to work perfectly last week but now only xhtml pages are hot-deploy. When a java class is changed it doesn't hot deploy.
What I noticed is when I changed a file and save eclipse will automatically build it. But the output folder file is not updated base on file stamp that's why jrebel doesn't pick it up. When I run maven-install it compiles everything and all the java classes are reloaded which is not efficient.
So the main problem is eclipse newly compiled classes don't go to output folder (project/target/classes), even though it's set in Build Path.
Any idea?
By default, the content of your local Maven repository is cached for a day. This can happen even for bad downloads (as I experienced). See https://stackoverflow.com/a/7421893/44089 for a short description of how to work around that.
After several minutes of testing, I found a warning on a jar file specifically guava being downloaded as dependency. I've delete in repo to be redownloaded and after that jrebel is working again.
So the problem is a corrupted jar that causes everything to be rebuild even if only a single file is changed. But the weird part is there's no corrupted file error.