I've been wracking my brain over this for the last two days trying to get it to work. I have a local Spark installation on my Mac that I'm trying to attach a debugger to. I set:
SPARK_JAVA_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005
Then I submit my job to spark-submit and launch my debug configuration in eclipse which is configured as a Socket Attach Remote Debugging session. The debugger attaches, my job resumes and executes, but none of my breakpoints are ever hit, no matter what I do.
The only way I can get it to hit a breakpoint is by attaching to a spark-shell, creating a Java Exception breakpoint and issuing
throw new java.lang.Exception()
The debugger will not stop at normal breakpoints for me.
I created a standalone Hellow World scala app and was able to attach to it and have it stop at a regular breakpoint without any issues.
Environment: Mac OS, latest Eclipse, latest Scala IDE, Spark 1.3.1, Scala 2.10.5
Thanks in advance.
I had a similar issue and there were 2 things that fixed my problem -
1. The .jar file and source not a little out of sync for me , so had to recompile and redeploy.
2. Next On the JAVAOPTS I had a suspend=n.
After correcting these two it worked for me .
Related
I'm working with Stainless, a software verifier for Scala programs. I would like to debug the verification process of a sample programme on Intellij Idea. On a previous post, I solved this integration problem for an interactive theorem prover. But now, I'm facing two problems:
Apparently, the verification software runs at compile time. That is, I enter in the sbt console and run the compile command and then the verification process seems to be done. You may try this with this verified example. This situation is new to me, since I was used to debug the program while executing.
All the setup in the sbt files of the example above (see for instance this file) seem to refer to online content, while I want to make sure that I work with my local copy forked from the original repository of the verifier.
None of the configurations I tried worked. Can you help me out of this problem?
Details
This is the current configuration page of stainless.
If the verification runs within the sbt process, you can debug it by attaching the debugger to sbt. IntelliJ makes this easy with the embedded sbt shell:
open the sbt shell toolwindow
click the "attach debugger to sbt shell" button on the left
set breakpoints in your code
run the task
I am working on a spark project using scala and maven, and some time I feel it would be very helpful if I can ran the project in an interactive mode.
My question if it is possible (and how) to bring up a spark environment in terminal that same as the environment running in a IntelliJ project?
Or even better (if it is possible) -- start a PERL environment, under IntelliJ debug model, during code ceased running at a break point. So we can continue play with all variables and instances created so far.
Yes, it is possible, though not very straightforward. I first build Fat jar using sbt assembly plugin (https://github.com/sbt/sbt-assembly) and then use a debug configuration like the one below to start it in debugger. Note that org.apache.spark.deploy.SparkSubmit is used as a main class, not your application main class. You app main class is specified in the --class parameter instead.
It is a bit tedious to have to create app jar file before starting each debug session (if sources were changed). I couldn't get SparkSubmit to work with the compiled by IntelliJ class files directly. I'd be happy to hear about alternative ways of doing this.
*Main class:*
org.apache.spark.deploy.SparkSubmit
*VM Options:*
-cp <SPARK_DIR>/conf/:<SPARK_DIR>/jars/* -Xmx6g -Dorg.xerial.snappy.lib.name=libsnappyjava.jnilib -Dorg.xerial.snappy.tempdir=/tmp
*Program arguments:*
--master
local[*]
--class
com.example.YourSparkApp
<PROJECT_DIR>/target/scala-2.11/YourSparkAppFat.jar
<APP_ARGS>
If you don't care much about initialization or can insert a loop in the code where the app waits for a keystroke or any other kind of signal before continuing, then you can start you app as usual and simply attach IntelliJ to the app process (Run > Attach to Local Process...).
I've been having trouble getting a weblogic server to run in eclipse, which have resulted in me not being able to open eclipse ni my current workspace at all. It is not my first attempt to work with weblogic servers in eclipse, and I do not know why it suddenly decided to stop working. I have had multiple errors, which all seem to be related to an abnormal high memory consumption.
One of my last errors, before eclipse quit on me for good, was an out of memory exception, on PermGenSpace. I set variables to 1024m, but it still complained it wasn't enough. After a lot of restarts, Eclipse suddenly wouldn't open my current workspace. I have tried others, and they work fine, but with my current one, the splash/loading screen disappears after about 10% of the loading bar is complete, and nothing happens at all. Except the fact that eclipse.exe shows up in the running processes tab on Windows task list. It does not show in the running programs list.
My attempts to start weblogic servers had the exact same symptoms. They showed in the process list, but did not respond when I tried to open the console in my browser.
Both eclipse and the weblogic tasks (shown as java.exe) has had over 400.000KB of memory consumption each, which I find very high.
I do not expect anyone to be able to find a complete solution here, but I am absolutely stuck. I cannot access any of my previous error messages. I have no experience debugging an error like this. Does anyone have any idea on how to find the error in a case like this?
My Weblogic version is 10.3.2, which is a company standard for now.
My Eclipse version is:
Eclipse Java EE IDE for Web Developers.
Version: Kepler Service Release 2
Build id: 20140224-0627
Adding the comment as an answer since it provided some help.
Try starting up to a new empty workspace; If you dont get to select workspace at startup use the argument "-data " If you still get issues, look at the log in /.metadata/.log
I have recently started developing an Eclipse plugin (which is basic stuff for now) and I am struggling with "default" way to run Eclipse plugin ("Run as Eclipse application").
The Eclipse is starting another instance with my plugin already installed in it (this is default behaviour).
The problem is that when I want to re-run my plugin project and I press "run" button again (or Ctrl + F11) (and the another Eclipse instance still running) I get following message:
"Could not launch the application because the associated workspace is currently in use by another Eclipse application".
The error makes sense, and when I close "testing" Eclipse instance I am able to run my plugin again.
The question is - "is it normal routine for plugin development?". Maybe I am missing something, e.g. special arguments for Eclipse?
This seems all pretty normal. The error message is since the run configuration is specifing a workspace and when you start a second instance using the same workspace it is locked and considered in use.
What I usually do when testing a plugin is to create a run configuration (click "Run...") where I disable all the plugins I wont need when testing. This makes sure that the test starts up a couple of seconds quicker. Make sure you save that run configuration as a *.launch file aswell, that makes it quicker to test the next time. Or it can be used to share the configuration.
There's a lot you can configure in the run configuration, such as eclipse arguments, vm argument, if you want environment variables set, etc. So be sure to experiment a little.
In your run configuration. Main tab->Workspace Data ->Location text box add this:
${workspace_loc}/../runtime-EclipseApplication${current_date:yyyyMMdd_HHmmss}
Note the suffix ${current_date:yyyyMMdd_HHmmss} by this every time you launch your application new workspace will be created. So you will not get any error message saying workspace is locked.
But be careful as the folder .metadata will be different for different instances as their work-spaces are different. Thus preferences stored/retrieved by different instances are NOT in sync.
You are probably missing one important point: Eclipse supports the Java hot code replacement. Therefore in many cases you can modify your Java code while your application Eclipse instance is running, save the code and continue without restarting.
If hot code replacement is not possible, Eclipse will tell you, so you always know whether the editing changes are applied to the running instance.
This works best with more recent versions of the JVM, so consider upgrading to the latest Java 7 version, even if you write code to be compliant with Java 1.5 or 6.
"Hot Code Replace Failed - add method not implemented".
I get this error message every time I change something in my test class (and save it).
Can't figure out what it means. Can somebody help?
Possibly, you have a test which is still running (in debug mode). Try finishing all tests (you can see them in the debug view: window->show view->debug) and try again...
See this thread:
This means you changed a class while it was debugging an application and it could not update the class for the application while it was running.
The error suggests you may be running an older JVM, i.e. pre-1.4.2 but this error can occur with any JVM if the change is incompatible with the previous version of the class.
Check carefully what JRE you are using in your debug session.
Also check you are deploying classes compiled with the debug attribute set. (see this thread)
Finally, as indicated here:
did you switch "Project->Build Automatically" off?
Hot code replacement works (only?) if automatic build is switched on...
I was running into this issues too. I found a build hiding in the backgound that was giving me fits. Check to make sure you have all tests/builds closed or completed. Nonetheless, you can still run the new code.
Whenever this error message appears it also gives options to terminate or restart. Select Terminate and it will terminate any running debug case which you are unable to locate.
if you have multiple projects involved, and multiple jre's installed, make sure all dependent projects are using the same javac/jre
Make sure your application server JRE and Eclipse JRE version should be same or else it will throw unsupported class version and hot code debug will not work.
I went to the same Problem,But my Debugging session ended hours ago.But still ECLIPSE error kept coming.
So I just simply restart the ECLIPSE IDE, then the problem just solved.