In my Java program, i am using "this.PID" to get the current process PID.
I am looking for something similar in Scala - how to get same 6 char unique PID (as by java prgm).
looked through multiple post. Once such post suggest to use "MemoryManagementFactory".
However this returns #Hostname. Though, I am looking for a way to get similar 6 char JVM format PID.
Please suggest. Scala version: 2.11
Scala does not have a special API to get the process id. This is of course limited because of the JVM. Since Scala runs on the JVM, there is no platform independent API that will work no matter what architecture you run your program on. So most of what Scala can provide in this context are wrappers on top of what the Java API offers already in the context of processes.
For example, you could try scala.sys.process but as the documentation says:
The underlying basis for the whole package is Java's Process and
ProcessBuilder classes. While there's no need to use these Java
classes, they impose boundaries on what is possible. One cannot, for
instance, retrieve a process id for whatever is executing.
However, in Windows you can use the ManagementFactory.getRuntimeMXBean.getName method that will result in a string of the form[PID]#[MACHINE_NAME] and you can further process that to get only the PID.
Since JDK 10 you can also directly obtain the PID using ManagementFactory.getRuntimeMXBean.getPid.
But in both cases you will get the PID of the current running JVM, so if you need the PID of some other running program, I don't think that is possible.
Here is the equivalent Scala code:
object Program extends App {
import java.lang.management.ManagementFactory
val bean = ManagementFactory.getRuntimeMXBean
// Get name representing the running Java virtual machine.
// It returns something like [PID]#[MACHINE_NAME]. The value before
// the # symbol is the PID.
val jvmName = bean.getName
System.out.println("Name = " + jvmName)
// Extract the PID by splitting the string returned by the
// bean.getName() method.
var pid = jvmName.split("#")(0).toLong
System.out.println("PID = " + pid)
// Since JDK 10 you can get the process ID representing the running Java virtual machine.
pid = ManagementFactory.getRuntimeMXBean.getPid
System.out.println("PID = " + pid)
}
For me it outputs:
Name = 12412#VALHALLA
PID = 12412
PID = 12412
VALHALLA is the name of my host machine, while 12412 is the PID of the current JVM instance.
Related
I'm developing a shared library in Jenkins to help with interacting with an internal API. I can make single call which starts a long running process to create an object. I have to continue to query the API to check for the process' completion.
I'm trying to get this done in using a simple loop, but I keep getting stuck. Here's my function to query the API until it's completed:
def url = new URL("http://myapi/endpoint")
HttpURLConnection = http = (HttpURLConnection) url.openConnection()
http.setDoOutput(true)
http.setRequestMethod('POST')
http.setRequestProperty("Content-Type", "application/x-www-form-urlencoded")
def body = ["string", "anotherstring"].join('=')
OutputStreamWriter osw = new OutputStreamWriter(http.outStream)
osw.write(body)
osw.flush()
osw.close()
for(int i = 0; i < 30; i++) {
Integer counter = 0
http.connect()
response = http.content.text
def status = new JsonSlurperClassic().parseText(response)
// Code to check values here
}
When I run this through a pipeline, the first iteration through the loop works fine. The next iteration bombs with this error:
Caused: java.io.NotSerializableException: sun.net.www.protocol.http.HttpURLConnection
I just started in Groovy, so I feel like I'm trying to do this wrong. I've looked all over trying to find answers and tried several things without any luck.
Thanks in advance.
When running a pipeline job, Jenkins will constantly save the state of the execution so it can be paused and later on resumed, this means that Jenkins must be able to serialize the state of the script and therefore all of the objects that you create in your pipeline must be serializable.
If an object is not serializable you will get the NotSerializableException whenever Jenkins will attempt to serialize your un-serializable object when it save the state.
To overcome this issue you can use the #NonCPS annotation, which will cause Jenkins to execute the function without trying to serialize it. Read more info on this issue at pipeline-best-practice.
While normal Pipeline is restricted to serializable local variables (see appendix at bottom), #NonCPS functions can use more complex, nonserializable types internally (for example regex matchers, etc). Parameters and return types should still be Serializable, however.
There are however some limitations so read the documentation carefully, for example the return value types from #NonCPS methods must be serializable and you can't use any pipeline steps or CPS transformed methods inside a #NonCPS annotated function. Additional info can be found here.
One last thing, to overcome all these issues you can also use the Jenkins HTTP Plugin which includes all the HTTP abilities you will probably need wrapped in an easy to use build in interface.
So, I have the following kind of code running in each map tasks on Spark.
#volatile var res = (someProgram + fileName) !
var cmdRes = ("rm " + fileName) !;
The filenames for each map tasks are unique. The basic idea is that once the first command finishes, the second command deletes the file. However, I notice that the program sometimes complain that the file does not exist. It seems that the subprocess call is not synchronous, that is, it does not wait for the subprocess to complete. Is that correct. And if it indeed the case, how can we correct that?
As you can see in the docs, the ! method blocks until exit. The docs say that it:
Starts the process represented by this builder, blocks until it exits, and returns the exit code.
It's possible that you should be checking the exit code to interpret the result and to deal with exceptional cases.
When creating process commands by concatenation, you are often better off using the Seq extensions (as opposed to the String ones) to create a ProcessBuilder. The docs even include this helper, which might help you:
// This uses ! to get the exit code
def fileExists(name: String) = Seq("test", "-f", name).! == 0
In Scala before 2.10, I can set the parallelism in the defaultForkJoinPool (as in this answer scala parallel collections degree of parallelism). In Scala 2.10, that API no longer exists. It is well documented that we can set the parallelism on a single collection (http://docs.scala-lang.org/overviews/parallel-collections/configuration.html) by assigning to its taskSupport property.
However, I use parallel collections all over my codebase and would not like to add an extra two lines to every single collection instantiation. Is there some way to configure the global default thread pool size so that someCollection.par.map(f(_)) automatically uses the default number of threads?
I know that the question is over a month old, but I've just had exactly the same question. Googling wasn't helpful and I couldn't find anything that looked halfway sane in the new API.
Setting -Dscala.concurrent.context.maxThreads=n as suggested here: Set the parallelism level for all collections in Scala 2.10? seemingly had no effect at all, but I'm not sure if I used it correctly (I run my application with 'java' in an environment without 'scala' installed explicitly, it might be the cause).
I don't know why scala-people removed this essential setter from the appropriate package object.
However, it's often possible to use reflection to work around an incomplete/weird interface:
def setParallelismGlobally(numThreads: Int): Unit = {
val parPkgObj = scala.collection.parallel.`package`
val defaultTaskSupportField = parPkgObj.getClass.getDeclaredFields.find{
_.getName == "defaultTaskSupport"
}.get
defaultTaskSupportField.setAccessible(true)
defaultTaskSupportField.set(
parPkgObj,
new scala.collection.parallel.ForkJoinTaskSupport(
new scala.concurrent.forkjoin.ForkJoinPool(numThreads)
)
)
}
For those not familiar with the more obscure features of Scala, here is a short explanation:
scala.collection.parallel.`package`
accesses the package object with the defaultTaskSupport variable (it looks somewhat like Java's static variable, but it's actually a member variable of the package object). The backticks are required for the identifier, because package is a reserved keyword. Then we get the private final field that we want (getField("defaultTaskSupport") didn't work for some reason?...), tell it to be accessible in order to be able to modify it, and then replace it's value by our own ForkJoinTaskSupport.
I don't yet understand the exact mechanism of the creation of parallel collections, but the source code of the Combiner trait suggests that the value of defaultTaskSupport should percolate to the parallel collections somehow.
Notice that the question is qualitatively of the same sort as a much older question: "I have Math.random() all over my codebase, how can I set the seed to a fixed number for debugging purposes?" (See e.g. : Set seed on Math.random() ). In both cases, we have some sort of global "static" variable that we implicitly use in a million different places, we want to change it, but there are no setters for this variable => we use reflection.
Ugly as hell, but seems to work just fine. If you need to limit the total number of threads, don't forget that the garbage collector runs on separate thread.
I'm trying to write an ABL binding for the ØMQ C library, but am crashing the Procedure Editor for my zmq_init function binding. I wrote a binding for zmq_version without issue, so I assume it's not a .dll file problem. Anyway, the C function prototype looks like this:
void *zmq_init (int io_threads);
The ABL code I've written is as follows:
PROCEDURE zmq_init EXTERNAL "libzmq.dll" CDECL:
DEFINE INPUT PARAMETER num_threads AS LONG.
DEFINE RETURN PARAMETER context_ptr AS MEMPTR.
END PROCEDURE.
DEF VAR mContext AS MEMPTR NO-UNDO.
RUN zmq_init(INPUT 0, OUTPUT mContext).
This particular function initializes a "0mq context" (potential problem: it's a thread pool) and returns a pointer to it. Could the error be related to OpenEdge's non-threaded nature, even though I'm calling an external library and requesting 0 threads be allocated in the pool?
Running the code in Procedure Editor makes it crash with a Windows "Progress Client has stopped working" error with Exception code: C0000005 ACCESS_VIOLATION (full stack trace here, although I don't always get a stack trace with info in it).
I've never done Windows C programming, but it looks like what I'd call a Segmentation Fault if I were on Unix using gcc (accessing protected memory). Speaking of which, I'd in fact rather run this code on a Unix machine but apparently Progress only provides evaluation developer OE environments for Windows :(.
I've disabled DEP without success. Is ABL trying to dereference the pointer when it returns from the library or something? Surely there has to be a way to handle signatures with a return value of void* that get malloc'ed in the library call.
I've paged through OpenEdge Development: Programming Interfaces, Tom Bascom's UNIX Shared Libraries PowerPoint, and some other PowerPoint called OpenEdge in an LDAP World that has some code samples but don't see anything obvious that I'm missing.
I'm on OpenEdge 10.2B, Windows 7 64-bit, however using the 32-bit version of OpenEdge as that's my only option for evaluation software. Anyone got a license I could borrow for compiling/running this code on CentOS? :-)
You are missing PERSISTENT keyword when you are declaring your procedures.
Please try the following code:
zmq.p
&SCOPED DLLNAME 'libzmq.so'
PROCEDURE zmq_init EXTERNAL {&DLLNAME} CDECL PERSISTENT:
DEF INPUT PARAMETER NumThreads AS LONG.
DEF RETURN PARAMETER Context AS MEMPTR.
END PROCEDURE.
PROCEDURE zmq_term EXTERNAL {&DLLNAME} CDECL PERSISTENT:
DEF INPUT PARAMETER Context AS MEMPTR.
DEF RETURN PARAMETER ResultStatus AS LONG.
END PROCEDURE.
PROCEDURE zmq_version EXTERNAL {&DLLNAME} CDECL PERSISTENT:
DEF OUTPUT PARAMETER major AS LONG.
DEF OUTPUT PARAMETER minor AS LONG.
DEF OUTPUT PARAMETER patch AS LONG.
END PROCEDURE.
check_zmq.p
DEF VAR Zmq AS HANDLE NO-UNDO.
DEF VAR MajorVersion AS INT NO-UNDO.
DEF VAR MinorVersion AS INT NO-UNDO.
DEF VAR PatchVersion AS INT NO-UNDO.
DEF VAR CallStatus AS INT NO-UNDO.
DEF VAR ZmqContext AS MEMPTR NO-UNDO.
RUN zmq.p PERSISTENT SET Zmq.
RUN zmq_version IN Zmq ( OUTPUT MajorVersion, OUTPUT MinorVersion, OUTPUT PatchVersion ).
MESSAGE MajorVersion MinorVersion PatchVersion.
RUN zmq_init IN Zmq ( 1, OUTPUT ZmqContext ).
RUN zmq_term IN Zmq ( ZmqContext, OUTPUT CallStatus ).
MESSAGE CallStatus.
FINALLY:
IF VALID-HANDLE(Zmq) THEN
DELETE PROCEDURE Zmq.
END FINALLY.
Could the error be related to OpenEdge's non-threaded nature, even though I'm calling an >external library and requesting 0 threads be allocated in the pool?
I ran into the same problem several times (Progress Client has stopped working) while i was working on some c# methods (called via clr bridge) in which I used threads.
Hiding the threads from progress by using some c# library classes(AsyncOperation,AsyncCallback) solved the problem, but most attempts caused the progress runtime to stop.
Related to .net the progress help states "You cannot use System.Threading.Thread, or any derived class — ABL is single-threaded."
I know that calling c# methods via clr bridge is completely different from invoking c libraries but maybe our problems are caused by the single threaded nature of OpenEdge.
I grabbed your code from github and changed the MEMPTR variables to INT64. This allowed it to run without crashing (10.2b05 32-bit on 64-bit Windows 7). Since the memory is being handled by zeromq, I think this is a safe way to handle the issue.
I just had a look at the new scala.sys and scala.sys.process packages to see if there is something helpful here. However, I am at a complete loss.
Has anybody got an example on how to actually start a process?
And, which is most interesting for me: Can you detach processes?
A detached process will continue to run when the parent process ends and is one of the weak spots of Ant.
UPDATE:
There seem to be some confusion what detach is. Have a real live example from my current project. Once with z-Shell and once with TakeCommand:
Z-Shell:
if ! ztcp localhost 5554; then
echo "[ZSH] Start emulator"
emulator \
-avd Nexus-One \
-no-boot-anim \
1>~/Library/Logs/${PROJECT_NAME}-${0:t:r}.out \
2>~/Library/Logs/${PROJECT_NAME}-${0:t:r}.err &
disown
else
ztcp -c "${REPLY}"
fi;
Take-Command:
IFF %#Connect[localhost 5554] lt 0 THEN
ECHO [TCC] Start emulator
DETACH emulator -avd Nexus-One -no-boot-anim
ENDIFF
In both cases it is fire and forget, the emulator is started and will continue to run even after the script has ended. Of course having to write the scripts twice is a waste. So I look into Scala now for unified process handling without cygwin or xml syntax.
First import:
import scala.sys.process.Process
then create a ProcessBuilder
val pb = Process("""ipconfig.exe""")
Then you have two options:
run and block until the process exits
val exitCode = pb.!
run the process in background (detached) and get a Process instance
val p = pb.run
Then you can get the exitcode from the process with (If the process is still running it blocks until it exits)
val exitCode = p.exitValue
If you want to handle the input and output of the process you can use ProcessIO:
import scala.sys.process.ProcessIO
val pio = new ProcessIO(_ => (),
stdout => scala.io.Source.fromInputStream(stdout)
.getLines.foreach(println),
_ => ())
pb.run(pio)
I'm pretty sure detached processes work just fine, considering that you have to explicitly wait for it to exit, and you need to use threads to babysit the stdout and stderr. This is pretty basic, but it's what I've been using:
/** Run a command, collecting the stdout, stderr and exit status */
def run(in: String): (List[String], List[String], Int) = {
val qb = Process(in)
var out = List[String]()
var err = List[String]()
val exit = qb ! ProcessLogger((s) => out ::= s, (s) => err ::= s)
(out.reverse, err.reverse, exit)
}
Process was imported from SBT. Here's a thorough guide on how to use the process library as it appears in SBT.
https://github.com/harrah/xsbt/wiki/Process
Has anybody got an example on how to
actually start a process?
import sys.process._ // Package object with implicits!
"ls"!
And, which is most interesting for me:
Can you detach processes?
"/path/to/script.sh".run()
Most of what you'll do is related to sys.process.ProcessBuilder, the trait. Get to know that.
There are implicits that make usage less verbose, and they are available through the package object sys.process. Import its contents, like shown in the examples. Also, take a look at its scaladoc as well.
The following function will allow easy use if here documents:
def #<<< (command: String) (hereDoc: String) =
{
val process = Process (command)
val io = new ProcessIO (
in => {in.write (hereDoc getBytes "UTF-8"); in.close},
out => {scala.io.Source.fromInputStream(out).getLines.foreach(println)},
err => {scala.io.Source.fromInputStream(err).getLines.foreach(println)})
process run io
}
Sadly I was not able to (did not have the time to) to make it an infix operation. Suggested calling convention is therefore:
#<<< ("command") {"""
Here Document data
"""}
It would be call if anybody could give me a hint on how to make it a more shell like call:
"command" #<<< """
Here Document data
""" !
Documenting process a little better was second on my list for probably two months. You can infer my list from the fact that I never got to it. Unlike most things I don't do, this is something I said I'd do, so I greatly regret that it remains as undocumented as it was when it arrived. Sword, ready yourself! I fall upon thee!
If I understand the dialog so far, one aspect of the original question is not yet answered:
how to "detach" a spawned process so it continues to run independently of the parent scala script
The primary difficulty is that all of the classes involved in spawning a process must run on the JVM, and they are unavoidably terminated when the JVM exits. However, a workaround is to indirectly achieve the goal by leveraging the shell to do the "detach" on your behalf. The following scala script, which launches the gvim editor, appears to work as desired:
val cmd = List(
"scala",
"-e",
"""import scala.sys.process._ ; "gvim".run ; System.exit(0);"""
)
val proc = cmd.run
It assumes that scala is in the PATH, and it does (unavoidably) leave a JVM parent process running as well.