When my connection is open, the application won't exit, this causes some nasty problems for me (highly concurrent and nested using a shared sesssion, don't know when each part is finished) - is there a way to make sure that the cluster doesn't "hang" the application?
For example here:
object ZombieTest extends App {
val session= Cluster.builder().addContactPoint("localhost").build().connect()
// app doesn't exit unless doing:
session.getCluster.close() // won't exit unless this is called
}
In a slightly biased answer, you could look at https://github.com/outworkers/phantom instead of using the standard java driver.
You get scala.concurrent.Future, monix.eval.Task or even com.twitter.util.Future from a query automatically. You can choose between all three.
DB connection pools are better isolated inside ContactPoint and Database abstraction layers, which have shutdown methods you can easily wire in to your app lifecycle.
It's far faster than the Java driver, as the serialization an de-serialisation of types is wired in compile time via more advanced macro mechanisms.
The short answer is that you need to have a lifecycle way of calling session.close or session.closeAsync when you shut down everything else, it's how it's designed to work.
Related
I’m using AsyncMachine to model multiple objects interacting asynchronously and in principle, all works as expected, really a cool extension ;-)
However, when I use logging, the millisecond delays which are reported in log entries between processing multiple, asynchronous events are higher than I would expect, so I’m wondering whether this is due to logging output created by a blocking call to e.g. logger.info(). So the timings I’m trying to obtain by looking at log entry timestamps could be distorted by the creation of exactly those logs.
Using https://pypi.org/project/aiologger/ seems like a reasonable way forward, given that aiologger implements non-blocking logging specifically for asyncio.
After a quick look at pytransitions source code I’m wondering however what would happen if pytransitions itself still uses the logging module from the standard library while my code would use the logger offered by aiologger. My guess is that in this case, only the logs created by my code would be non-blocking, so how could I make pytransitions use aiologger as well?
Thanks in advance for any help,
Eradian
so how could I make pytransitions use aiologger as well?
As of September 2022 there is no way of making transitions work with aiologger. Due to backward compatibility reasons, transitions uses the %-formatting style for debug messages:
logging.warn("Object %s is initialized with value %d", object.name, object.value)
Theoretically you could override the internal logger instance used by AsyncMachine:
from aiologger import Logger
import transitions.extensions.asyncio as taio
taio._LOGGER = Logger.with_default_handlers(name='transitions.async')
# ...
m = taio.AsyncMachine(states=['A', 'B', 'C'], initial='A')
But this will throw errors during the first logging attempt because of the aforementioned reason:
Invalid LogRecord args type: <class 'str'>. Expected Mapping
I have the following configuration in application.conf for an in-memory database HSQLDB:
db {
inmemory {
jndiName = jndiInMemory
driver = org.hsqldb.jdbc.JDBCDriver
url = "jdbc:hsqldb:mem:inmemory"
}
}
And connect in a controller with the following statements
val database = Database.forName("jndiInMemory")
val session = database.createSession
val conn = session.conn
// JDBC statements
Problem is that when the code runs several times, I get an exception in session.conn:
HikariPool-34 - Connection is not available, request timed out after
30000ms.
Since I'm using JNDI, I figured that the connections are reused. Do I have to drop the session after I finish using it? How to fix this code?
Hard to tell without looking at the actual code but in general: when you create a database connection at the start of the application, you usually reuse it until application ends - then you should close connection.
If you spawn a new connection every time you do query, without ending previous ones you will run at the connection limit pretty fast.
Easy to use pattern is: create a session in the beginning and then use dependency injection to pass it to wherever you need to run it.
BTW, I noticed that for some configurations e.g. Slick create connection statically (as in: stores them as static class properties). So, you need to create a handler, that closes session when application exits. It run ok... until you start it several times over in SBT, which by default uses the same JVM to run itself and spawned application. In such cases it is better to run things as fork. For tests I use Test / fork := true, for run I use sbt-revolver, though I am not sure how that would play out with Play.
I am using Neo4j embedded in my Scala project. I have been including
ShutdownHookThread {
shutdown(ds)
}
the above piece of code in each and every function before the beginning of transaction. Do I need to include it in every function. What happens if I don't include it?
ShutdownHookThread registers a piece of code to be executed when your application is about to exit. You need to use it only once - somewhere in your app bootstrap code, cause there is no sense to shutdown the database more than one time.
For an open-source multiplayer programming game written in Scala that loads players' bot code via a plug-in system from .jar files, I'd like to prevent the code of the bots from doing harm on the server system by running them under a restrictive SecurityManager implementation.
The current implementation uses a URLClassLoader to extract a control function factory for each bot from its related plug-in .jar file. The factory is then used to instantiate a bot control function instance for each new round of the game. Then, once per simulation cycle, all bot control functions are invoked concurrently to get the bot's responses to their environment. The concurrent invocation is done using Akka's Future.traverse() with an implicitly provided ActorSystem shared by other concurrently operating components (compile service, web server, rendering):
val future = Future.traverse(bots)(bot => Future { bot.respondTo(state) })
val result = Await.result(future, Duration.Inf)
To restrict potentially malicious code contained in the bot plug-ins from running, it appears that following the paths taken in this StackOverflow question and this one I need to have the bot control functions execute in threads running under an appropriately restrictive SecurityManager implementation.
Now the question: how can I get Akka to process the work currently done in Future.traverse() with actors running in threads that have the desired SecurityManager, while the other Actors in the system, e.g. those running the background compile service, continue to run as they do now, i.e. unrestricted?
You can construct an instance of ExecutionContext (eg. via ExecutionContext.fromExecutorService) that runs all work under the restrictive security manager, and bring it into the implicit scope for Future.apply and Future.traverse.
If the invoked functions do not need to interact with the environment, I don't think you need a separate ActorSystem.
For a multi-player programming game, I'm working on a background compilation server for Scala that supports compilation of multiple, independent source trees submitted by the players. I succeeded in running fast, sequential compilations without reloading the compiler by instantiating the Global compiler object via
val compilerGlobal = new Global(settings, reporter)
and then running individual compile jobs via
val run = new compilerGlobal.Run
run.compile(sourceFilePathList)
I would now ideally like to parallelize the server (i.e. make multiple compilation runs concurrently), but still without reloading the compiler (primarily to avoid re-parsing the lib) from scratch each time. Is this possible, i.e. is the second part shown above (safely :-) re-entrant, or does it hold global state? If not, is there something else I can try? I am currently focused on supporting Scala 2.9.1.
Yes, compiler Runs share state, so you should not share them between threads. It's one of the issues that comes up in the Eclipse plugin. As #EJP noted, the symbol table is shared.
This is not so important in your case, but comes up in an IDE: the compiler uses laziness in types, meaning additional computation (and mutation) may happen when calling methods on Symbol. Because of visibility issues, it's important that these methods are called on the same thread as the one that created them.