I am implementing a file storage service, which is taking a file and saving it into gridFS with special metadata. Of course, I want to be sure everything's working in integration -- files are really stored in database and then retrieved from it.
I use Play Framework 2.1.3 Scala and ReactiveMongo 0.9.
My test cases looks like the following:
"show empty uploaded size on init" in {
running(FakeApplication()) {
Await.result(FileStorage.getFilesSize(profileId), duration) must beNone
}
}
I have tried to wrap every case with running, or all cases, or even Thread.sleep. But database is always up after test fails.
[error] There is no started application
[error] play.api.Play$$anonfun$current$1.apply(Play.scala:51)
[error] play.api.Play$$anonfun$current$1.apply(Play.scala:51)
[error] play.api.Play$.current(Play.scala:51)
[error] content.FileStorage$.db$lzycompute(FileStorage.scala:32)
...
[info] Total for specification FileStorageSpec
[info] Finished in 21 ms
[info] 5 examples, 1 failure, 4 errors
[info]
[info] application - ReactiveMongoPlugin starting...
[info] application - ReactiveMongoPlugin successfully started with db 'test'! Servers:
[localhost:27017]
[info] play - Starting application default Akka system.
[info] play - Shutdown application default Akka system.
What am I doing wrong? How do you test ReactiveMongo applications?
In the FileStorage object you have these lines:
lazy val db = ReactiveMongoPlugin.db
val gridFS = GridFS(db, "file")
val collection = db.collection[JSONCollection]("file.files")
collection.indexesManager.ensure(Index(Seq("metadata.profileId" -> IndexType.Ascending)))
When the object is instantiated, the above lines of code are executed. Since you have no control of object instantiation you can not be sure when that happens.
It will probably help to change the lines into this:
def db = ReactiveMongoPlugin.db
def gridFS = GridFS(db, "file")
def collection = {
val collection = db.collection[JSONCollection]("file.files")
collection.indexesManager.ensure(Index(Seq("metadata.profileId" -> IndexType.Ascending)))
collection
}
Related
Sbt seems to be using different classloaders, making some tests failing when run more than once in an sbt session, with the following error:
[info] java.lang.ClassCastException: net.i2p.crypto.eddsa.EdDSAPublicKey cannot be cast to net.i2p.crypto.eddsa.EdDSAPublicKey
[info] at com.advancedtelematic.libtuf.crypt.EdcKeyPair$.generate(RsaKeyPair.scala:120)
I tried equivalent code using pattern matching instead of asInstanceOf and I get the same result.
How can I make sure sbt uses the same class loader for all test executions in the same session?
I think it's related to this: Do security providers cause ClassLoader leaks in Java?. Basically Security is re-using providers from old class-loaders. So this could happen in any multi-classpath environment (like OSGi), not just SBT.
Fix for your build.sbt (without forking):
testOptions in Test += Tests.Cleanup(() =>
java.security.Security.removeProvider("BC"))
Experiment:
sbt-classloader-issue$ sbt
> test
[success] Total time: 1 s, completed Jul 6, 2017 11:43:53 PM
> test
[success] Total time: 0 s, completed Jul 6, 2017 11:43:55 PM
Explanation:
As I can see from your code (published here):
Security.addProvider(new BouncyCastleProvider)
you're reusing the same BouncyCastleProvider provider every-time you run a test, as your Security.addProvider works only first time. As sbt creates new class-loader for every "test" run, but re-uses the same JVM - Security is kind-of JVM-scoped singleton as it was loaded by JVM-bootstrap, so classOf[java.security.Security].getClassLoader() == null and sbt cannot reload/reinitialize this class.
And you can easily check that
classOf[org.bouncycastle.jce.spec.ECParameterSpec].getClassLoader()
res30: ClassLoader = URLClassLoader with NativeCopyLoader with RawResources
org.bouncycastle classes are loaded with custom classloader (from sbt) which changes every-time you run test.
So this code:
val generator = KeyPairGenerator.getInstance("ECDSA", "BC")
gets instance of class loaded from old classloader (the one used for first "test" run) and you're trying to initialize it with spec from new classloader:
generator.initialize(ecSpec)
That's why you're getting "parameter object not a ECParameterSpec" exception. The reasoning around "net.i2p.crypto.eddsa.EdDSAPublicKey cannot be cast to net.i2p.crypto.eddsa.EdDSAPublicKey" is basically same.
I've read up everything I could on SO and the ReactiveMongo community list and I am stumped. I am using ReactiveMongo version 0.12 and am just trying to test it out since I have some other problems.
The code in my scala worksheet is:
import reactivemongo.api.{DefaultDB, MongoConnection, MongoDriver}
import reactivemongo.bson.{
BSONDocumentWriter, BSONDocumentReader, Macros, document
}
import com.typesafe.config.{Config, ConfigFactory}
lazy val conf = ConfigFactory.load()
val driver1 = new reactivemongo.api.MongoDriver
val connection3 = driver1.connection(List("localhost"))
and the error I get is
[NGSession 3: 127.0.0.1: compile-server] INFO reactivemongo.api.MongoDriver - No mongo-async-driver configuration found
com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka'
at com.typesafe.config.impl.SimpleConfig.findKey(testMongo.sc:120)
at com.typesafe.config.impl.SimpleConfig.find(testMongo.sc:143)
at com.typesafe.config.impl.SimpleConfig.find(testMongo.sc:155)
at com.typesafe.config.impl.SimpleConfig.find(testMongo.sc:160)
at com.typesafe.config.impl.SimpleConfig.getString(testMongo.sc:202)
at akka.actor.ActorSystem$Settings.<init>(testMongo.sc:165)
at akka.actor.ActorSystemImpl.<init>(testMongo.sc:501)
at akka.actor.ActorSystem$.apply(testMongo.sc:138)
at reactivemongo.api.MongoDriver.<init>(testMongo.sc:879)
at #worksheet#.driver1$lzycompute(testMongo.sc:9)
at #worksheet#.driver1(testMongo.sc:9)
at #worksheet#.get$$instance$$driver1(testMongo.sc:9)
at #worksheet#.#worksheet#(testMongo.sc:30)
My application.conf is in src/main/resources of the sub-project which this worksheet is found and contains this:
mongo-async-driver {
akka {
loglevel = WARNING
}
}
I added the ConfigFactory precisely because I got this error and thought it might help. I looked at the code and that's what ReactiveMongo is doing at this point so I thought perhaps a call here would force it to load at this point. I have moved the application.conf file into every conceivable place including a conf directory (thinking it might require play conventions) and the src/main/resources of the top level directory. Nothing works. So my first question is what am I doing wrong? Where should application.conf file go?
This info message causes my program to crash and driver doesn't get created so I can't move on from here.
Also, I added an akka key to reference.conf just in case - that didnt help either.
I'm playing with Mongo database through the Reactive Mongo driver
import org.slf4j.LoggerFactory
import reactivemongo.api.MongoDriver
import reactivemongo.api.collections.default.BSONCollection
import reactivemongo.bson.BSONDocument
import scala.concurrent.Future
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
object Main {
val log = LoggerFactory.getLogger("Main")
def main(args: Array[String]): Unit = {
log.info("Start")
val conn = new MongoDriver().connection(List("localhost"))
val db = conn("test")
log.info("Done")
}
}
My build.sbt file:
lazy val root = (project in file(".")).
settings(
name := "simpleapp",
version := "1.0.0",
scalaVersion := "2.11.4",
libraryDependencies ++= Seq(
"org.reactivemongo" %% "reactivemongo" % "0.10.5.0.akka23",
"ch.qos.logback" % "logback-classic" % "1.1.2"
)
)
When I run: sbt compile run
I get this output:
$ sbt compile run
[success] Total time: 0 s, completed Apr 25, 2015 5:36:51 PM
[info] Running Main
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
17:36:52.328 [run-main-0] INFO Main - Start
17:36:52.333 [run-main-0] INFO Main - Done
And application doesn't stop.... :/
I have to press Ctrl + C to kill it
I've read that MongoDriver() creates ActorSystem so I tried to close connection manually with conn.close() but I get this:
[info] Running Main
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
17:42:23.252 [run-main-0] INFO Main - Start
17:42:23.258 [run-main-0] INFO Main - Done
17:42:23.403 [reactivemongo-akka.actor.default-dispatcher-2] ERROR reactivemongo.core.actors.MongoDBSystem - (State: Closing) UNHANDLED MESSAGE: ChannelConnected(-973180998)
[INFO] [04/25/2015 17:42:23.413] [reactivemongo-akka.actor.default-dispatcher-3] [akka://reactivemongo/deadLetters] Message [reactivemongo.core.actors.Closed$] from Actor[akka://reactivemongo/user/$b#-1700211063] to Actor[akka://reactivemongo/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [04/25/2015 17:42:23.414] [reactivemongo-akka.actor.default-dispatcher-3] [akka://reactivemongo/user/$a] Message [reactivemongo.core.actors.Close$] from Actor[akka://reactivemongo/user/$b#-1700211063] to Actor[akka://reactivemongo/user/$a#-1418324178] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
And app doesn't exit also
So, what am i doing wrong? I can'f find answer...
And it seems to me that official docs doesn't explain whether i should care about graceful shutdown at all.
I don't have much experience with console apps, i use play framework in my projects but i want to create sub-project that works with mongodb
I see many templates (in activator) such as: Play + Reactive Mongo, Play + Akka + Mongo but there's no Scala + Reactive Mongo that would explain how to work properly :/
I was having the same problem. The solution I found was invoking close on both object, the driver and the connection:
val driver = new MongoDriver
val connection = driver.connection(List("localhost"))
...
connection.close()
driver.close()
If you close only the connection, then the akka system remains alive.
Tested with ReactiveMongo 0.12
This looks like a known issue with Reactive Mongo, see the relevant thread on GitHub
A fix for this was introduced in this pull request #241 by reid-spencer, merged on the 3rd of February 2015
You should be able to fix it by using a newer version. If no release has been made since February, you could try checking out a version that includes this fix and building the code yourself.
As far as I can see, there's no mention of this bugfix in the release notes for version 0.10.5
Bugfixes:
BSON library: fix BSONDateTimeNumberLike typeclass
Cursor: fix exception propagation
Commands: fix ok deserialization for some cases
Commands: fix CollStatsResult
Commands: fix AddToSet in aggregation
Core: fix connection leak in some cases
GenericCollection: do not ignore WriteConcern in save()
GenericCollection: do not ignore WriteConcern in bulk inserts
GridFS: fix uploadDate deserialization field
Indexes: fix parsing for Ascending and Descending
Macros: fix type aliases
Macros: allow custom annotations
The name of the committer does not appear as well:
Here is the list of the commits included in this release (since 0.9, the top commit is the most recent one):
$ git shortlog -s -n refs/tags/v0.10.0..0.10.5.x.akka23
39 Stephane Godbillon
5 Andrey Neverov
4 lucasrpb
3 Faissal Boutaounte
2 杨博 (Yang Bo)
2 Nikolay Sokolov
1 David Liman
1 Maksim Gurtovenko
1 Age Mooij
1 Paulo "JCranky" Siqueira
1 Daniel Armak
1 Viktor Taranenko
1 Vincent Debergue
1 Andrea Lattuada
1 pavel.glushchenko
1 Jacek Laskowski
Looking at the commit history for 0.10.5.0.akka23 (the one you reference in build.sbt), it seems the fix was not merged into it.
I have a class with a custom logger. Here's a trivial example:
package models
import play.Logger
object AModel {
val log = Logger.of("amodel")
def aMethod() {
if (! log.isInfoEnabled) log.error("Can't log info...")
log.info("Logging aMethod in AModel")
}
}
and then we'll enable this logger in application.conf:
logger.amodel=DEBUG
and in development (Play console, use run) this logger does indeed log. But in production, once we hit the message
[info] play - Application started (Prod)
loggers defined like the above logger fail to log any further and instead we go through the error branch. It seems their log level has been changed to ERROR.
Is there anyway to correct this undesirable state of affairs? Is there special configuration for production logs?
edit
Play's handling of logs in production is a source of difficulty to more than a few people... https://github.com/playframework/playframework/issues/1186
For some reason it ships its own logger.xml which overrides application.conf.
I'd like to use ANORM to connect to db in play console, just simply test some stuff. But there's some errors when I create a DataSource
val ds=DB.getDataSource()
java.lang.RuntimeException: There is no started application
You can start an application from within the play console:
[My application] $ console
scala> import play.core.StaticApplication
import play.core.StaticApplication
scala> new StaticApplication(new java.io.File("."))
[info] play - Application started (Prod)
res0: play.core.StaticApplication = play.core.StaticApplication#...