I am not getting any useful information about "how to close connection for mongodb using casbah API". Actually, I have defined multiple methods and in each method I need to establish a connection with mongodb. After working I need to close that too. I am using Scala.
one of the method like (code example in scala):
import com.mongodb.casbah.Imports._
import com.mongodb.casbah.MongoConnection
def index ={
val mongoConn = MongoConnection(configuration("hostname"))
val log = mongoConn("ab")("log")
val cursor = log.find()
val data = for {x <- cursor} yield x.getAs[BasicDBObject]("message").get
html.index(data.toList)
//mongoConn.close() <-- here i want to close the connection but this .close() is not working
}
It is unclear, from your question why exactly close is not working. Does it throw some exception, it is not compiling, or has no effect?
But since MongoConnection is a thin wrapper over com.mongodb.Mongo, you could work with underlying Mongo directly, just like in plain old Java driver:
val mongoConn = MongoConnection(configuration("hostname"))
mongoConn.underlying.close()
Actually, that's exactly, how close is implemented in casbah.
Try using .close instead. If a function doesn't have arguments in scala, you sometimes don't use parentheses after it.
EDIT: I had wrong information, edited to include correct information + link.
Related
Let's say I have a method that permits to update some date in DB:
def updateLastConsultationDate(userId: String): Unit = ???
How can I throttle/debounce that method easily so that it won't be run more than once an hour per user.
I'd like the simplest possible solution, not based on any event-bus, actor lib or persistence layer. I'd like an in-memory solution (and I am aware of the risks).
I've seen solutions for throttling in Scala, based on Akka Throttler, but this really looks to me overkill to start using actors just for throttling method calls. Isn't there a very simple way to do that?
Edit: as it seems not clear enough, here's a visual representation of what I want, implemented in JS. As you can see, throttling may not only be about filtering subsequent calls, but also postponing calls (also called trailing events in js/lodash/underscore). The solution I'm looking for can't be based on pure-synchronous code only.
This sounds like a great job for a ReactiveX-based solution. On Scala, Monix is my favorite one. Here's the Ammonite REPL session illustrating it:
import $ivy.`io.monix::monix:2.1.0` // I'm using Ammonite's magic imports, it's equivalent to adding "io.monix" %% "monix" % "2.1.0" into your libraryImports in SBT
import scala.concurrent.duration.DurationInt
import monix.reactive.subjects.ConcurrentSubject
import monix.reactive.Consumer
import monix.execution.Scheduler.Implicits.global
import monix.eval.Task
class DbUpdater {
val publish = ConcurrentSubject.publish[String]
val throttled = publish.throttleFirst(1 hour)
val cancelHandle = throttled.consumeWith(
Consumer.foreach(userId =>
println(s"update your database with $userId here")))
.runAsync
def updateLastConsultationDate(userId: String): Unit = {
publish.onNext(userId)
}
def stop(): Unit = cancelHandle.cancel()
}
Yes, and with Scala.js this code will work in the browser, too, if it's important for you.
Since you ask for the simplest possible solution, you can store a val lastUpdateByUser: Map[String, Long], which you would consult before allowing an update
if (lastUpdateByUser.getOrElse(userName, 0)+60*60*1000 < System.currentTimeMillis) updateLastConsultationDate(...)
and update when a user actually performs an update
lastUpdateByUser(userName) = System.currentTimeMillis
One way to throttle, would be to maintain a count in a redis instance. Doing so would ensure that the DB wouldn't be updated, no matter how many scala processes you were running, because the state is stored outside of the process.
I successfully inserted data into a mongodb database, but I don't know how to extract data out of a query. I use the default scala mongodb drive :
"org.mongodb.scala" %% "mongo-scala-driver" % "1.1.1"
The documentation seems to contains errors, by the way. This line rises a compilation error while this is copy pasted from the doc :
collection.find().first().printHeadResult()
This is how I query a collection:
collection.find()
How to convert it to a scala collection of object on which I can iterate and process ? Thanks
Yes, I agree with the compilation error. I think "collection.find().first().printHeadResult()" is not part of scala driver 1.1.1 release. The current scala driver github which uses this code is "1.2.0-SNAPSHOT" version.
You can get the results using the below code. However, you may experience some async behavior using the below code. Please refer the driver documentation.
val observable: FindObservable[Document] = collection.find();
observable.subscribe ( new Observer[Document] {
override def onNext(result: Document): Unit = println(result.toJson())
override def onError(e: Throwable): Unit = println("Failed" + e.getMessage)
override def onComplete(): Unit = println("Completed")
})
Mongo driver Observables link
This is answered from the best of my current knowledge. I spent a lot of time using casbah and I've recently switched to using the new async scala driver, so there may be more ergonomic ways to do some of this stuff that I don't know yet.
Basically you need to transform the result of the observable and then eventually, you'll probably want to turn it into something that isn't an observable so you can have synchronous code interact with it (maybe, depending on what you're doing).
In the current Mongo Scala API (2.7.0 as of writing this), you might process a list of documents like this:
coll.find(Document("head" -> 1)).map(dbo => dbo.getInteger("head"))
That takes the list of documents where head is equal to one and then applies the map function to convert it from the Document (dbo) into an Int by extracting the "head" element (note this will fall in ugly fashion if there isn't a head field or the field is not an int. There are more robust ways to get values out using get[T]).
You can find a full list of the operations that an Observable supports here:
https://mongodb.github.io/mongo-scala-driver/2.7/reference/observables/
under the list of Monadic operators.
The other part is how do you get the good stuff out the Observable because you want to do something synchronous with them. The best answer I have found so far is to dump the Observable into a Future and then calling Await.result on that.
val e = coll.find(Document("head" -> 1)).map(dbo => dbo.getInteger("head"))
val r = Await.result(e.toFuture(), Duration.Inf)
println(r)
That will print out the List[Int] that was created by evaluating the map function for each Document in the Observable.
I've been using Slick for quite a while and now I'm migrating from Slick 2.1 to 3.0. Unfortunatelly I got stuck with ordinary stuff like counting lines. My code worked perfectly in Slick 2.1 when I used to do this:
connection.withSession {
implicit session => coffees.length.run
}
On the code above I would get my result as an Int, but I can't get it to work now after I moved to Slick 3.0.2 though the documentation tells me that the code should be the same.
I tried the following (I already removed the withSession deprecated call):
connection.createSession.withTransaction {
coffees.length
}
But this code will return a slick.lifted.Rep[Int] which does not have any method to get the integer value. Am I missing some implicit import?
As you have probably realised, the result of the run call is to produce a Future, which will resolve at some later point.
While this means that eventually somewhere in the code the future will need to be waited on in a manner like you show in your answer, this can, and should, be pushed back as late as possible. If you are working with, for example, the Play framework, use async Actions and let Play handle it for you.
In the meantime work with the Future as you would any other monadic construct (like Option) - calling map, flatMap, onSuccess and so on to chain your computations inside the propagated Future context.
Please, someone tell me there is a better way to answer my question. I got it to work doing this, but this looks terrible:
import scala.concurrent.duration._
import scala.concurrent.Await
val timeout = Duration(10, SECONDS)
val count = Await.result(connection.run(coffees.length.result), timeout)
Hi i am new in Elastic Search and i want to use it with scala so i found some codes examples on github, but there was very complex examples were given as for a beginner I spend my whole day in trying to understand this tutorial but at the end i am confused how to start this is,its very complex to understand same as with other Scala client examples
https://github.com/scalastuff/esclient
https://github.com/bsadeh/scalastic
https://github.com/gphat/wabisabi also i tried this but it contains error and i posted it here as well https://stackoverflow.com/questions/27145015/scalagetstatuscode-getresponsebody-is-not-a-member-of-dispatch-future
All these examples are very complex for a new learner like me as i go through first chapter of Elastic Search from its guide then I want to do these same things pro-grammatically with Scala.Please suggest me some starting point from where can i start learning and also there is a request do not mark this question as nonconstructive first i tried myself after then i am posting this question,Please i need help i want to learn elastic search using scala
The Elastic4s project contains, near the top of the readme, a simple example on how to use the driver. This example is a complete Scala program that you can execute.
import com.sksamuel.elastic4s.ElasticClient
import com.sksamuel.elastic4s.ElasticDsl._
object Test extends App {
val client = ElasticClient.local
// await is a helper method to make this operation sync instead of async
// You would normally avoid doing this in a real program as it will block
client.execute { index into "bands/artists" fields "name"->"coldplay" }.await
val resp = client.execute { search in "bands/artists" query "coldplay" }.await
println(resp)
}
If this is too complicated, then that is not because the Scala client is too complicated, but because you don't yet understand enough about Elasticsearch or Scala. The Scala client you are looking at is a typical DSL so it uses some Scala tricks that make it nice to use as a client, but not necessarily easy to to understand under the covers.
Here are some good links to understanding Elasticsearch:
http://spinscale.github.io/elasticsearch/2012-03-jugm.html#/20
http://exploringelasticsearch.com/
http://joelabrahamsson.com/elasticsearch-101/
http://www.slideshare.net/karmi/your-data-your-search-elasticsearch-euruko-2011
Before you use any of the Scala drivers, you should at least understand the basic concepts of an index/type, the query DSL, and what a node is in Elasticsearch. It might also be helpful to look at the JSON that you can send with the HTTP interface as that is a bit easier to see what is going on, because the Elasticsearch docs can be heavy going at first.
Simple elastic search client
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.5.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.5.0</version>
</dependency>
Scala code to ES with basic auth:
import org.apache.http.HttpHost
import org.apache.http.auth.{AuthScope, Credentials, UsernamePasswordCredentials}
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest.AliasActions
import org.elasticsearch.client._
import org.apache.http.client.CredentialsProvider
import org.apache.http.impl.client.BasicCredentialsProvider
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder
import org.elasticsearch.client.RestClient
import org.elasticsearch.client.RestClientBuilder
val credentials = new UsernamePasswordCredentials("<username>", "<password>");
val credentialsProvider:CredentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, credentials)
val client = RestClient.builder(new HttpHost("<host>", 9200,"https")).setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
}).build
val request = new Request(
"GET",
/_cat/aliases?format=JSON )
val response = client.performRequest(request);
println("Response:"+response.getEntity.getContent)
client.close
I'm using slick 2.0 for my interaction with the database.
As recommended I've added external connection pool using BonesCP.
val driver = Class.forName(Database.driver)
val ds = new BoneCPDataSource();
ds.setJdbcUrl(Database.jdbcUri);
ds.setUsername(Database.user);
ds.setPassword(Database.password);
using it I've created my connection:
scala.slick.jdbc.JdbcBackend.Database.forDataSource(ds)
db withSession {implicit session =>
(...)
}
}
Now if I do on my TableQuery object something like this:
provisioning.foreach {(...)}
it says that there is no foreach method.
So I've imported:
import scala.slick.driver.PostgresDriver.simple._
and now everything works well.
What I don't like is that my code is tied to a certain database implementation.
Can I somehow make it read the "db dialect" from the config file?
So now I know how this should be done properly.
I've been to a presentation by Stefan Zeiger on Slick who is the original author of the library and he pointed out that since PostgresDriver as well as other drivers is a trait we can mixin it instead of importing. Thus having chance to dynamically choose the driver that fits our database.