I am trying to migrate from a very old version of elasticsearch to the latest version 6.3.0. I have found this link ElasticSearch TransportClient version 5.6 that shows a pattern that I tried to implement but I can't get it to work:
import org.elasticsearch.common.settings.Settings
import org.elasticsearch.client.transport.TransportClient
import org.elasticsearch.transport.client.PreBuiltTransportClient
trait ElasticSearchClient {
lazy val client: TransportClient = new PreBuiltTransportClient(
Settings.builder()
.put("cluster.name", clusterName)
// etc...
.build())
I get the error:
[error] object client is not a member of package org.elasticsearch.transport
[error] import org.elasticsearch.transport.client.PreBuiltTransportClient
[error] ^
I've tried the alternate import org.elasticsearch.client.transport.PreBuiltTransportClient but still not resolving.
If I try lazy val client: TransportClient = new TransportClient( it says class TransportClient is abstract; cannot be instantiated.
If I try lazy val client: TransportClient = TransportClient.builder(). it says value builder is not a member of object org.elasticsearch.client.transport.TransportClient.
I cannot find any other documentation or posts on this and I am asking what is the correct pattern to create a TransportClient object?
The answer is the above works if you add to the libraryDependencies of your build.sbt file:
"org.elasticsearch.client" % "transport" % "6.3.0"
Note that TransportClient is to be deprecated in 7.0 and removed in 8.0. Use RestClient instead.
Related
I am getting below error while using HttpClient. Can you let me know how to use HttpClient exactly. I am new with elastic4s.
I want to connect scala with ssl configured elasticsearch. I also want to know how I can pass SSL details with link such as keystore path, truststore path and user name , password.
scala> import com.sksamuel.elastic4s.http.{HttpClient, HttpResponse}
import com.sksamuel.elastic4s.http.{HttpClient, HttpResponse}
scala> import com.sksamuel.elastic4s.http.ElasticDsl._
import com.sksamuel.elastic4s.http.ElasticDsl._
scala> val client = HttpClient(ElasticsearchClientUri(uri))
<console>:39: error: not found: value HttpClient
val client = HttpClient(ElasticsearchClientUri(uri))
HttpClient appears to be a trait in the codebase.You seem to be using the same as an object. You can check the implementation Here. For your use case i think the better approach would be to use ElasticClient. Code would look something like this
import com.sksamuel.elastic4s.http._
import com.sksamuel.elastic4s.{ElasticClient, ElasticDsl, ElasticsearchClientUri}
val client = elastic4s.ElasticClient(ElasticsearchClientUri(uri))
I got the same problem, i.e. in my setup I got errors (not found) when trying to use HttpClient (elastic4s-core,elastic4s-http-streams and elastic4s-client-esjava version 7.3.1 on scala 2.12.10).
The solution: you should be able to find and use JavaClient, an implementation of HttpClient that wraps the Elasticsearch Java Rest Client.
An example of how to use the JavaClient can be found here.
Thus, your code should look like the following:
import com.sksamuel.elastic4s.http.JavaClient
import com.sksamuel.elastic4s.{ElasticClient, ElasticDsl, ElasticProperties}
...
val client = ElasticClient(JavaClient(ElasticProperties(uri)))
I want to send messages from a Twitter application to an Azure event hub. However, I am getting an error that says:
notebook:20: error: type mismatch;
found : java.util.concurrent.ExecutorService
required: java.util.concurrent.ScheduledExecutorService
val eventHubClient = EventHubClient.create(connStr.toString(), pool)
I do not know how to create the EventHubClient.create now. Please help.
I am referring to code from the link
https://learn.microsoft.com/en-us/azure/azure-databricks/databricks-stream-from-eventhubs.
Also, I have tried the solution from link:
Stream data into Azure Databricks using Event Hubs and it doesn't work for me.
The version of the cluster is 5.2 (includes Apache Spark 2.4.0, Scala 2.11) which should include the Java SE 8 libraries that have the new ScheduledExecutorService member. Also, the libraries attached are com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.9 and org.twitter4j:twitter4j-core:4.0.7, so again all the prerequisites are met.
The code is:
import java._
import java.util._
import scala.collection.JavaConverters._
import com.microsoft.azure.eventhubs._
import java.util.concurrent._
import java.util.concurrent.ExecutorService
import java.util.concurrent.ScheduledExecutorService
val pool = Executors.newFixedThreadPool(1)
val eventHubClient = EventHubClient.create(connStr.toString(), pool)
I've read up everything I could on SO and the ReactiveMongo community list and I am stumped. I am using ReactiveMongo version 0.12 and am just trying to test it out since I have some other problems.
The code in my scala worksheet is:
import reactivemongo.api.{DefaultDB, MongoConnection, MongoDriver}
import reactivemongo.bson.{
BSONDocumentWriter, BSONDocumentReader, Macros, document
}
import com.typesafe.config.{Config, ConfigFactory}
lazy val conf = ConfigFactory.load()
val driver1 = new reactivemongo.api.MongoDriver
val connection3 = driver1.connection(List("localhost"))
and the error I get is
[NGSession 3: 127.0.0.1: compile-server] INFO reactivemongo.api.MongoDriver - No mongo-async-driver configuration found
com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka'
at com.typesafe.config.impl.SimpleConfig.findKey(testMongo.sc:120)
at com.typesafe.config.impl.SimpleConfig.find(testMongo.sc:143)
at com.typesafe.config.impl.SimpleConfig.find(testMongo.sc:155)
at com.typesafe.config.impl.SimpleConfig.find(testMongo.sc:160)
at com.typesafe.config.impl.SimpleConfig.getString(testMongo.sc:202)
at akka.actor.ActorSystem$Settings.<init>(testMongo.sc:165)
at akka.actor.ActorSystemImpl.<init>(testMongo.sc:501)
at akka.actor.ActorSystem$.apply(testMongo.sc:138)
at reactivemongo.api.MongoDriver.<init>(testMongo.sc:879)
at #worksheet#.driver1$lzycompute(testMongo.sc:9)
at #worksheet#.driver1(testMongo.sc:9)
at #worksheet#.get$$instance$$driver1(testMongo.sc:9)
at #worksheet#.#worksheet#(testMongo.sc:30)
My application.conf is in src/main/resources of the sub-project which this worksheet is found and contains this:
mongo-async-driver {
akka {
loglevel = WARNING
}
}
I added the ConfigFactory precisely because I got this error and thought it might help. I looked at the code and that's what ReactiveMongo is doing at this point so I thought perhaps a call here would force it to load at this point. I have moved the application.conf file into every conceivable place including a conf directory (thinking it might require play conventions) and the src/main/resources of the top level directory. Nothing works. So my first question is what am I doing wrong? Where should application.conf file go?
This info message causes my program to crash and driver doesn't get created so I can't move on from here.
Also, I added an akka key to reference.conf just in case - that didnt help either.
I need to consume an xmlrpc service from Scala, and so far it looks like my only option is the Apache XML-RPC library.
I added this dependency to my Build.scala:
"org.apache.xmlrpc" % "xmlrpc" % "3.1.3"
and sbt reported no problem in downloading the library. However, I don't know how to go about actually accessing the library.
val xml = org.apache.xmlrpc.XmlRpcClient("http://foo") wouldn't compile
and
import org.apache.xmlrpc._
reported that object xmlrpc was not a member of package org.apache.
What would be the correct package to import?
(Or, is there a better library for XmlRpc from Scala?)
Try
"org.apache.xmlrpc" % "xmlrpc-client" % "3.1.3"
and so :
class XmlRpc(val serverURL: String) {
import org.apache.xmlrpc.client.XmlRpcClient
import org.apache.xmlrpc.client.XmlRpcClientConfigImpl
import org.apache.xmlrpc.client.XmlRpcSunHttpTransportFactory
import java.net.URL
val config = new XmlRpcClientConfigImpl();
config.setServerURL(new URL(serverURL));
config.setEncoding("ISO-8859-1");
val client = new XmlRpcClient();
client.setTransportFactory(new XmlRpcSunHttpTransportFactory(client));
client.setConfig(config);
client.execute(...)
}
There is a good module for this kind of tasks:
https://github.com/jvican/xmlrpc
I have installed eclipse, the cloudfoundry plugin, the scala plugin,the vaadin plugin(for web developments) and the mongodb libraries.
I created a class like this :
import vaadin.scala.Application
import vaadin.scala.VerticalLayout
import com.mongodb.casbah.MongoConnection
import com.mongodb.casbah.commons.MongoDBObject
import vaadin.scala.Label
import vaadin.scala.Button
class Launcher extends Application {
val label=new Label
override def main = new VerticalLayout() {
val coll=MongoConnection()("mybd")("somecollection")
val builder=MongoDBObject.newBuilder
builder+="foo1" -> "bar"
var newobj=builder.result()
coll.save(newobj)
val mongoColl=MongoConnection()("mybd")("somecollection")
val withFoo=mongoColl.findOne()
label.value=withFoo
add(label)
//bouton pour faire joli
add(new Button{
caption_=("click me!")
})
}
}
the error (the access to the mongodb database is denied) comes from the parameters, which are the default ones.
do you know how to set up the good parameters in scala or in java?
Looks like you got some help on the vcap-dev mailing list
package com.example.vaadin_1
import vaadin.scala.Application
import org.cloudfoundry.runtime.env.CloudEnvironment
import org.cloudfoundry.runtime.env.MongoServiceInfo
import com.mongodb.casbah.MongoConnection
class Launcher extends Application {
val cloudEnvironment = new CloudEnvironment()
val mongoServices = cloudEnvironment.getServiceInfos(classOf[MongoServiceInfo])
val mongo = mongoServices.get(0)
val mongodb = MongoConnection(mongo.getHost(), mongo.getPort())("abc")
mongodb.authenticate(mongo.getUserName(),mongo.getPassword())
}
I would suggest to do it using Spring Data for MongoDB there is a sample application for Cloudfoundry in particular put together by the Spring guys. With a bit of xml configuration you have ready to inject the mongoTemplate similar to the familiar Spring xxxTemplate paradigm.
when deploying to CloudFoundry, the information relative to connecting to a service (i.e mongo in your case) is made available to the app through environment variable VCAP_SERVICES. It is a json document with one entry per service. You could of course parse it yourself, but you will find the class http://cf-runtime-api.cloudfoundry.com/org/cloudfoundry/runtime/env/CloudEnvironment.html useful. You will need to add the org.cloudfoundry/cloudfoundry-runtime/0.8.1 jar to your project. You can use it without Spring.
Have a look at http://docs.cloudfoundry.com/services.html for explanation of the VCAP_SERVICES underlying var