Am trying to get my hands on elastic4s by one of the samples as given
here
But I keep getting the below exception when am trying to connect through TcpClient.transport :
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{IFyYWnE_S4aHRVxT9v60LQ}{dockerhost}{192.168.99.100:9300}]]
Am trying to connect to an elastic instance on docker, elastic version is 2.3.4
Here is my code below dependencies.
import com.sksamuel.elastic4s.{ElasticClient, ElasticsearchClientUri, TcpClient}
import org.elasticsearch.action.support.WriteRequest.RefreshPolicy
import org.elasticsearch.common.settings.Settings
import com.sksamuel.elastic4s.ElasticDsl._
object Main extends App {
//val settings = Settings.builder().put("cluster.name", "elasticsearch").build()
val client = TcpClient.transport(ElasticsearchClientUri("elasticsearch://dockerhost:9300"))
client.execute {
bulk(
indexInto("myindex" / "mytype").fields("country" -> "Mongolia", "capital" -> "Ulaanbaatar"),
indexInto("myindex" / "mytype").fields("country" -> "Namibia", "capital" -> "Windhoek")
).refresh(RefreshPolicy.WAIT_UNTIL)
}.await
val result = client.execute {
search("myindex").matchQuery("capital", "ulaanbaatar")
}.await
println(result.hits.head.sourceAsString)
client.close()
}
build.gradle:
compile group: 'com.sksamuel.elastic4s', name: 'elastic4s-core_2.11', version: '5.4.9'
compile group: 'com.sksamuel.elastic4s', name: 'elastic4s-tcp_2.11', version: '5.4.9'
compile group: 'com.sksamuel.elastic4s', name: 'elastic4s-http_2.11', version: '5.4.9'
compile group: 'com.sksamuel.elastic4s', name: 'elastic4s-streams_2.11', version: '5.4.9'
Any help regarding this issue would be helpful.
I am asked this question a lot, and 99% of the time, the answer to this question is
The cluster name is not the default (elasticsearch) and therefore it must be specified in the connection string.
The server is not setup to listen outside of localhost. https://www.elastic.co/blog/elasticsearch-unplugged
Related
is there a way to get data from a database in postgres using wasm?. I'd tried to get it using a library in rust but I got some errore when I build the package using "wasm-pack building--target web". The idea is to build a function in lib.rs file that return data from a db. I have the below code inside lib.rs:
use postgres::{Client, Error, NoTls};
use wasm_bindgen::prelude::*;
...
struct Author {
_id: i32,
name: String,
}
#[wasm_bindgen]
pub fn select_name(name: &String) -> Result<(), Error> {
let mut client = Client::connect("postgresql://user:1234#localhost:5432/db", NoTls)?;
for row in client.query(
"SELECT id, name FROM author WHERE name = $1",
&[&name],
)? {
let author = Author {
_id: row.get(0),
name: row.get(1),
};
println!(
"Select_Name => Author {} :",
author.name
);
}
Ok(())
}
but I get some errors:
error[E0432]: unresolved import `crate::sys::IoSourceState`
error[E0432]: unresolved import `crate::sys`
...
It is not possible directly (as Njuguna Mureithi rightly wrote) but it can be circumvented.
We can use the project: https://github.com/PostgREST/postgrest
and expose the API to our sql server.
We download the latest version of postgrest
https://github.com/PostgREST/postgrest/releases/tag/v9.0.0
in case of Linux we unpack
tar -xf postgrest-v9.0.0-linux-static-x64.tar.xz
then run help
./postgrest -h
create a configuration file for ./postgrest
postgrest -e > cfg.psqlrest
change the user and password for the database in the configuration file.
e.g. with
db-uri = "postgres://user:pasword#localhost:5432/dbname"
to your dbname database access user:pasword configuration
db-uri = "postgres://postgres:zaqwsxc#localhost:5432/dbname"
and run the server which will issue the api to our postgres
./postgrest cfg.psqlrest
The address http://localhost:3000 will start accessing the database dbname, which must be created in the database server beforehand.
Here is a description of the libraries needed to call the query using the API.
https://rustwasm.github.io/wasm-bindgen/examples/fetch.html
examples of API
https://postgrest.org/en/stable/api.html
I have a Mongo DB Docker container running on 192.168.0.229. From another computer, I can access it via:
> mongo "mongodb://192.168.0.229:27017/test"
But when I add that configuration string (host="192.168.0.229") to my Play Framework app, I get a timeout error:
[debug] application - Login Form Success: UserData(larry#gmail.com,testPW)
[error] p.a.h.DefaultHttpErrorHandler -
! #7m7kggikl - Internal server error, for (POST) [/] ->
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[TimeoutException: Future timed out after [30 seconds]]]
In the past, the connection was successful with host="localhost" and even an Atlas cluster (host="mycluster.gepld.mongodb.net") for the hostname, so there were no problems connecting previously with the same code. For some reason, Play Framework does not want to connect to this endpoint!
Could it be because the hostname is an IP address? Or, maybe Play/ Akka is doing something under the covers to stop the connection (or something to make Mongo/Docker refuse to accept it?)?
I'm using this driver:
"org.mongodb.scala" %% "mongo-scala-driver" % "4.4.0"
Perhaps I should switch to the Reactive Scala Driver? Any help would be appreciated.
Clarifications:
The Mongo DB Docker container is running on a linux machine on my local network. This container is reachable from within my local network at 192.168.0.229. The goal is to set my Play Framework app configuration to point to the DB at this address, so that as long as the Docker container is running, I can develop from any computer on my local network. Currently, I am able to access the container through the mongo shell on any computer:
> mongo "mongodb://192.168.0.229:27017/test"
I have a Play Framework app with the following in the Application.conf:
datastore {
# Dev
host: "192.168.0.229"
port: 27017
dbname: "test"
user: ""
password: ""
}
This data is used in a connection helper file called DataStore.scala:
package model.db.mongo
import org.mongodb.scala._
import utils.config.AppConfiguration
trait DataStore extends AppConfiguration {
lazy val dbHost = config.getString("datastore.host")
lazy val dbPort = config.getInt("datastore.port")
lazy val dbUser = getConfigString("datastore.user", "")
lazy val dbName = getConfigString("datastore.dbname", "")
lazy val dbPasswd = getConfigString("datastore.password", "")
//MongoDB Atlas Method (Localhost if DB User is empty)
val uri: String = s"mongodb+srv://$dbUser:$dbPasswd#$dbHost/$dbName?retryWrites=true&w=majority"
//val uri: String = "mongodb+svr://192.168.0.229:27017/?compressors=disabled&gssapiServiceName=mongodb"
System.setProperty("org.mongodb.async.type", "netty")
val mongoClient: MongoClient = if (getConfigString("datastore.user", "").isEmpty()) MongoClient() else MongoClient(uri)
print(mongoClient.toString)
print(mongoClient.listDatabaseNames())
val database: MongoDatabase = mongoClient.getDatabase(dbName)
def close = mongoClient.close() //Do this when logging out
}
When you start the app, you open localhost:9000 which is simply a login form. When you fill out the data that corresponds with the data in the users collection, the Play app times out:
[error] p.a.h.DefaultHttpErrorHandler -
! #7m884abc4 - Internal server error, for (POST) [/] ->
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[TimeoutException: Future timed out after [30 seconds]]]
at play.api.http.HttpErrorHandlerExceptions$.$anonfun$convertToPlayException$2(HttpErrorHandler.scala:381)
at scala.Option.map(Option.scala:242)
at play.api.http.HttpErrorHandlerExceptions$.convertToPlayException(HttpErrorHandler.scala:380)
at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:373)
at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:264)
at play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:430)
at play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:422)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:454)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100)
Caused by: java.util.concurrent.TimeoutException: Future timed out after [30 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.tryAwait0(Promise.scala:212)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:225)
at scala.concurrent.Await$.$anonfun$result$1(package.scala:201)
at akka.dispatch.MonitorableThreadFactory$AkkaForkJoinWorkerThread$$anon$3.block(ThreadPoolBuilder.scala:174)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at akka.dispatch.MonitorableThreadFactory$AkkaForkJoinWorkerThread.blockOn(ThreadPoolBuilder.scala:172)
at akka.dispatch.BatchingExecutor$BlockableBatch.blockOn(BatchingExecutor.scala:116)
at scala.concurrent.Await$.result(package.scala:124)
at model.db.mongo.DataHelpers$ImplicitObservable.headResult(DataHelpers.scala:27)
at model.db.mongo.DataHelpers$ImplicitObservable.headResult$(DataHelpers.scala:27)
The call to the Users collection is defined in UserAccounts.scala:
case class UserAccount(_id: String, fullname: String, username: String, password: String)
object UserAccount extends DataStore {
val logger: Logger = Logger("database")
//Required for using Case Classes
val codecRegistry = fromRegistries(fromProviders(classOf[UserAccount]), DEFAULT_CODEC_REGISTRY)
//Using Case Class to get a collection
val coll: MongoCollection[UserAccount] = database.withCodecRegistry(codecRegistry).getCollection("users")
//Using Document to get a collection
val listings: MongoCollection[Document] = database.getCollection("users")
def isValidLogin(username: String, password: String): Boolean = {
findUser(username) match {
case Some(u: UserAccount) => if(password.equals(u.password)) { true } else {false }
case None => false
}
}
Just an FYI if anyone runs into this problem. I had a bad line in my DataStore.scala file:
val mongoClient: MongoClient = if (getConfigString("datastore.user", "").isEmpty()) MongoClient() else MongoClient(uri)
Since I was trying to connect without a username (there's no auth on my test db), the above line was saying, "If there's no username, you must be trying to connect to the default MongoClient() location, which is localhost". My mistake.
I simply changed the above line to this:
val mongoClient: MongoClient = MongoClient(uri)
I am running my DSE Graph inside a Docker container with this command.
docker run -e DS_LICENSE=accept -p 9042:9042 --name my-dse -d datastax/dse-server -g -k -s
On my Scala Code I am referencing it as follows
object GrDbConnection {
val dseCluster = DseCluster.builder()
.addContactPoints("127.0.0.1").withPort(9042)
.build()
val graphName = "graphName"
val graphOptions = new GraphOptions()
.setGraphName(graphName)
var graph:ScalaGraph = null
try {
val session = dseCluster.connect()
// The following uses the DSE graph schema API, which is currently only supported by the string-based
// execution interface. Eventually there will be a programmatic API for making schema changes, but until
// then this needs to be used.
// Create graph
session.executeGraph("system.graph(name).ifNotExists().create()", ImmutableMap.of("name", graphName))
// Clear the schema to drop any existing data and schema
session.executeGraph(new SimpleGraphStatement("schema.clear()").setGraphName(graphName))
// Note: typically you would not want to use development mode and allow scans, but it is good for convenience
// and experimentation during development.
// Enable development mode and allow scans
session.executeGraph(new SimpleGraphStatement("schema.config().option('graph.schema_mode').set('development')")
.setGraphName(graphName))
session.executeGraph(new SimpleGraphStatement("schema.config().option('graph.allow_scan').set('true')")
.setGraphName(graphName))
// Create a ScalaGraph from a remote Traversal Source using withRemote
// See: http://tinkerpop.apache.org/docs/current/reference/#connecting-via-remotegraph for more details
val connection = DseRemoteConnection.builder(session)
.withGraphOptions(graphOptions)
.build()
graph = EmptyGraph.instance().asScala
.configure(_.withRemote(connection))
} finally {
dseCluster.close()
}
}
Then In one of my controllers I have used this to invoke a query to the DSE Graph.
def test = Action {
val r = GrDbConnection.graph.V().count()
print(r.iterate())
Ok
}
This returns an error
lay.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[NoHostAvailableException: All host(s) tried for query failed (no host was tried)]]
at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:251)
at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:178)
at play.core.server.AkkaHttpServer$$anonfun$1.applyOrElse(AkkaHttpServer.scala:363)
at play.core.server.AkkaHttpServer$$anonfun$1.applyOrElse(AkkaHttpServer.scala:361)
at scala.concurrent.Future.$anonfun$recoverWith$1(Future.scala:413)
at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)
at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:221)
at com.datastax.driver.core.RequestHandler.access$1000(RequestHandler.java:41)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:292)
at com.datastax.driver.core.RequestHandler.startNewExecution(RequestHandler.java:109)
at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:89)
at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:124)
at com.datastax.driver.dse.DefaultDseSession.executeGraphAsync(DefaultDseSession.java:123)
at com.datastax.dse.graph.internal.DseRemoteConnection.submitAsync(DseRemoteConnection.java:74)
at org.apache.tinkerpop.gremlin.process.remote.traversal.step.map.RemoteStep.promise(RemoteStep.java:89)
at org.apache.tinkerpop.gremlin.process.remote.traversal.step.map.RemoteStep.processNextStart(RemoteStep.java:65)
Turns out all I needed to do was to remove
finally {
dseCluster.close()
}
I want to develop my own workflow named "backup" in cloudify with my own plugin, but when i ran that workflow, the below error occured
'backup' workflow execution failed: RuntimeError: Workflow failed: Task failed 'script_runner.tasks.run' -> Missing cloudify_agent runtime information. This most likely means that the Compute node never started successfully
I don't understand why, anybody can solved me this problem?
Here is my main blueprint code and plugin code
My main blueprint
tosca_definitions_version: cloudify_dsl_1_2
imports:
- plugins/backup.yaml
- types/types.yaml
node_templates:
mynode:
type: cloudify.nodes.Compute
properties:
ip: "ip"
agent_config:
install_method: none
user: "user"
key: "key_uri"
myapp:
type: cloudify.nodes.ApplicationModule
interfaces:
test_platform_backup:
backup:
implementation: scripts/backup.sh
inputs:
port: 6969
post_backup:
implementation: scripts/post_backup.sh
relationships:
- type: cloudify.relationships.contained_in
target: mynode
My plugin code:
from cloudify.decorators import workflow
from cloudify.workflows import ctx
from cloudify.workflows.tasks_graph import forkjoin
#workflow
def backup(operation, type_name, operation_kwargs, is_node_operation, **kwargs):
graph = ctx.graph_mode()
send_event_starting_tasks = {}
send_event_done_tasks = {}
for node in ctx.nodes:
if type_name in node.type_hierarchy:
for instance in node.instances:
send_event_starting_tasks[instance.id] = instance.send_event('Starting to run operation')
send_event_done_tasks[instance.id] = instance.send_event('Done running operation')
for node in ctx.nodes:
if type_name in node.type_hierarchy:
for instance in node.instances:
sequence = graph.sequence()
if is_node_operation:
operation_task = instance.execute_operation(operation, kwargs=operation_kwargs)
else:
forkjoin_tasks = []
for relationship in instance.relationships:
forkjoin_tasks.append(relationship.execute_source_operation(operation))
forkjoin_tasks.append(relationship.execute_target_operation(operation))
operation_task = forkjoin(*forkjoin_tasks)
sequence.add(
send_event_starting_tasks[instance.id],
operation_task,
send_event_done_tasks[instance.id])
for node in ctx.nodes:
for instance in node.instances:
for rel in instance.relationships:
instance_starting_task = send_event_starting_tasks.get(instance.id)
target_done_task = send_event_done_tasks.get(rel.target_id)
if instance_starting_task and target_done_task:
graph.add_dependency(instance_starting_task, target_done_task)
return graph.execute()
It seems that your VM did not start.
From your code I can't understand what you are trying to do.
You don't install and agent and you don't have a fabric connection to the VM, yet you are trying to run operations on the VM.
You should either install an agent, E.g remove the "install_method: none", or add a fabric connection to the VM and run the operations with it.
I am connecting to MongoDB while using the Scala Play! framework. I end up getting this timeout error:
! #6j672dke5 - Internal server error, for (GET) [/accounts] ->
play.api.Application$$anon$1: Execution exception[[MongoTimeoutException: Timed out while waiting to connect after 10000 ms]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10-2.2.1.jar:2.2.1]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10-2.2.1.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$12$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:165) [play_2.10-2.2.1.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$12$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:162) [play_2.10-2.2.1.jar:2.2.1]
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33) [scala-library-2.10.4.jar:na]
at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:185) [scala-library-2.10.4.jar:na]
Caused by: com.mongodb.MongoTimeoutException: Timed out while waiting to connect after 10000 ms
at com.mongodb.BaseCluster.getDescription(BaseCluster.java:131) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.DBTCPConnector.getClusterDescription(DBTCPConnector.java:396) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.DBTCPConnector.getType(DBTCPConnector.java:569) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.DBTCPConnector.isMongosConnection(DBTCPConnector.java:370) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.Mongo.isMongosConnection(Mongo.java:645) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.DBCursor._check(DBCursor.java:454) ~[mongo-java-driver-2.12.3.jar:na]
Here is my Scala code for connecting to the database:
//models.scala
package models.mongodb
//imports
package object mongoContext {
//context stuff
val client = MongoClient(current.configuration.getString("mongo.host").toString())
val database = client(current.configuration.getString("mongo.database").toString())
}
Here is the actual model that is making the connection:
//google.scala
package models.mongodb
//imports
case class Account(
id: ObjectId = new ObjectId,
name: String
)
object AccountDAO extends SalatDAO[Account, ObjectId](
collection = mongoContext.database("accounts")
)
object Account {
def all(): List[Account] = AccountDAO.find(MongoDBObject.empty).toList
}
Here's the Play! framework MongoDB conf information:
# application.conf
# mongodb connection details
mongo.host="localhost"
mongo.port=27017
mongo.database="advanced"
Mongodb is running on my local machine. I can connect to it by typing mongo at the terminal window. Here's the relevant part of the conf file:
# mongod.conf
# Where to store the data.
# Note: if you run mongodb as a non-root user (recommended) you may
# need to create and set permissions for this directory manually,
# e.g., if the parent directory isn't mutable by the mongodb user.
dbpath=/var/lib/mongodb
#where to log
logpath=/var/log/mongodb/mongod.log
logappend=true
#port = 27017
# Listen to local interface only. Comment out to listen on all interfaces.
#bind_ip = 127.0.0.1
So what's causing this timeout error and how do I fix it? Thanks!
I figured out that I needed to change:
val client = MongoClient(current.configuration.getString("mongo.host").toString())
val database = client(current.configuration.getString("mongo.database").toString())
to:
val client = MongoClient(conf.getString("mongo.host"))
val database = client(conf.getString("mongo.database"))