I'm currently trying to create a percolator query with Elastic4s. I've got about this far but I can't seem to find any examples so I'm not sure how this quite works. So I've got:
val percQuery = percolate in esIndex / esType query myQuery
esClient.execute(percQuery)
Every time it runs it doesn't match anything. I figured out I need to be able to percolate on an Id but I can't seem to find any examples on how to do it, not even in the docs. I know with Elastic4s creating queries other than a percolator query lets you specify an id field like:
val query = index into esIndex / esType source myDoc id 12345
I've tried this way for percolate but it doesn't like the id field, does anyone know how this can be done?
I was using Dispatch Http to do this previously but I'm trying to move away from it. Before, I was doing this to submit the percolator query:
url(s"$esUrl/.percolator/$queryId)
.setContentType("application/json", "utf-8")
.setBody(someJson)
.POST
notice the queryId just need something similar to that but in elastic4s.
So you want to add a document and return the queries that are waiting for that id to be added? That seems an odd use for percolate as it will be a one time use only, as only one document can be added per id. You can't do a percolate currently on id in elastic4s, and I'm not sure if you can even do it in elasticsearch itself.
This is the best attempt I can come up with, where you have your own "id" field, which could mirror the 'proper' _id field.
object Test extends App {
import ElasticDsl._
val client = ElasticClient.local
client.execute {
create index "perc" mappings {
"idtest" as(
"id" typed StringType
)
}
}.await
client.execute {
register id "a" into "perc" query {
termQuery("id", "a")
}
}.await
client.execute {
register id "b" into "perc" query {
termQuery("id", "b")
}
}.await
val resp1 = client.execute {
percolate in "perc/idtest" doc("id" -> "a")
}.await
// prints a
println(resp1.getMatches.head.getId)
val resp2 = client.execute {
percolate in "perc/idtest" doc("id" -> "b")
}.await
// prints b
println(resp2.getMatches.head.getId)
}
Written using elastic4s 1.7.4
So after much more researching I figured out how this works with elastic4s. To do this in Elastic4s you actually have to use register instead of percolate like so:
val percQuery = register id queryId into esIndex query myQuery
This will register a percolator query at the id.
Related
I have approximately this database wrapper class:
class MyDatabase {
private def getMongoClientSettings(): MongoClientSettings = {
val uri = "mongodb://localhost:27017"
val mongoClientSettings = MongoClientSettings.builder().applyConnectionString(ConnectionString(uri))
.serverApi(ServerApi.builder().version(ServerApiVersion.V1).build())
.build()
mongoClientSettings
}
private val mongoClient = MongoClient(getMongoClientSettings)
private val db = mongoClient.getDatabase("myDatabase")
object groups {
val collection = db.getCollection("groups")
...
}
object posts {
val collection = db.getCollection("posts")
...
}
object likes {
val collection = db.getCollection("likes")
...
}
}
The problem is that some inserts succeed, while other inserts silently fail.
For example, in the groups object, this command succeed:
val doc = Document("_id" -> group.id, "name" -> group.name, "screenName" -> group.screenName,
"membersCount" -> group.membersCount, "lastInspectionDate" -> group.lastInspectionDate)
collection.insertOne(doc)
But, in the posts object, the inserts never succeed (here, the id field is not the primary key, the _id key is to be autogenerated and is not the same as id):
val doc = Document("id" -> post.id, "fromId" -> post.fromId, "ownerId" -> post.ownerId,
"publishDate" -> post.publishDate, "text" -> post.text, "isPinned" -> post.isPinned)
collection.insertOne(doc)
The question is: why for the posts collection, the inserts never succeed?
My thoughts:
maybe db.getCollection command somewhat disconfigures the other collections, and I would need to call db.getCollection right before the insertOne command? This is unrealistic.
I thought that the method exited before the insertOne succeed, but for the groups collection, there is no problem.
I tried to perform waiting using this command, still no posts inserted: Await.result(collection.insertOne(doc).toFuture, Duration.Inf)
I found some mentions about the need to "subscribe" to the observable to make the cold steam to start functioning, but I think this was relevant for older versions only.
Configuration:
Linux Ubuntu 18.04, Scala 2.12.14, Mongo-Scala-Driver 4.3.1, MongoDB 5.0.2.
Looking forward to your replies.
After the computer reload and using the Await.result(), everything started to work. Do not know why.
I'm using grails2.4.4 with mongodb plugin version 3.0.3. I'm facing issue while getting results of my domain object. I'm using below code:
My domain:
Employee{
ObjectId id
String name
}
I have list of ids , using below code to fetch employees: (Please note that below data is just for representing my problem. In realtime, my ids are random and so i can't use sorting, but i just want the result in the order of input.)
def idsList=[new ObjectId("2001"), new ObjectId("2002"), new ObjectId("2003")]
def results=Employee.findAllByIdInList(idsList)
Expected result:
[Employee#2001,Employee#2002,Employee#2003]
Actual result (not in order):
[Employee#2002, Employee#2003 , Employee#2001] or sometimes
[Employee#2003, Employee#2001 , Employee#2002]
For now i'm doing like this to get the output in desired order:
def results=[]
for(id in idsList){
def emp=Employee.findById(id)
results<<emp
}
But i want to do this with single call(findAllBy*InList) without iterating over objects. Can anyone advise how can i get the results in the order of input ids?
Have you tried
Employee.findAllByIdInList(idsList, [sort: 'id', order:'asc'])
? It should work as expected
If it doesn't, instead of for loop you can use
def result = idList.collect { id -> Employee.findById(id) }
This post is related to the issue already raised at Dynamically changing the database shard that I am connecting too.
Pointers to code that should be changed to implement this feature is given at https://github.com/slick/slick/issues/703
Am a newbie to Scala and Slick , Can I get some help ? , as to how to proceed implementing this feature. Is there any slick/scala pattern to do this at application level.
My problem is "I have pool of connections of different shards of MySQL, when I write a query/queries involving ID's (sharding keys), slick should dynamically run that particular query on the respective database shard"
For Example: If I write a query like this
val q = for ( user <- users.filter(_.name === "cat")
post <- posts.filter(_.postedBy === user.id)
comment <- comments.filter(_.postId === post.id)
} yield comment.content
q.run
a trivial case should be like one below.
users += User(id = 1, name = "cat", email = "cat#mat.com") => hits shard no 1
Even if the User ID, Post ID, Comment ID are dynamically produced, slick should hit the correct database shard using some sharding criteria ( key (ID) % 3 ) and everything should happen at the background just like single database query.
To implement the feature at application level
Is there any way to read Query object state dynamically so that I can write a function like
def func(q: Query[Something], shards: Map[Int, Database], num: Int): Unit = {
shards(q.getId % num).withSession{ implicit session => {
q.run
}
}
Usage:
val q = users.insert(User(id = 1, name = "cat", email = "cat#cat.com"))
func(q, shards, 10) => q executes on one of the 10 shards.
Thanks.
Using OrientDB 1.7-rc and Scala, I would like to insert a document (ODocument), into a document (not graph) database, with connections to other documents. How should I do this?
I've tried the following, but it seems to insert an embedded list of documents into the Package document, rather than connect the package to a set of Version documents (which is what I want):
val doc = new ODocument("Package")
.field("id", "MyPackage")
.field("versions", List(new ODocument("Version").field("id", "MyVersion")))
EDIT:
I've tried inserting a Package with connections to Versions through SQL, and that seems to produce the desired result:
insert into Package(id, versions) values ('MyPackage', [#10:3, #10:4] )
However, I need to be able to do this from Scala, which has yet to produce the correct results when loading the ODocument back. How can I do it (from Scala)?
You need to create the individual documents first and then inter-link them using below SQL commands.
Some examples given in OrientDB documentation
insert into Profile (name, friends) values ('Luca', [#10:3, #10:4] )
OR
insert into Profile SET name = 'Luca', friends = [#10:3, #10:4]
Check here for more details.
I tried posting in comments above, but somehow the code is not readable, so posting the response separately again.
Here is an example of linking two documents in OrientDB. This is take from documentation. Here we are adding new user in DB and connecting it to give role:
var db = orient.getDatabase();
var role = db.query("select from ORole where name = ?", roleName);
if( role == null ){
response.send(404, "Role not found", "text/plain", "Error: role name not found" );
} else {
db.begin();
try{
var result = db.save({ "#class" : "OUser", name : "Gaurav", password : "gauravpwd", roles : role});
db.commit();
return result;
}catch ( err ){
db.rollback();
response.send(500, "Error: Server", "text/plain", err.toString() );
}
}
Hope it helps you and others.
This is how to insert a Package with a linkset referring to an arbitrary number of Versions:
val version = new ODocument("Version")
.field("id", "1.0")
version.save()
val versions = new java.util.HashSet[ODocument]()
versions.add(version)
val package = new ODocument("Package")
.field("id", "MyPackage")
.field("versions", versions)
package.save()
When inserting a Java Set into an ODocument field, OrientDB understands this to mean one wants to insert a linkset, which is an unordered, unique, collection of references.
When reading the Package back out of the database, you should get hold of its Versions like this:
val versions = doc.field[java.util.HashSet[ODocument]]("versions").asScala.toSeq
As when the linkset of versions is saved, a HashSet should be used when loading the referenced ODocument instances.
Optionally, to enforce that Package.versions is in fact a linkset of Versions, you may encode this in the database schema (in SQL):
create property Package.versions linkset Version
I have not clearly issue during query using two criterials like Id and Other. I use a Repository storing some data like id,iso,value. I have created an index("_id","Iso") to performs queries but queries are only returning my cursor if i use only one criterial like _id, but is returning nothing if a use two (_id, Iso) (commented code).
Are the index affecting the response or the query method are failing?
use :v1.6.5 and C# official.
Sample.
//Getting Data
public List<BsonObject> Get_object(string ID, string Iso)
{
using (var helper = BsonHelper.Create())
{
//helper.Db.Repository.EnsureIndex("_Id","Iso");
var query = Query.EQ("_Id", ID);
//if (!String.IsNullOrEmpty(Iso))
// query = Query.And(query, Query.EQ("Iso", Iso));
var cursor = helper.Db.Repository.FindAs<BsonObject>(query);
return cursor.ToList();
}
}
Data:
{
"_id": "2345019",
"Iso": "UK",
"Data": "Some data"
}
After that I have Updated my data using Update.Set() methods. I can see the changed data using MongoView. The new data are correct but the query is always returning the sames olds values. To see these values i use a page that can eventually cached, but if add a timestamp at end are not changing anything, page is always returning the same olds data. Your comments are welcome, thanks.
I do not recall offhand how the C# driver creates indexes, but the shell command for creating an index is like this:
db.things.ensureIndex({j:1});
Notice the '1' which is like saying 'true'.
In your code, you have:
helper.Db.Repository.EnsureIndex("_Id","Iso");
Perhaps it should be:
helper.Db.Repository.EnsureIndex("_Id", 1);
helper.Db.Repository.EnsureIndex("Iso", 1);
It could also be related to the fact that you are creating indexes on "_Id" and the actual id field is called "_id" ... MongoDB is case sensitive.
Have a quick look through the index documentation: http://www.mongodb.org/display/DOCS/Indexes