Elasticsearch mongodb river script in index doesn't work - mongodb

I'm trying to change few fields strings using javascript.
For example take only the last part of the URL taken from mongo through the river so in elasticsearch I'll have only the end of it.
When creating the index (using curl) I added under "options" the following script:
"script": "ctx.document.shorturl = ctx.document.url.substr(-4);delete ctx.document.url;
I tried some manipulations such as adding \"...\" or use ctx['doc']['url'] and others but nothing seems to work.
I always get only url field with the full url (shorturl is not created at all).
Can anyone suggest what is the right syntax to make it work?
Another thing I need to do is combine to fields - lat & long, to one "location" field in order to use it in Kibana, can anyone suggest the right script for that? (create new field called "location" which contain both field "lat" & "long" with comma between them).
Thanks.

You did substring(-4), hence it will return the whole string. You should use substring(4) instead:
ctx.document.shorturl = ctx.document.url.substr(4);delete ctx.document.url;

Related

Data factory lookup (dot) in the item() name

I am having lookup wherein salesforce query is there. I am using elements (item()) in subsequent activities. Till now i had item().name or item().email but now i have item().NVMStatsSF__Related_Lead__r.FirstName which has (dot) in the field name.
How should i parse it through body tag so that it reads it correctly?
So I have the following data in item()
{
"NVMStatsSF__Related_Lead__c": "00QE000egrtgrAK",
"NVMStatsSF__Agent__r.Name": "ABC",
"NVMStatsSF__Related_Lead__r.Email": "geggegg#gmail.com",
"NVMStatsSF__Related_Lead__r.FirstName": "ABC",
"NVMStatsSF__Related_Lead__r.OwnerId": "0025434535IIAW"
}
now when i use item().NVMStatsSF__Agent__r.Name it will not parse because of (dot) after NVMStatsSF__Agent__r. And it is giving me the following error.
'item().NVMStatsSF__Related_Lead__r.Email' cannot be evaluated because property 'NVMStatsSF__Related_Lead__r' doesn't exist, available properties are 'NVMStatsSF__Related_Lead__c, NVMStatsSF__Agent__r.Name, NVMStatsSF__Related_Lead__r.Email, NVMStatsSF__Related_Lead__r.FirstName, NVMStatsSF__Related_Lead__r.OwnerId'.",
"failureType": "UserError",
"target": "WebActivityToAddPerson"
this is because ADF uses '.' for object reading.
Could you find a way to rename the field name which contains '.'?
Seems like you need a built-in function to get the value of an object according to the key. Like getValue(item(), 'key.nestkey'). But unfortunately, seems there isn't such a function. You may need handle your key first.
Finally, it worked. I was being silly.
Instead of taking the value from the child table with the help of (dot) operator I just used subquery. Silly see.
And it worked.

How can you filter search by matching String to a field in Algolia?

I'm trying to create filters for a search on an Android app where a specific field in Algolia must exactly match the given String in order to come up as a hit. For example if Algolia has a field like "foo" and I only want to return hits where "foo" is equal to "bar", then I would expect that I would have to use a line of code like this:
query.setFilters("foo: \"bar\"");
Any guesses as to why this isn't working like I see in the examples or how to do so?
Ah, I thought that attributesForFaceting was done by setting what was searchable or not. It was on a different page within the dashboard than I was previously using. Thanks #pixelastic.

AEM CQ5 Query Builder: How to get result by searching for 2 different values in same property?

I want to get result matches with all nodes contains property 'abc' value as 'xyz' or 'pqr'.
I am trying in below ways:
http://localhost:4502/bin/querybuilder.json?path=/content/campaigns/asd&property=abc&property.1_value=/%xyz/%&property.2_value=/%pqr/%property.operation=like&p.limit=-1&orderby:path
http://localhost:4502/bin/querybuilder.json?path=/content/campaigns/asd&property=abc&property.1_value=/%xyz/%&property.2_value=/%pqr/%&property.1_operation=like&property.2_operation=like&p.limit=-1&orderby:path
http://localhost:4502/bin/querybuilder.json?path=/content/campaigns/asd&1_property=abc&1_property.1_value=/%xyz/%&1_property.1_operation=like&2_property=abc&1_property.1_value=/%xyz/%&2_property.1_operation=like&p.limit=-1&orderby:path
But none of them served my purpose. Any thing that I am missing in this?
The query looks right and as such should work. However if it is just xyz or pqr you would like to match in the query, you may not need the / in the values.
For eg.
path=/content/campaigns/asd
path.self=true //In order to include the current path as well for searching
property=abc
property.1_value=%xyz%
property.2_value=%abc%
property.operation=like
p.limit=-1
Possible things which you can check
Check if the path that you are trying to search contains the desired nodes/properties.
Check if the property name that you are using is right.
If you want to match exact values, you can avoid using the like operator and remove the wild cards from the values.
You can actually use the 'OR' operator in your query to combine two or more values of a property.
For example in the query debug interface : http:///libs/cq/search/content/querydebug.html
path=/content/campaigns/asd
property=PROPERTY1
property.1_value=VALUE1
property.2_value=VALUE2
property.operation=OR
p.limit=-1
It worked with below query:
http://localhost:4502/bin/querybuilder.json?orderby=path
&p.limit=-1
&path=/content/campaigns
&property=jcr:content/par/nodeName/xyz
&property.1_value=pqr
&property.2_value=%abc%
&property.operation=like
&type=cq:Page
Note: property name should be fully specified form the type of node we are expecting.
Ex: jcr:content/par/nodeName/xyz above instead of just xyz

whoosh doesn't search for short words like "C#"

i am using whoosh to index over 200,000 books. but i have encountered some problems with it.
the whoosh query parser returns NullQuery for words like "C#", "C++" with meta-characters in them and also for some other short words. this words are used in the title and body of some documents so i am not using keyword type for them. i guess the problem is in the analysis or query-parsing phase of searching or indexing but i can't touch my data blindly. can anyone help me to correct this issue. Tnx.
i fixed the problem by creating a StandardAnalyzer with a regex pattern that meets my requirements,here is the regex pattern:
'\w+[#+.\w]*'
this will make tokenizing of fields to be done successfully, and also the searching goes well.
but when i use queries like "some query++*" or "some##*" the parsed query will be a single Every query, just the '*'. also i found that this is not related to my analyzer and this is the Whoosh's default behavior. so here is my new question: is this behavior correct or it is a bug??
note: removing the WildcardPlugin from the query-parser solves this problem but i also need the WildcardPlugin.
now i am using the following code:
from whoosh.util import rcompile
#for matching words like: '.NET', 'C++' and 'C#'
word_pattern = rcompile('(\.|[\w]+)(\.?\w+|#|\+\+)*')
#i don't need words shorter that two characters so i don't change the minsize default
analyzer = analysis.StandardAnalyzer(expression=word_pattern)
... now in my schema:
...
title = fields.TEXT(analyzer=analyzer),
...
this will solve my first problem, yes. but the main problem is in searching. i don't want to let users to search using the Every query or *. but when i parse queries like C++* i end up an Every(*) query. i know that there is some problem but i can't figure out what it is.
I had the same issue and found out that StandardAnalyzer() uses minsize=2 by default. So in your schema, you have to tell it otherwise.
schema = whoosh.fields.Schema(
name = whoosh.fields.TEXT(stored=True, analyzer=whoosh.analysis.StandardAnalyzer(minsize=1)),
# ...
)

Mongoid, find object by searching by part of the Id?

I want to be able to search for my objects by searching for the last 4 characters of the id. How can I do that?
Book.where(_id: params[:q])
Where the param would be something like a3f4, and in this case the actual id for the object that I want to be found would be:
bc313c1f5053b66121a8a3f4
Notice the last for characters are what we searched for. How can I search for just "part" of my objects id? instead of having my user search manually by typing in the entire id?
I found in MongoDB's help docs, that I can provide a regex:
db.x.find({someId : {$regex : "123\\[456\\]"}}) // use "\\" to escape
Is there a way for me to search using the regular mongo ruby driver and not using Mongoid?
Usually, in Mongoid you can search with a regexp like you normally would with a string in your call to where() ie:
Book.where(:title => /^Alice/) # returns all books with titles starting with 'Alice'
However this doesn't work in your case, because the _id field is not stored as a string, but as an ObjectID. However, you could add (and index) a field on your models which could provide this functionality for you, which you can populate in an after_create callback.
<shameless_plug>
Alternatively, if you're just looking for a shorter solution to the default Mongoid IDs, I could suggest something like mongoid_token which makes it pretty easy to add shorter tokens/ids to your Mongoid documents.
</shameless_plug>