Atlas Search Autocomplete on Subdocument Array - mongodb

I am having some trouble with the autocomplete atlas search data type when trying to define an index for an array of subdocuments in my document.
My data structure for the documents in my collection looks like this:
{
"data": {
"equipment": {
"entries": [
{
"name": "abcdefg"
}
{
"name": "hijklmno"
}
]
}
}
}
When I define a string index for searching the entries array, it works as intended and I get logical results. Here is my index definition using the lucene.keyword analyzer:
{
"mappings": {
"dynamic": false,
"fields": {
"data": {
"fields": {
"equipment": {
"fields": {
"entries": {
"fields": {
"name": {
"analyzer": "lucene.keyword",
"searchAnalyzer": "lucene.keyword",
"type": "string"
}
},
"type": "document"
}
},
"type": "document"
}
},
"type": "document"
}
}
}
}
However, when I try the same thing with the autocomplete type, I get an empty result, but no error. Here is how I defined the autocomplete:
{
"mappings": {
"dynamic": false,
"fields": {
"data": {
"fields": {
"equipment": {
"fields": {
"entries": {
"fields": {
"name": {
"tokenization": "nGram",
"type": "autocomplete"
}
},
"type": "document"
}
},
"type": "document"
}
},
"type": "document"
}
}
}
}
The documentation for Atlas Search states the following:
The autocomplete type can't be used to index fields whose value is an array of strings. So either this sentence has to be changed to say all kinds of arrays or I am doing something wrong here. Can someone clarify if this is even possible?
Thanks in Advance

Your syntax is completely wrong. its would be like:
{
"mappings": {
"dynamic": false,
"fields": {
"data.equipment.entries.name": [
{
"type": "autocomplete",
"tokenization": "nGram",
"minGrams": 3,
"maxGrams": 7,
}
]
}
}
}
But I am not sure that,if it support on array of document, But let me know if your problem is solved.

Related

Elastic4s ngram mapping

I have to create an ElasticSearch mapping like this using elastic4s:
"mappings": {
"properties": {
"id": {
"type": "keyword"
},
"name": {
"type": "text",
"analyzer": "ngram_analyzer",
"fielddata": true
},
"lang": {
"type": "keyword"
},
"order": {
"type": "long"
},
"active": {
"type": "boolean"
}
"description": {
"type": "text"
}
}
}
I can do
def mapping: Option[MappingDefinition] =
Some(
properties(
KeywordField("id"),
KeywordField("lang"),
BasicField("order", "long"),
BasicField("active", "boolean"),
TextField("description")
)
)
for id, lang, order, active and description.
But, how can I do such mapping for name. the problem is analyzer and fielddata inside it.
You should use this:
TextField("name").fielddata(true).analyzer("ngram_analyzer")
You also need to make sure to properly create the ngram_analyzer in your index settings.

Using a structured query to fetch values based on key in array

In my firestore I have a document structured as shown below.
I am trying to set up a Zapier Zap that will allow me to fetch the store name, based on the storeId. It requires a JSON structured query that fetches the data.
Is it possible to do, and where should I begin, I find the documentation lacking examples. The only query I have right now is as shown below, but obviously, it does not work since all data is in a single document.
"where": {
"fieldFilter": {
"field": {
"fieldPath": "stores/*/storeId"
},
"op": "EQUAL",
"value": {
"stringValue": "def"
}
}
}
Document structure
{
"stores": {
"0": {
"storeId": "abc",
"name": "Store 1"
},
"1": {
"storeId": "def",
"name": "Store 2"
}
}
}
I don't think you can do this with an array field, but you can if you change that array to a map field. Something like this:
mycollection/mydoc
{
"someotherfield": "foo",
"stores": {
"abc": {
"name": "Store 1"
},
"def": {
"name": "Store 2"
}
}
}
Now you can use a field mask:
GET https://firestore.googleapis.com/v1/projects/MY-PROJECT/databases/(default)/documents/mycollection/mydoc?mask.fieldPaths=stores.abc.name
Response:
{
"name": "projects/MY-PROJECT/databases/(default)/documents/mycollection/mydoc",
"fields": {
"stores": {
"mapValue": {
"fields": {
"abc": {
"mapValue": {
"fields": {
"name": {
"stringValue": "Store 1"
}
}
}
}
}
}
}
},
"createTime": "2019-04-20T19:14:01.792855Z",
"updateTime": "2019-04-22T18:07:05.660561Z"
}

swagger with list of elements in an array

I am new to swagger implementation. I have a query parameter 'Geschaeftsvorfall' which can be of string value A or P and when I hit the end point. I expect an array[validPsd2Ids] filled with integers.
I have formulated below code and I don't know how to validate it. can someone tell me if I am going wrong some where?
Also what can I do to print a List instead of array in my response?
"parameters": {
"Geschaeftsvorfall": {
"name": "Geschaeftsvorfall",
"in": "query",
"description": "Geschaeftsvorfall",
"required": true,
"type": "string",
"enum": [
"A",
"P"
]
}
},
"definitions": {
"ValidePsd2Ids": {
"type": "array",
"items": {
"properties": {
"ValidePsd2Ids": {
"type": "integer",
example: [100000005,
100000006,
100000007,
100000008,
100000009,
100000010,
100000011,
100000012,
100000013,
100000014,
100000015,
100000016,
100000017,
100000018,
100000019,
100000020,
100000021,
100000022,
100000023,
100000024,
100000025,
100000034,
100000035,
100000036,
100000037,
100000038,
100000039,
100000048,
100000049,
100000050,
100000054,
100000055,
100000056,
100000057,
100000058,
100000117,
100000163,
100000165,
100000195,
100000196,
100000197,
100000198,
100000199,
100000201,
100000214,
100000217,
100000218]
}
}
}
}
},
"paths": {
"/payments/validaccounttypes/": {
"get": {
"tags": [
"payments"
],
"summary": "Valid PSD2 relevant accounts",
"description": "Reads the list of valid PSD2 revelant IDs.",
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"parameters": [
{
"$ref": "#/parameters/Geschaeftsvorfall"
}
],
"responses": {
"200": {
"description": "OK",
"schema": {
"type": "array",
"items": {
"properties": {
"ValidePsd2Ids": {
"type": "integer"
}
}
},
"properties": {
"ValidePsd2Ids": {
"$ref": "#/definitions/ValidePsd2Ids"
}
}
}
}
}
}
}
}
The parameter definition is correct.
The response definition is not correct. You say that the response looks like
{"ValidePsd2Ids" : [1,2,3,4,5,6,7,...]}
In OpenAPI terms, this is a type: object with a property ValidePsd2Ids that contains an array of integers. This can be described as:
"definitions": {
"ValidePsd2Ids": {
"type": "object",
"properties": {
"ValidePsd2Ids": {
"type": "array",
"items": {
"type": "integer"
},
"example": [
100000005,
100000006,
100000007
]
}
}
}
},
and the responses should be:
"responses": {
"200": {
"description": "OK",
"schema": {
"$ref": "#/definitions/ValidePsd2Ids"
}
}
}

How can I use CloudKit web services to query based on a reference field?

I've got two CloudKit data objects that look somewhat like this:
Parent Object:
{
"records": [
{
"recordName": "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57",
"recordType": "ParentObject",
"fields": {
"fsYear": {
"value": "2015",
"type": "STRING"
},
"displayOrder": {
"value": 2015221153856287200,
"type": "INT64"
},
"fjpFSGuidForReference": {
"value": "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57",
"type": "STRING"
},
"fsDateSearch": {
"value": "2015221153856287158",
"type": "STRING"
},
},
"recordChangeTag": "id4w7ivn",
"created": {
"timestamp": 1439149087571,
"userRecordName": "_0d26968032e31bbc72c213037b6cb35d",
"deviceID": "A19CD995FDA3093781096AF5D818033A241D65C1BFC3D32EC6C5D6B3B4A9AA6B"
},
"modified": {
"timestamp": 1439149087571,
"userRecordName": "_0d26968032e31bbc72c213037b6cb35d",
"deviceID": "A19CD995FDA3093781096AF5D818033A241D65C1BFC3D32EC6C5D6B3B4A9AA6B"
}
}
],
"total":
}
Child Object:
{
"records": [
{
"recordName": "2015221153856287168",
"recordType": "ChildObject",
"fields": {
"District": {
"value": "002",
"type": "STRING"
},
"ZipCode": {
"value": "12345",
"type": "STRING"
},
"InspecReference": {
"value": {
"recordName": "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57",
"action": "NONE",
"zoneID": {
"zoneName": "_defaultZone"
}
},
"type": "REFERENCE"
},
},
"recordChangeTag": "id4w7lew",
"created": {
"timestamp": 1439149090856,
"userRecordName": "_0d26968032e31bbc72c213037b6cb35d",
"deviceID": "A19CD995FDA3093781096AF5D818033A241D65C1BFC3D32EC6C5D6B3B4A9AA6B"
},
"modified": {
"timestamp": 1439149090856,
"userRecordName": "_0d26968032e31bbc72c213037b6cb35d",
"deviceID": "A19CD995FDA3093781096AF5D818033A241D65C1BFC3D32EC6C5D6B3B4A9AA6B"
}
}
],
"total": 1
}
I'm trying to write a query to directly access the CloudKit web service and return the Child Object based on the reference of the parent object.
My test JSON looks something like this:
{"query":{"recordType":"ChildObject","filterBy":{"fieldName":"InspecReference","fieldValue":{ "value" : "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57", "type" : "string" },"comparator":"EQUALS"}},"zoneID":{"zoneName":"_defaultZone"}}
However, I'm getting the following error from CloudKit:
{"uuid":"33db91f3-b768-4a68-9056-216ecc033e9e","serverErrorCode":"BAD_REQUEST","reason":"BadRequestException:
Unexpected input"}
I'm guessing I have the Record Field Dictionary in the query wrong. However, the documentation isn't clear on what this should look like on a reference object.
You have to re-create the actual object of the reference. In this particular case, the JSON looks like this:
{
"query": {
"recordType": "ChildObject",
"filterBy": {
"fieldName": "InspecReference",
"fieldValue": {
"value": {
"recordName": "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57",
"action": "NONE"
},
"type": "REFERENCE"
},
"comparator": "EQUALS"
}
},
"zoneID": {
"zoneName": "_defaultZone"
}
}

ElasticSearch autocomplete returning 0 hits

I am trying to build an autocomplete feature for our database running on MongoDB. We need to provide autocomplete which lets users complete their queries by offering suggestions while they are typing in the search box.
I have a collection of articles from various sources, which is having the following fields :
{
"title" : "Its the title of a random article",
"cont" : { "paragraphs" : [ .... ] },
and so on..
}
I went through a video by Clinton Gormley. From 37:00 through 42:00 minute, Gormley describes an autocomplete using edgeNGram. Also, I referred to this question to recognize that both are almost the same things, just the mappings differ.
So based on these experiences, I built almost identical settings and mapping and then restored articles collection to ensure that it is indexed by ElasticSearch
The indexing scheme is as follows:
POST /title_autocomplete/title
{
"settings": {
"analysis": {
"filter": {
"autocomplete": {
"type": "edgeNGram",
"min_gram": 2,
"max_gram": 50
}
},
"analyzer": {
"title" : {
"type" : "standard",
"stopwords":[]
},
"autocomplete": {
"type" : "autocomplete",
"tokenizer": "standard",
"filter": ["lowercase", "autocomplete"]
}
}
}
},
"mappings": {
"title": {
"type": "multi_field",
"fields" : {
"title" : {
"type": "string",
"analyzer": "title"
},
"autocomplete" : {
"type": "string",
"index_analyzer": "autocomplete",
"search_analyzer" : "title"
}
}
}
}
}
But when I run the search query, I am unable to get any hits!
GET /title_autocomplete/title/_search
{
"query": {
"bool" : {
"must" : {
"match" : {
"title.autocomplete" : "Its the titl"
}
},
"should" : {
"match" : {
"title" : "Its the titl"
}
}
}
}
}
Can anybody please explain what's wrong with the mapping query or settings? I have been reading ElasticSearch docs for over 7 days now but seem to get nowhere more than full text searches!
ElastiSearch version : 0.90.10
MongoDB version : v2.4.9
using _river
Ubuntu 12.04 64bit
UPDATE
I realised that mapping is screwed after applying previous settings:
GET /title_autocomplete/_mapping
{
"title_autocomplete": {
"title": {
"properties": {
"analysis": {
"properties": {
"analyzer": {
"properties": {
"autocomplete": {
"properties": {
"filter": {
"type": "string"
},
"tokenizer": {
"type": "string"
},
"type": {
"type": "string"
}
}
},
"title": {
"properties": {
"type": {
"type": "string"
}
}
}
}
},
"filter": {
"properties": {
"autocomplete": {
"properties": {
"max_gram": {
"type": "long"
},
"min_gram": {
"type": "long"
},
"type": {
"type": "string"
}
}
}
}
}
}
},
"content": {
... paras and all ...
}
"title": {
"type": "string"
},
"url": {
"type": "string"
}
}
}
}
}
Analyzers and filters are actually mapped into the document after the settings are applied whereas original title field is not affected at all! Is this normal??
I guess this explains why the query is not matching. There is no title.autocomplete field or title.title field at all.
So how should I proceed now?
For those facing this problem, its better to delete the index and start again instead of wasting time with the _river just as DrTech pointed out in the comment.
This saves time but is not a solution. (Therefore not marking it as answer.)
The key is to set up the mappings and index before you initiate the river.
We had an existing setup with a mongodb river and an index called coresearch that we wanted to add autocomplete capacity to, this is the set of commands we used to delete the existing index and river and start again.
Stack is:
ElasticSearch 1.1.1
MongoDB 2.4.9
ElasticSearchMapperAttachments v2.0.0
ElasticSearchRiverMongoDb/2.0.0
Ubuntu 12.04.2 LTS
curl -XDELETE "localhost:9200/_river/node"
curl -XDELETE "localhost:9200/coresearch"
curl -XPUT "localhost:9200/coresearch" -d '
{
"settings": {
"analysis": {
"filter": {
"autocomplete_filter": {
"type": "edge_ngram",
"min_gram": 1,
"max_gram": 20
}
},
"analyzer": {
"autocomplete": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"autocomplete_filter"
]
}
}
}
}
}'
curl -XPUT "localhost:9200/coresearch/_mapping/users" -d '{
"users": {
"properties": {
"firstname": {
"type": "string",
"search_analyzer": "standard",
"index_analyzer": "autocomplete"
},
"lastname": {
"type": "string",
"search_analyzer": "standard",
"index_analyzer": "autocomplete"
},
"username": {
"type": "string",
"search_analyzer": "standard",
"index_analyzer": "autocomplete"
},
"email": {
"type": "string",
"search_analyzer": "standard",
"index_analyzer": "autocomplete"
}
}
}
}'
curl -XPUT "localhost:9200/_river/node/_meta" -d '
{
"type": "mongodb",
"mongodb": {
"servers": [
{ "host": "127.0.0.1", "port": 27017 }
],
"options":{
"exclude_fields": ["time"]
},
"db": "users",
"gridfs": false,
"options": {
"import_all_collections": true
}
},
"index": {
"name": "coresearch",
"type": "documents"
}
}'