Using a structured query to fetch values based on key in array - google-cloud-firestore

In my firestore I have a document structured as shown below.
I am trying to set up a Zapier Zap that will allow me to fetch the store name, based on the storeId. It requires a JSON structured query that fetches the data.
Is it possible to do, and where should I begin, I find the documentation lacking examples. The only query I have right now is as shown below, but obviously, it does not work since all data is in a single document.
"where": {
"fieldFilter": {
"field": {
"fieldPath": "stores/*/storeId"
},
"op": "EQUAL",
"value": {
"stringValue": "def"
}
}
}
Document structure
{
"stores": {
"0": {
"storeId": "abc",
"name": "Store 1"
},
"1": {
"storeId": "def",
"name": "Store 2"
}
}
}

I don't think you can do this with an array field, but you can if you change that array to a map field. Something like this:
mycollection/mydoc
{
"someotherfield": "foo",
"stores": {
"abc": {
"name": "Store 1"
},
"def": {
"name": "Store 2"
}
}
}
Now you can use a field mask:
GET https://firestore.googleapis.com/v1/projects/MY-PROJECT/databases/(default)/documents/mycollection/mydoc?mask.fieldPaths=stores.abc.name
Response:
{
"name": "projects/MY-PROJECT/databases/(default)/documents/mycollection/mydoc",
"fields": {
"stores": {
"mapValue": {
"fields": {
"abc": {
"mapValue": {
"fields": {
"name": {
"stringValue": "Store 1"
}
}
}
}
}
}
}
},
"createTime": "2019-04-20T19:14:01.792855Z",
"updateTime": "2019-04-22T18:07:05.660561Z"
}

Related

What is the reason fo this difference in Facebook messages webhook JSON scheme?

I'm reading the Facebook Send API and events docs, and I'm surprised to see some examples define the message at the top level of the JSON object hierarchy (here for "text message")
{
"sender":{
"id":"<PSID>"
},
"recipient":{
"id":"<PAGE_ID>"
},
"timestamp":1458692752478,
"message":{
"mid":"mid.1457764197618:41d102a3e1ae206a38",
"text":"hello, world!",
"quick_reply": {
"payload": "<DEVELOPER_DEFINED_PAYLOAD>"
}
}
}
While others examples have a different object structure:
{
"id": "682498302938465",
"time": 1518479195594,
"messaging": [
{
"sender": {
"id": "<PSID>"
},
"recipient": {
"id": "<PAGE_ID>"
},
"timestamp": 1518479195308,
"message": {
"mid": "mid.$cAAJdkrCd2ORnva8ErFhjGm0X_Q_c",
"attachments": [
{
"type": "<image|video|audio|file>",
"payload": {
"url": "<ATTACHMENT_URL>"
}
}
]
}
}
]
}
Here is seems like the object is the first example is instead contained in a messaging array. I can't find either what id corresponds to in this second example : there is no table detailing the fields on why the hierarchy is different here.
Finally, there is another example with a different structure:
{
"object": "page",
"entry": [
{
"id": "<PAGE_ID>",
"time": 1583173667623,
"messaging": [
{
"sender": {
"id": "<PSID>"
},
"recipient": {
"id": "<PAGE_ID>"
},
"timestamp": 1583173666767,
"message": {
"mid": "m_toDnmD...",
"text": "This is where I want to go: https:\/\/youtu.be\/bbo_fZAjIhg",
"attachments": [
{
"type": "fallback",
"payload": {
"url": "<ATTACHMENT_URL >",
"title": "TAHITI - Heaven on Earth"
}
}
]
}
}
]
}
]
}
Are these differences documented elsewhere? Why do they exist in the first place? Why is messaging an array? Can it contain multiple messages at once?

Atlas Search Autocomplete on Subdocument Array

I am having some trouble with the autocomplete atlas search data type when trying to define an index for an array of subdocuments in my document.
My data structure for the documents in my collection looks like this:
{
"data": {
"equipment": {
"entries": [
{
"name": "abcdefg"
}
{
"name": "hijklmno"
}
]
}
}
}
When I define a string index for searching the entries array, it works as intended and I get logical results. Here is my index definition using the lucene.keyword analyzer:
{
"mappings": {
"dynamic": false,
"fields": {
"data": {
"fields": {
"equipment": {
"fields": {
"entries": {
"fields": {
"name": {
"analyzer": "lucene.keyword",
"searchAnalyzer": "lucene.keyword",
"type": "string"
}
},
"type": "document"
}
},
"type": "document"
}
},
"type": "document"
}
}
}
}
However, when I try the same thing with the autocomplete type, I get an empty result, but no error. Here is how I defined the autocomplete:
{
"mappings": {
"dynamic": false,
"fields": {
"data": {
"fields": {
"equipment": {
"fields": {
"entries": {
"fields": {
"name": {
"tokenization": "nGram",
"type": "autocomplete"
}
},
"type": "document"
}
},
"type": "document"
}
},
"type": "document"
}
}
}
}
The documentation for Atlas Search states the following:
The autocomplete type can't be used to index fields whose value is an array of strings. So either this sentence has to be changed to say all kinds of arrays or I am doing something wrong here. Can someone clarify if this is even possible?
Thanks in Advance
Your syntax is completely wrong. its would be like:
{
"mappings": {
"dynamic": false,
"fields": {
"data.equipment.entries.name": [
{
"type": "autocomplete",
"tokenization": "nGram",
"minGrams": 3,
"maxGrams": 7,
}
]
}
}
}
But I am not sure that,if it support on array of document, But let me know if your problem is solved.

Multifield wildcard search in ElasticSearch

Consider this very basic T-SQL query:
select * from Users
where FirstName like '%dm0e776467#mail.com%'
or LastName like '%dm0e776467#mail.com%'
or Email like '%dm0e776467#mail.com%'
How can I write this in Lucene?
I have tried the following:
The query way (does not work at all, no results):
{
"query": {
"bool": {
"should": [
{
"wildcard": {
"firstName": "dm0e776467#mail.com"
}
},
{
"wildcard": {
"lastName": "dm0e776467#mail.com"
}
},
{
"wildcard": {
"email": "dm0e776467#mail.com"
}
}
]
}
}
}
The Multimatch way (returns anything where mail.com is present)
{
"query": {
"multi_match": {
"query": "dm0e776467#mail.com",
"fields": [
"firstName",
"lastName",
"email"
]
}
}
}
A third attempt (returns expected result, but if I only insert "mail", then no results are returned)
{
"query": {
"query_string": {
"query": ""dm0e776467#mail.com"",
"fields": [
"firstName",
"lastName",
"email"
],
"default_operator": "or",
"allow_leading_wildcard": true
}
}
}
It seems to me as there is no way to force Elasticsearch to force a query to use the input string as ONE substring?
The standard (default) analyzer will tokenize this email as follows:
GET _analyze
{
"text": "dm0e776467#mail.com",
"analyzer": "standard"
}
yielding
{
"tokens" : [
{
"token" : "dm0e776467",
...
},
{
"token" : "mail.com",
...
}
]
}
This explains why the multi-match works with any *mail.com suffix and why the wildcards are failing.
I suggest the following modifications to your mapping, inspired by this answer:
PUT users
{
"settings": {
"analysis": {
"filter": {
"email": {
"type": "pattern_capture",
"preserve_original": true,
"patterns": [
"([^#]+)",
"(\\p{L}+)",
"(\\d+)",
"#(.+)",
"([^-#]+)"
]
}
},
"analyzer": {
"email": {
"tokenizer": "uax_url_email",
"filter": [
"email",
"lowercase",
"unique"
]
}
}
}
},
"mappings": {
"properties": {
"email": {
"type": "text",
"analyzer": "email"
},
"firstName": {
"type": "text",
"fields": {
"as_email": {
"type": "text",
"analyzer": "email"
}
}
},
"lastName": {
"type": "text",
"fields": {
"as_email": {
"type": "text",
"analyzer": "email"
}
}
}
}
}
}
Note that I've used .as_email fields on your first- & lastName fields -- you may not want to force them to be mapped as emails by default.
Then after indexing a few samples:
POST _bulk
{"index":{"_index":"users","_type":"_doc"}}
{"firstName":"abc","lastName":"adm0e776467#mail.coms","email":"dm0e776467#mail.com"}
{"index":{"_index":"users","_type":"_doc"}}
{"firstName":"xyz","lastName":"opr","email":"dm0e776467#mail.com"}
{"index":{"_index":"users","_type":"_doc"}}
{"firstName":"zyx","lastName":"dm0e776467#mail.com","email":"qwe"}
{"index":{"_index":"users","_type":"_doc"}}
{"firstName":"abc","lastName":"efg","email":"ijk"}
the wildcards are working perfectly fine:
GET users/_search
{
"query": {
"bool": {
"should": [
{
"wildcard": {
"email": "dm0e776467#mail.com"
}
},
{
"wildcard": {
"lastName.as_email": "dm0e776467#mail.com"
}
},
{
"wildcard": {
"firstName.as_email": "dm0e776467#mail.com"
}
}
]
}
}
}
Do check how this tokenizer works under the hood to prevent 'surprising' query results:
GET users/_analyze
{
"text": "dm0e776467#mail.com",
"field": "email"
}

How to query an array of objects in Firestore using structuredQuery of REST API?

I am trying to fetch data from Firestore using REST API. I am not able to write a structuredQuery to query an array of objects.
The user data is present like:
{
"tokens": [{
"token": "123345",
"expiry": "2019-08-07T05:42:48.567Z"
},
{
"token": "345667",
"expiry": "2019-08-06T05:42:48.567Z"
}
],
"userName": "Jane"
}
{
"structuredQuery": {
"select":{
"fields":[{
"fieldPath":"userName"
}]
},
"where": {
"fieldFilter": {
"field": {
"fieldPath": "tokens.token"
},
"op": "EQUAL",
"value": {
"stringValue": "123345"
}
}
},
"from": [{
"collectionId": "userData"
}]
}
The expected result would be to get userName. But the actual result is:
[{ "readTime": "2019-08-06T07:00:48.624353Z" }]
Even though the data is present in the database, I am getting the above result.

Elasticsearch: Date Aggregation Most Recent

I have query that works. It aggregates data based on Id and finds the MOST RECENT object based on the created field. The problem I have is that I would like to find the SECOND MOST RECENT instead of MOST RECENT. How would I go about this? I have been looking all through the docs and all I can find is range which doesn't help me to much. Thank you :)
{
"query":{
"match": {
"name": "Robert"
}
},
"aggs": {
"previous": {
"terms": {
"field": "Id",
"order": {"timeCreated": "desc"}
},
"aggs": {
"timeCreated": {
"max": {"field": "created"}
}
}
}
}
}
Top_hits is what you are looking for. Use this:
{
"query":{
"match": {
"name": "A"
}
},
"aggs": {
"previous": {
"terms": {
"field": "Id"
},
"aggs": {
"latestRecords": {
"top_hits": {
"sort": {
"created": {
"order": "desc"
}
},
"size" :2
}
}
}
}
}
}