Extract ? number of JSON array objects into rows - postgresql

So I am trying to pivot some large Json arrays into a row by row format (postgres db). So imagine
{“searchResults”:[{“id”:“89705”,“score”:42,“title”:“Foo.”,“properties”:{…
Currently the most results we have in an array is about 300~ id's and explicitly saying;
Data::Json->'searchResults'->0->'tags'->0->>'label' as "Tag - Result 1",
...
Data::Json->'searchResults'->300->'tags'->0->>'label' as "Tag - Result
1",
Ideal Output
Array, ID , Score, Title
----
1 89705, 42, foo
1 89706, 34, bar
2 90003, 54, thing
2 98594, 53, that
(so 1,2 represent different rows in the initial table that both contain ??? amounts of objects the JSON data array)
Expanded JSON
{
"searchResults": [
{
"id": "897096",
"score": 42,
"title": "foo.",
"properties": {
"#type": "blah",
},
"publishedDate": "2018-06-30T10:20:20.555040Z",
"comments": [
{
"content": "",
"owner": {
"firstName": "",
"id": 0,
"lastName": ""
},
"id": 0,
"contentType": "",
"documentPk": 0,
"workflowStep": 0,
"order": 0
}
],
"tags": [
{
"tag": 783,
"label": "NO",
"iconClass": "",
"subGroup": "",
"exclude": false
},
{
"tag": 786,
"label": "Different name",
"iconClass": "",
"subGroup": "",
"exclude": false
}
],
"reviewTags": [
{
"tag": 2,
"label": "Accept",
"iconClass": "",
"subGroup": "",
"exclude": false
}
],
"original": {
..."names": [
{
"full_name": "This name"
}
],
"Entry Type": "Organization",
"Last Updated": "2018/05/03",
"Hit Category": "N/A",
"Aliases": [
"Olaj",
"hbhbhb"
]
},
"snippet": "",
"url": "",
"source": "_"
},
{
"id": "879057",
"score": 36,
"title": "name of company",
"properties": {
"#type": "",
"category": "SOE",
"type": "Organization",
"country": "Korea, Republic Of",
"subcategory": ""
},
"publishedDate": "2018-05-31T10:20:20.559714Z",
"comments": [
{
"content": "",
"owner": {
"firstName": "",
"id": 0,
"lastName": ""
},
"id": 0,
"contentType": "",
"documentPk": 0,
"workflowStep": 0,
"order": 0
}
],
"tags": [
{
"tag": 783,
"label": "NO",
"iconClass": "",
"subGroup": "",
"exclude": false
},
{
"tag": 786,
"label": "Different name",
"iconClass": "",
"subGroup": "",
"exclude": false
}
Any advise on what my options are here ?
Thanks #a_horse_with_no_name this worked perfectly.

it's unclear to me how the JSON continues in the array, but it seems you are looking for something like this:
with testdata(props) as (
values (
'{"searchResults":
[
{"id":"89705","score":42,"title":"Foo"},
{"id":"89706","score":34, "title":"bar"}
]}'::jsonb
)
)
select x.idx,
x.val ->> 'id' as id,
x.val ->> 'score' as score,
x.val ->> 'title' as title
from testdata, jsonb_array_elements(props -> 'searchResults') with ordinality as x (val, idx);
returns
idx | id | score | title
----+-------+-------+------
1 | 89705 | 42 | Foo
2 | 89706 | 34 | bar

Related

Mongoose - Joining based on Object name

Currently, I have 3 schemas that I want to join. The first schema is the user schema.
{
"_id": {
"$oid": "6147f87ac51f060e8c1bc8c7"
},
"userID": "344431410360090625",
"currency": 72590,
"level": 10,
"exp": 78.5,
"sp": 338,
"location": {
"area": 2,
"floor": 3
},
"inv": {
"Rag Hood#363": {
"emote": "",
"description": "",
"rarity": "Common",
"type": "equipment",
"image": "",
"equipmentType": "helmet",
"level": 20,
"ascension": 0,
"exp": 0,
"quantity": 0,
"expToLevelUp": 0,
"equipped": false
},
"Jericho Jehammad": {
"emote": "<:Jericho:823551572029603840>",
"description": "Enhance your weapons with this mysterious item",
"rarity": "Common",
"type": "special",
"image": "",
"quantity": 7147,
"listed": 6964
},
},
"__v": 0,}
I want to be able to use the Object names of "Rag Hood#363" and "Jericho Jehammad". Firstly the equipment schema is shown below.
{
"_id": {
"$oid": "61474cb047a1b66f2cb1b6d8"
},
"itemName": "Rag Hood",
"stats": {
"defense": {
"flat": 2,
"multi": 0
}
},
"equipmentType": "helmet",
"ascensionRequirements": [],
"statsUpPerLvl": {
"defense": 0.5
}}
Next, the items schema is used to join Jericho Jehammad. Equipment in our database is named with #, with the item number after. While other items are identified with just the itemName.
{
"_id": {
"$oid": "60c5c5e6d2d78c794d33b7ae"
},
"itemName": "Jericho Jehammad",
"emote": "<:Jericho:823551572029603840>",
"description": "",
"rarity": "Common",
"type": "special",
"image": ""}
I want to return an object that overrides the value if the value is present in the equipment and items schemas. If the value is not present, I will use the value present in the user schema.

Filter for one attribute (array) for one of its value (json)

Having the following record
{
"name": "
 Festões Plástico, 12mt x 17cm - Festas Populares",
"categories": [
"Festas",
"Casamentos",
"Decorações"
],
"hierarchicalCategories": {
"lvl0": "Festas",
"lvl1": "Festas > Casamentos",
"lvl2": "Festas > Casamentos > Decorações"
},
"description": "",
"brand": "Misterius",
"price": 14.94,
"stock": "Disponível",
"prices": [
{
"value": 12,
"type": "specificValue",
"family": "fatos",
"subfamily": "example"
},
{
"value": 13,
"type": "specificValue13",
"family": "fatos13",
"subfamily": "example13"
},
{
"value": 14,
"type": "specificValue14",
"family": "fatos14",
"subfamily": "example14"
},
{
"value": 15,
"type": "specificValue15",
"family": "fatos15",
"subfamily": "example15"
},
{
"value": 16,
"type": "specificValue16",
"family": "fatos16",
"subfamily": "example16"
}
],
"color": [
{
"name": "Amarelo",
"label": "Amarelo,#FFFF00",
"hexa": "#FFFF00"
},
{
"name": "Azul",
"label": "Azul,#0000FF",
"hexa": "#0000FF"
},
{
"name": "Branco",
"label": "Branco,#FFFFFF",
"hexa": "#FFFFFF"
},
{
"name": "Laranja",
"label": "Laranja,#FFA500",
"hexa": "#FFA500"
},
{
"name": "Verde Escuro",
"label": "Verde Escuro,#006400",
"hexa": "#006400"
},
{
"name": "Vermelho",
"label": "Vermelho,#FF0000",
"hexa": "#FF0000"
}
],
"specialcategorie": "",
"reference": "3546",
"rating": 0,
"free_shipping": false,
"popularity": 0,
"objectID": "30"
}
Now by searching for "Festas Populares" will return the record and its attributes, is it possible to also filter for one attribute array as "prices" to only return one json. for example "prices.type"="specificValue14" and "family"="fatos14" and "family"="fatos" and "subfamily"="example"
{
“value”: 14,
“type”: “specificValue14”,
“family”: “fatos14”,
“subfamily”: “example14”
}
the record return would be:
{
"name": "
 Festões Plástico, 12mt x 17cm - Festas Populares",
"categories": [
"Festas",
"Casamentos",
"Decorações"
],
"hierarchicalCategories": {
"lvl0": "Festas",
"lvl1": "Festas > Casamentos",
"lvl2": "Festas > Casamentos > Decorações"
},
"description": "",
"brand": "Misterius",
"price": 14.94,
"stock": "Disponível",
"prices": [
{
"value": 14,
"type": "specificValue14",
"family": "fatos14",
"subfamily": "example14"
}
],
"color": [
{
"name": "Amarelo",
"label": "Amarelo,#FFFF00",
"hexa": "#FFFF00"
},
{
"name": "Azul",
"label": "Azul,#0000FF",
"hexa": "#0000FF"
},
{
"name": "Branco",
"label": "Branco,#FFFFFF",
"hexa": "#FFFFFF"
},
{
"name": "Laranja",
"label": "Laranja,#FFA500",
"hexa": "#FFA500"
},
{
"name": "Verde Escuro",
"label": "Verde Escuro,#006400",
"hexa": "#006400"
},
{
"name": "Vermelho",
"label": "Vermelho,#FF0000",
"hexa": "#FF0000"
}
],
"specialcategorie": "",
"reference": "3546",
"rating": 0,
"free_shipping": false,
"popularity": 0,
"objectID": "30"
}
for some context a product can have multiple prices associated, for a specific user, or one day there is campaign giving discount, etc so for that cases want to filter price associated to the product/record.
No, this is not possible with Algolia. Records are always returned with the attributes specified inside attributesToRetrieve. These attributes are returned in full.

Grouping select with partial

Hello i have here this code:
public function getConversations($user, $limit, $page)
{
$offset = ($page - 1) * $limit;
$qb = $this->createQueryBuilder('message');
$qb
->select('message, partial o.{id}, partial userFrom.{id, username, avatar}, partial userTo.{id, username, avatar}, partial availability.{id, timeFrom}')
->leftJoin('message.from', 'userFrom')
->leftJoin('message.to', 'userTo')
->leftJoin('message.order', 'o')
->leftJoin('o.availabilities', 'availability')
->where($qb->expr()->orX(
$qb->expr()->eq('message.from', ':user'),
$qb->expr()->eq('message.to', ':user')
))
->setParameter('user', $user)
->setFirstResult($offset)
->setMaxResults($limit)
->addOrderBy('message.created', Criteria::DESC);
return $qb->getQuery()->getArrayResult();
}
I want to get only last message connected to each order. If i have 2 messages to order it gives me 2 rows but i need only one (the last one). I tried to use GROUP BY but it didn't work well. It wanted me to put all foreign ids to group by and then the result was really bad.
Result for this is:
{
"id": "813fa986-f89b-411d-8324-727e427cbc01",
"isRead": false,
"created": {
"date": "2016-12-12 18:01:00.000000",
"timezone_type": 3,
"timezone": "UTC"
},
"content": "Hello to you too",
"contentType": "text",
"from": {
"id": "3e1806d6-f47e-48b4-afdb-1081d9555a3f",
"avatar": "",
"username": "User two"
},
"to": {
"id": "52ead9fa-b498-400b-967d-b49886ab3118",
"avatar": "",
"username": "User one"
},
"order": {
"id": "8349e718-405a-4e88-8155-3896e9a89fdf",
"availabilities": [
{
"id": "9ddc07e0-be3d-453c-b12f-0ee31bfce444",
"timeFrom": {
"date": "2016-12-03 12:00:00.000000",
"timezone_type": 3,
"timezone": "UTC"
}
}
]
}
},
{
"id": "813fa986-f89b-411d-8324-727e427cbc00",
"isRead": false,
"created": {
"date": "2016-12-12 18:00:00.000000",
"timezone_type": 3,
"timezone": "UTC"
},
"content": "Helllooooooo",
"contentType": "text",
"from": {
"id": "52ead9fa-b498-400b-967d-b49886ab3118",
"avatar": "",
"username": "User one"
},
"to": {
"id": "3e1806d6-f47e-48b4-afdb-1081d9555a3f",
"avatar": "",
"username": "User two"
},
"order": {
"id": "8349e718-405a-4e88-8155-3896e9a89fdf",
"availabilities": [
{
"id": "9ddc07e0-be3d-453c-b12f-0ee31bfce444",
"timeFrom": {
"date": "2016-12-03 12:00:00.000000",
"timezone_type": 3,
"timezone": "UTC"
}
}
]
}
and should be just like this:
{
"id": "813fa986-f89b-411d-8324-727e427cbc01",
"isRead": false,
"created": {
"date": "2016-12-12 18:01:00.000000",
"timezone_type": 3,
"timezone": "UTC"
},
"content": "Hello to you too",
"contentType": "text",
"from": {
"id": "3e1806d6-f47e-48b4-afdb-1081d9555a3f",
"avatar": "",
"username": "User two"
},
"to": {
"id": "52ead9fa-b498-400b-967d-b49886ab3118",
"avatar": "",
"username": "User one"
},
"order": {
"id": "8349e718-405a-4e88-8155-3896e9a89fdf",
"availabilities": [
{
"id": "9ddc07e0-be3d-453c-b12f-0ee31bfce444",
"timeFrom": {
"date": "2016-12-03 12:00:00.000000",
"timezone_type": 3,
"timezone": "UTC"
}
}
]
}
}

How would I write a query to give me the following results?

I have 2 classes.
Activities and Comments.
Activities, is linked to comments.
Comments, has the fields username, comment and created_at
When I execute select from Activities, this is an example of my current results. Below, you will see an example of my desired results (Example 2)
Example 1
{
"#type": "d",
"#rid": "#28:38",
"#version": 10,
"#class": "Activities",
"activity_type_id": 1,
"username": "johnt",
"title": "random new activitie test",
"image": "http://www.someurl.com/images/blah-2.png",
"comments": [
"#24:26",
"#24:27",
"#24:28",
"#24:29",
"#24:30",
"#24:31"
],
"Activity_Properties": {
"comment_count": 0,
"share_count": 0,
"like_count": 0
}
}
my desired results are the following
Example 2
{
"#type": "d",
"#rid": "#28:38",
"#version": 10,
"#class": "Activities",
"activity_type_id": 1,
"username": "ultimaterandomuser",
"title": "random new activitie test",
"image": "http://www.someurl.com/images/blah.png",
"comments": [
{
"username": "randomuser1",
"comment": "randomuser1 comment"
"created_at": "2016-06-23 00:00:00",
},
{
"username": "randomuser2",
"comment": "randomuser2 comment"
"created_at": "2016-06-23 00:00:00",
},
{
"username": "randomuser3",
"comment": "randomuser3 comment"
"created_at": "2016-06-23 00:00:00",
}
],
"Activity_Properties": {
"comment_count": 0,
"share_count": 0,
"like_count": 0,
}
}
I tried your case with this simple dataset:
Output Query 1:
SELECT FROM Activities
{
"result": [
{
"#type": "d",
"#rid": "#12:0",
"#version": 1,
"#class": "Activities",
"activity_type_id": 1,
"username": "ultimaterandomuser",
"title": "random new activitie test",
"image": "http://www.someurl.com/images/blah.png",
"comments": [
"#13:0",
"#13:1",
"#13:2"
],
"Activity_Properties": {
"comment_count": 0,
"share_count": 0,
"like_count": 0
},
"#fieldTypes": "comments=n"
}
],
"notification": "Query executed in 0.01 sec. Returned 1 record(s)"
}
Output Query 2:
SELECT expand(#this.exclude('comments')) FROM (SELECT *, $comments as commentsList FROM Activities
LET $comments = (SELECT expand(#this.exclude('#rid')) FROM (SELECT username, comment, created_at FROM Comments WHERE #rid IN $parent.current.comments)))
{
"result": [
{
"#type": "d",
"#rid": "#12:0",
"#version": 1,
"#class": "Activities",
"activity_type_id": 1,
"username": "ultimaterandomuser",
"title": "random new activitie test",
"image": "http://www.someurl.com/images/blah.png",
"Activity_Properties": {
"comment_count": 0,
"share_count": 0,
"like_count": 0
},
"commentsList": [
{
"#type": "d",
"#version": 0,
"username": "randomuser1",
"comment": "randomuser1 comment",
"created_at": "2016-06-25 03:33:13",
"#fieldTypes": "created_at=t"
},
{
"#type": "d",
"#version": 0,
"username": "randomuser2",
"comment": "randomuser2 comment",
"created_at": "2016-06-25 03:33:13",
"#fieldTypes": "created_at=t"
},
{
"#type": "d",
"#version": 0,
"username": "randomuser3",
"comment": "randomuser3 comment",
"created_at": "2016-06-25 03:33:13",
"#fieldTypes": "created_at=t"
}
],
"#fieldTypes": "commentsList=z"
}
],
"notification": "Query executed in 0.041 sec. Returned 1 record(s)"
}
Hope it helps

Search in mongodb nested arrays

Here i have this database
{
"products": [
[
{
"title": "CS:GO:0",
"description": "500 x 6MM CARBON STEEL BALL BEARINGS",
"moreInfo": "SMK.",
"oldPrice": "269.99",
"newPrice": "109.99",
"offPrice": "59%",
"image": "",
"z-index": 0,
"shown": true,
"type": "product",
"category": "0",
"id": "0"
},
{
"title": "Grid autosport",
"description": "G&P G&P Tokyo Marui M870 Gas Charging...",
"moreInfo": "",
"oldPrice": "",
"newPrice": "129.99",
"offPrice": "",
"image": "",
"z-index": 1,
"shown": true,
"type": "product",
"category": "0",
"id": "1"
}
],
[
{
"title": "CS:GO:0",
"description": "500 x 6MM CARBON STEEL BALL BEARINGS",
"moreInfo": "SMK.",
"oldPrice": "269.99",
"newPrice": "109.99",
"offPrice": "59%",
"image": "",
"z-index": 0,
"shown": true,
"type": "product",
"category": "1",
"id": "0"
},
{
"title": "Grid autosport",
"description": "G&P G&P Tokyo Marui M870 Gas Charging...",
"moreInfo": "",
"oldPrice": "",
"newPrice": "129.99",
"offPrice": "",
"image": "",
"z-index": 1,
"shown": true,
"type": "product",
"category": "1",
"id": "1"
}
].........
I use this as a query to get the object i want
var query = {
"products": {
"$elemMatch": {
"category": '0',
"id": '0'
}
}
};
var secondaryQuery = {
'products.$': 1
};
collection.find(query, secondaryQuery)
The database returns null array [].
Does someone have idea why and how to get the object I want from it?