Related
I have almost ready what I want to do, however the method that converts to a JSON object does not help me to solve what is missing. I want to get the same thing, but there will be more content inside "add" and inside "firsts" and so I need them to be arrays of objects.
My code:
case class FirstIdentity(docType: String, docNumber: String, pId: String)
case class SecondIdentity(firm: String, code: String, orgType: String,
orgNumber: String, typee: String, perms: Seq[String])
case class General(id: Int, pName: String, description: String, add: Seq[SecondIdentity],
delete: Seq[String], act: String, firsts: Seq[FirstIdentity])
val someDF = Seq(
("0010XR_TYPE_6","0010XR", "222222", "6", "TYPE", "77444478", "6", 123, 1, "PF 1", "name", "description",
Seq("PERM1", "PERM2"))
).toDF("firm", "code", "org_number", "org_type", "type", "doc_number",
"doc_type", "id", "p_id", "p_name", "name", "description", "perms")
someDF.createOrReplaceTempView("vw_test")
val filter = spark.sql("""
select
firm, code, org_number, org_type, type, doc_number,
doc_type, id, p_id, p_name, name, description, perms
from vw_test
""")
val group =
filter.rdd.map(x => {
(
x.getInt(x.fieldIndex("id")),
x.getString(x.fieldIndex("p_name")),
x.getString(x.fieldIndex("description")),
SecondIdentity(
x.getString(x.fieldIndex("firm")),
x.getString(x.fieldIndex("code")),
x.getString(x.fieldIndex("org_type")),
x.getString(x.fieldIndex("org_number")),
x.getString(x.fieldIndex("type")),
x.getSeq(x.fieldIndex("perms"))
),
"act",
FirstIdentity(
x.getString(x.fieldIndex("doc_number")),
x.getString(x.fieldIndex("doc_type")),
x.getInt(x.fieldIndex("p_id")).toString
)
)
})
.toDF("id", "name", "desc", "add", "actKey", "firsts")
.groupBy("id", "name", "desc", "add", "actKey", "firsts")
.agg(collect_list("add").as("null"))
.drop("null")
group.toJSON.show(false)
result:
{
"id": 123,
"name": "PF 1",
"desc": "description",
"add": {
"firm": "0010XR_TYPE_6",
"code": "0010XR",
"orgType": "6",
"orgNumber": "222222",
"typee": "TYPE",
"perms": [
"PERM1",
"PERM2"
]
},
"actKey": "act",
"firsts": {
"docType": "77444478",
"docNumber": "6",
"pId": "1"
}
}
I want to have an array of "add" and also of "firsts"
this:
EDIT
{
"id": 123,
"name": "PF 1",
"desc": "description",
"add": [ <----
{
"firm": "0010XR_TYPE_6",
"code": "0010XR",
"orgType": "6",
"orgNumber": "222222",
"typee": "TYPE",
"perms": [
"PERM1",
"PERM2"
]
},
{
"firm": "0010XR_TYPE_6",
"code": "0010XR",
"orgType": "5",
"orgNumber": "11111",
"typee": "TYPE2",
"perms": [
"PERM1",
"PERM2"
]
}
],
"actKey": "act",
"firsts": [ <----
{
"docType": "77444478",
"docNumber": "6",
"pId": "1"
},
{
"docType": "411133",
"docNumber": "6",
"pId": "2"
}
]
}
As per your comment, you want to aggregate add depending on some grouping. Please check what all columns you want to group by. The columns which you want to Agrregate cannot be part of grouping. That will never work, and will give you always separate records.
This will work as per your expectations (I suppose):
val group =
filter.rdd.map(x => {
(
x.getInt(x.fieldIndex("id")),
x.getString(x.fieldIndex("p_name")),
x.getString(x.fieldIndex("description")),
SecondIdentity(
x.getString(x.fieldIndex("firm")),
x.getString(x.fieldIndex("code")),
x.getString(x.fieldIndex("org_type")),
x.getString(x.fieldIndex("org_number")),
x.getString(x.fieldIndex("type")),
x.getSeq(x.fieldIndex("perms"))
),
"act",
FirstIdentity(
x.getString(x.fieldIndex("doc_number")),
x.getString(x.fieldIndex("doc_type")),
x.getInt(x.fieldIndex("p_id")).toString
)
)
})
.toDF("id", "name", "desc", "add", "actKey", "firsts")
.groupBy("id", "name", "desc", "actKey")
.agg(collect_list("add").as("null"))
.drop("null")
Result:
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|value |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|{"id":123,"name":"PF 1","desc":"description","actKey":"act","collect_list(add)":[{"firm":"0010XR_TYPE_6","code":"0010XR","orgType":"6","orgNumber":"222222","typee":"TYPE","perms":["PERM1","PERM2"]},{"firm":"0010XR_TYPE_5","code":"0010XR","orgType":"5","orgNumber":"222223","typee":"TYPE","perms":["PERM1","PERM2"]}]}|
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Inside your map function, you are not mapping the FirstEntity and SecondEntity as Seq hence the add is not getting converted to array.
Change your map function to this:
filter.rdd.map(x => {
(
x.getInt(x.fieldIndex("id")),
x.getString(x.fieldIndex("p_name")),
x.getString(x.fieldIndex("description")),
Seq(SecondIdentity(
x.getString(x.fieldIndex("firm")),
x.getString(x.fieldIndex("code")),
x.getString(x.fieldIndex("org_type")),
x.getString(x.fieldIndex("org_number")),
x.getString(x.fieldIndex("type")),
x.getSeq(x.fieldIndex("perms"))
)),
"act",
Seq(FirstIdentity(
x.getString(x.fieldIndex("doc_number")),
x.getString(x.fieldIndex("doc_type")),
x.getInt(x.fieldIndex("p_id")).toString
))
)
})
Will result into this:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|value |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|{"id":123,"name":"PF 1","desc":"description","add":[{"firm":"0010XR_TYPE_6","code":"0010XR","orgType":"6","orgNumber":"222222","typee":"TYPE","perms":["PERM1","PERM2"]}],"actKey":"act","firsts":[{"docType":"77444478","docNumber":"6","pId":"1"}]}|
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I am building a Cloudformation template for Media Live I have everything working with the AWS PHP SDK. I create a channel and return the Channels data to be used in the next call.
Return example.
(
[Arn] => arn:aws:mediapackage:us-east-1:000000000000:channels/000000000000
[Description] => Tests
[HlsIngest] => Array
(
[IngestEndpoints] => Array
(
[0] => Array
(
[Id] => 000000000000
[Password] => 000000000000
[Url] => https://000000000000.mediapackage.us-east-1.amazonaws.com/in/v2/1e803c424d2947f58c07d9a6a5ff3d31/000000000000/channel
[Username] => 000000000000
)
[1] => Array
(
[Id] => 000000000000
[Password] => 000000000000
[Url] => https://2bcff136c2fbf1e5.mediapackage.us-east-1.amazonaws.com/in/v2/000000000000/10392b8387fd442eae56f29ac2656837/channel
[Username] => 000000000000
)
)
)
[Id] => sdv
[Tags] => Array
(
)
)
I can then use these values in PHP like this.
$destinationOneUrl = $createChannel['HlsIngest']['IngestEndpoints'][0]['Url'];
Which works but with Cloudformation you have to use the Fn::GetAtt like this.
"Destinations": [{
"Id": "destination1",
"Settings": [{
"Url": {
"Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
},
"Username": {
"Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
},
"PasswordParam": {
"Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
},
},
{
"Url": {
"Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
},
"Username": {
"Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
},
"PasswordParam": {
"Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
},
}
]
}],
You cannot go into the array with this is there another function that can be used in combination with the Fn::GetAtt to get the inner values.
Something along these lines
"Fn::GetAtt": ["MediaPackageChannel", "HlsIngest['IngestEndpoints'][0]['Url']"]
Here is my full template its hard to explain but basically I need to attach my medialive channel to the mediapackage channel as I mention I have done with the php code above Cloudformation doesn't make this easy.
You will see the Destinations section line 39 I need to populate the Url, Username, PasswordParam with the returned values from the mediapackage channel creation.
https://gist.github.com/samueleastdev/03b050f937855a81f36bb1e3260aa5d3
How to create a json of a hierarchical structure in SQL with PostgreSQL 12?
I need to create a menu like the image below, this menu is already working, but the solution I am using is not very optimized and has a great overhead because I am making many accesses to the database in PHP code with foreach and recursion.
I would like a solution to access the database only once and get the JSON tree ready.
I made a SQL code that is not working very well because it is not merging menu items from the same parent
My current test implementation in SQL is failing, because it is repeating "Jubarte", it is not merging the items of "Jubarte":
CREATE TABLE menus
(
id bigserial NOT NULL PRIMARY KEY,
customer_id integer NOT NULL,
"idPai" bigint,
label text,
rota text
);
INSERT INTO menus(customer_id, "idPai", label, rota) VALUES
(1,NULL,'Jubarte', ''),
(2,1,'Cadastros', ''),
(3,NULL,'Ciente', ''),
(4,1,'Autorizações', ''),
(5,4,'Menus', ''),
(6,2,'Organograma', ''),
(7,1,'Minha Conta', '');
WITH RECURSIVE menus_tree("id", "customer_id", "idPai", "label", "rota", "children") AS (
-- tree leaves (no matching children)
(SELECT c.*, json '[]'
FROM menus c
WHERE NOT EXISTS(SELECT * FROM menus AS hypothetic_child WHERE hypothetic_child."idPai" = c.id)
)
UNION ALL
-- pozs's awesome "little hack"
SELECT (parent).*, json_agg(child) AS "children"
FROM (
SELECT parent, child
FROM menus_tree AS child
JOIN menus parent ON parent.id = child."idPai"
) branch
GROUP BY branch.parent
)
SELECT jsonb_pretty(json_agg(t)::jsonb)
FROM menus_tree t
LEFT JOIN menus AS hypothetic_parent ON(hypothetic_parent.id = t."idPai")
WHERE hypothetic_parent.id IS NULL;
Result:
[
{
"id": 3,
"rota": "",
"idPai": null,
"label": "Ciente",
"children": [
],
"customer_id": 3
},
{
"id": 1,
"rota": "",
"idPai": null,
"label": "Jubarte",
"children": [
{
"id": 7,
"rota": "",
"idPai": 1,
"label": "Minha Conta",
"children": [
],
"customer_id": 7
}
],
"customer_id": 1
},
{
"id": 1,
"rota": "",
"idPai": null,
"label": "Jubarte",
"children": [
{
"id": 2,
"rota": "",
"idPai": 1,
"label": "Cadastros",
"children": [
{
"id": 6,
"rota": "",
"idPai": 2,
"label": "Organograma",
"children": [
],
"customer_id": 6
}
],
"customer_id": 2
},
{
"id": 4,
"rota": "",
"idPai": 1,
"label": "Autorizações",
"children": [
{
"id": 5,
"rota": "",
"idPai": 4,
"label": "Menus",
"children": [
],
"customer_id": 5
}
],
"customer_id": 4
}
],
"customer_id": 1
}
]
I got a solution, I don’t know if it’s the best but it’s working like this, if someone has a better idea, please can suggest
WITH RECURSIVE location_with_level AS (
(
SELECT
menus.*,
0 AS level
FROM
menus,
usuarios u, permissoes P -- <== part to filter by user permissions
WHERE
"idPai" IS NULL
-- part to filter by user permissions
AND u."idPerfil" = P."idPerfil"
AND u."idSistema" = P."idSistema"
AND P."idMenu" = menus."id"
AND u."idPessoa" = 2
AND menus."ativo" = 't'
) UNION ALL
(
SELECT
child.*,
parent.level + 1
FROM
usuarios u, permissoes P, -- <== part to filter by user permissions
menus child
JOIN location_with_level parent ON parent.ID = child."idPai"
-- part to filter by user permissions
WHERE
u."idPerfil" = P."idPerfil"
AND u."idSistema" = P."idSistema"
AND P."idMenu" = child."id"
AND u."idPessoa" = 2
AND child."ativo" = 't'
)
),
-- pega o nivel
maxlvl AS ( SELECT MAX ( level ) maxlvl FROM location_with_level ),
-- generates the JSON of the menu items for each system module
c_tree AS (
(
SELECT location_with_level.*, NULL::JSONB nodes FROM location_with_level, maxlvl WHERE level = maxlvl )
UNION
(
(
SELECT
( branch_parent ).*,
jsonb_agg ( branch_child order by (branch_child->>'ordem')::numeric)
FROM
(
SELECT
branch_parent,
to_jsonb ( branch_child ) /*- 'level' - 'idPai' - 'id'*/ AS branch_child
FROM
location_with_level branch_parent
JOIN c_tree branch_child ON branch_child."idPai" = branch_parent.ID
) branch
GROUP BY
branch.branch_parent
) UNION
(
SELECT C
.*,
NULL :: JSONB
FROM
location_with_level C
WHERE
NOT EXISTS ( SELECT 1 FROM location_with_level hypothetical_child WHERE hypothetical_child."idPai" = C.ID )
)
)
)
SELECT array_to_json(
array_agg(
row_to_json(c_tree)::JSONB /*- 'level' - 'idPai' - 'id'*/ order by sistemas.ordem
)
)::JSONB
AS tree
FROM c_tree
JOIN sistemas on c_tree."idSistema" = sistemas.id
WHERE level=0
My Fulfilment End Point isnt receiving accessToken. Is this normal?
Becouse when im on the simulator it doesnt really ask to login to my authorization end point thus, it wont get a access token. My Oauth is working perfectly after tesing with Oauth Play Ground.
action.json
{
"accountLinking": {
"clientId": "", // SENSITIVE INFORMATION BLANK
"clientSecret": "", // SENSITIVE INFORMATION BLANK
"grantType": "AUTH_CODE",
"authenticationUrl": "", // SENSITIVE INFORMATION BLANK
"accessTokenUrl": "" // SENSITIVE INFORMATION BLANK
},
"actions": [{
"description": "",
"name": "MAIN",
"fulfillment": {
"conversationName": "PASS TEXT"
},
"intent": {
"name": "actions.intent.MAIN",
"trigger": {
"queryPatterns": [
"talk to APP NAME"
]
}
}
}],
"conversations": {
"PASS TEXT": {
"name": "PASS TEXT",
"url": "" // MY FULFILLMENT END POINT
"fulfillmentApiVersion": 2
}
}
}
Fulfilment Received Request
The request was logged into a text file
jsondata = file_get_contents('php://input');
$postdata = json_decode($jsondata, true);
$RAWfile = print_r($postdata, true);
file_put_contents('RAWfile.txt', $RAWfile);
RAWfile.txt
Array
(
[user] => Array
(
[userId] => APhe68HH0PP0nTYnY8jBJed31WqF
[locale] => en-US
)
[conversation] => Array
(
[conversationId] => 1500924702161
[type] => NEW
)
[inputs] => Array
(
[0] => Array
(
[intent] => actions.intent.MAIN
[rawInputs] => Array
(
[0] => Array
(
[inputType] => VOICE
[query] => ask APP NAME to bla bla
)
)
[arguments] => Array
(
[0] => Array
(
[name] => trigger_query
[rawText] => bla bla
[textValue] => bla bla
)
)
)
)
[surface] => Array
(
[capabilities] => Array
(
[0] => Array
(
[name] => actions.capability.AUDIO_OUTPUT
)
)
)
[device] => Array
(
)
[isInSandbox] => 1
)
Simulator Request
{
"query": "bla bla",
"accessToken": "ya29.Gl2TBLrbKjcgK-6jsARmc4Zvkx8qT2X2rE3vsuwEVc_Ey2-q4OUqNWaJPBlXzIjONb_u2MbrE-rgnfwQSZpbZReXsmZjoGOy18Tvp7xzzSJb-cW9SjZ32uLBaKO7vGE",
"expectUserResponse": true,
"conversationToken": "CiZDIzU5O...",
"surface": "GOOGLE_HOME",
"inputType": "VOICE",
"locale": "en-US",
"location": {
"address": "Googleplex, Mountain View, CA, United States",
"position": {
"lat": 37.421980615353675,
"lng": -122.08419799804688
},
"zipCode": "94043",
"city": "Mountain View"
},
"debugInfo": {
"assistantToAgentDebug": {
"assistantToAgentJson": {
"user": {
"userId": "bla",
"locale": "en-US"
},
"conversation": {
"conversationId": "1501004260313",
"type": "NEW"
},
"inputs": [
{
"intent": "actions.intent.MAIN",
"rawInputs": [
{
"inputType": "VOICE",
"query": "bla"
}
],
"arguments": [
{
"name": "trigger_query",
"rawText": "bla",
"textValue": "bla"
}
]
}
],
"surface": {
"capabilities": [
{
"name": "actions.capability.AUDIO_OUTPUT"
}
]
},
"device": {},
"isInSandbox": true
}
}
}
}
Simulator Response
{
"response": "Sure. Here's the test version of bla.\nsure\n",
"conversationToken": "bla",
"audioResponse": "//NExAASS...",
"debugInfo": {
"agentToAssistantDebug": {
"agentToAssistantJson": {
"conversationToken": "{\"state\":null,\"data\":{}}",
"expectUserResponse": false,
"finalResponse": {
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "sure",
"displayText": "sure"
}
}
],
"suggestions": []
}
}
}
}
}
}
Account linking is done through the Actions Console at https://console.actions.google.com/
From the "Overview" menu item, you'll need to get to step 5: Account Linking.
In there, you can set your endpoints, scopes required, etc.
Think you have to manually complete that part - if you look in the response, you'll see a URL - copy/paste and run that, it should complete the account linking process -
https://developers.google.com/actions/identity/account-linking
"In the Log area, copy and paste the URL from the debugInfo.sharedDebugInfo.debugInfo field into your web browser. You will be redirected to the authentication URL for your own server that was specified in API.AI or in the Actions Console."
I am currently developing an adaptive payment system using paypal however a 58001 error keeps occurring
Paypal Response:
{
"responseEnvelope": {
"timestamp": "2013-09-05T23:38:25.762-07:00",
"ack": "Failure",
"correlationId": "a72d1f30e37d4",
"build": "6941298"
},
"error": [
{
"errorId": "580001",
"domain": "PLATFORM",
"subdomain": "Application",
"severity": "Error",
"category": "Application",
"message": "Invalid request: {0}"
}
]
}
Paypal Request:
$createPacket = array(
"actionType" => "PAY",
"currencyCode " => "USD",
"receiverList" => array(
"reveiver" => array(
array(
"amount" => "1.00",
"email" => USER_EMAIL1
),
array(
"amount" => "3.00",
"email" => USER_EMAIL2
)
)
),
"returnUrl" => "http://localhost/",
"errorUrl" => "http://localhost/",
"cancelUrl" => "http://localhost/",
"requestEnvelope" => array(
"errorLanguage" => "en_US",
"detailLevel" => "ReturnAll"
)
);
And I can't figure out what's wrong. I have double checked everything.
I figured it out. I just had and extra space after currencyCode. It was "currencyCode " => "USD" when is should have been "currencyCode" => "USD" :)