In azure data factory, I am getting a single record back from a database.
I need to take one column from this and pass it to a web call body.
The body takes data in this format:
["cdd-lm-54"]
I have tried multiple expressions but none of them work. appreciate any advice on how to perform this.
The data returned from the database looks like this:
"value": [
{
"RowNumber": 1,
"Tag": "cdd-lm-54",
"Val1": "val 1",
"Val2": "val b",
"LastSyncDateTime": "2022-07-26T13:14:28Z",
"LastTimeModified": "2021-07-28T10:33:47.7Z"
}
]
The below expressions are the closest i have gotten, they output the data as i expect it to be but the web call still rejects it:
#concat('[','"',pipeline().parameters.DeviceRecord[0]['Tag'], '"',']')
#concat('[','''',pipeline().parameters.DeviceRecord[0]['Tag'], '''',']')
Odd thing is if i paste the exact value from ADF into postman, it works.
is adf doing something odd to the body?
Odd thing is if I paste the exact value from ADF into postman, it works.
The reason behind this is that the value that you take from the value and give it as ["cdd-lm-54"], it indicates that the body accepts an array containing required string.
Using #concat() to build ["cdd-lm-54"] will return a just a string, which is not the required data that the body accepts.
Instead use the following dynamic content
#array(pipeline().parameters.DeviceRecord[0]['Tag'])
The above returns an array containing the required value.
Related
i am using Copy Data activity(Can't use Web activity due to security reasons) to connect to Rest API(source) (via REST dataset)and I get a response in json format as shown below which is what i need, only that this is the first page and there are many other pages that i need to retrieve. The retrieved information is then saved in SQL database table (destination). In process to retrieve the next pages i need to extract the token from "continuationToken" and send it back to my REST dataset as an dynamic value. Not sure if this is the correct way but gave it a try.
[
{
"pagedResponse":[
{
"docType":"current",
"orgNumber":"98789765",
"persons":[
],
"clientId":43533,
"name":null,
"clientResponsible":{
"id":null,
"name":null
}
}
],
"continuationToken":{
"token":"-RID:Om1+ANeDbNMWASEAAAAAAAAA==#RT:1#TRC:10#ISrV:2#IEO:6554436",
"range":{
"min":"",
"max":"05C1DFFFFFFFFFFC"
}
}
}
]
I tried to run some test by saving the json response as a file. Created Lookup activity(Lookup1) to read in the json file and in the then created a for-each activity and in the items i added a dynamic content
#activity('Lookup1').output.value[0].continuationToken
but got this error message.:
{
"errorCode": "InvalidTemplate",
"message": "The function 'length' expects its parameter to be an
array or a string. The provided value is of type 'Object'.",
"failureType": "UserError",
"target": "ForEach1"
}
expected result was:
"token":"-RID:Om1+ANeDbNMWASEAAAAAAAAA==#RT:1#TRC:10#ISrV:2#IEO:6554436",
"range":{"min":"","max":"05C1DFFFFFFFFFFC"}
I think your solution mentioned in the question should work.I suppose that you want to know whether the continuationToken is null so that the loop should be ended,so you use length function.However,according to the error message:The function 'length' expects its parameter to be an array or a string. The provided value is of type 'Object'. It is stated in the document clearly:
Then continuationToken is an object,should be used with empty function:
I am using Databricks Resi API to create a job with notebook_task in an existing cluster and getting the job_id in return.
Then I am calling the run-now api to trigger the job.
In this step, I want to send a list as argument via the notebook_params, which throws an error saying "Expected non-array for field value".
Is there any way I can send a list as an argument to the job?
I have tried sending the list argument in base_params as well with same error.
user_json={
"name": job_name,
"existing_cluster_id": cluster_id,
"notebook_task": {
"notebook_path": notebook_path
},
"email_notifications":{
"on_failure":[email_id]
},
"max_retries": 0,
"timeout_seconds": 3600
}
response=requests.post('https://<databricks_uri>/2.0/jobs/create',headers=head,json=user_json,timeout=5, verify=False)
job_id=response.json()['job_id']
json_job={"job_id":job_id,"notebook_params":{"name":"john doe","my_list":my_list}}
response = requests.post('https://<databricks_uri>/2.0/jobs/run-now', headers=head, json=json_job, timeout=200, verify=False)
Not found any native solution yet, but my solution was to pass the list as a string and parse it back out on the other side:
json_job={"job_id":job_id,
"notebook_params":{
"name":"john doe",
"my_list":"spam,eggs"
}
}
Then in databricks:
my_list=dbutils.widgets.get("my_list")
my_list=my_list.split(",")
With appropriate care around special characters or e.g. conversion to numeric types.
If the objects in the list are more substantial, then sending them as a file to dbfs using the CLI or API before running the job may be another option to explore.
Hi may be I'm bit late but found a better solution.
Step 1:
Use JSON.stringyfy() in the console of any browser to convert your value(object, array, JSON etc) into string
Ex:
Now use this value in the body of URL
In Databricks notebook convert string to JSON using python json module.
Hope this helps
I'm trying to automate talend job executions using the Talend API but I'm getting an error when I try to pass the context params using the api.
The json I'm encoding to 64 is the following:
JSON='{ "actionName":"runTask", "authPass": "TalendPass", "authUser": "name#example.com", "jvmParams": [ "-Xmx256m" , "-Xms64m" ], "contextParams": ["host_mysql_db01": "failed", "database_analytics": "testing.it"],"mode": "synchronous", "taskId": 43}'
Error message:
{"error":"Expected a ',' or ']' at character 172","returnCode":2}
I found another stackoverflow issue Add context parameters to Talend job in Tac via API without actually running it but he doesn't say how he pass it and I cannot reply with a comment asking how he did it
The real talend api call is:
wget -O file http://localhost:8080/org.talend.administrator/metaServlet?$JSON_ENCODED
Can I get some help?
Actually, the json your are passing to the metaservlet is not valid json. You can check it with an online validator like http://jsonlint.com.
You are specifying the contextParams attribute as an array, but that syntax is not valid in json. An array can contain either a list of values (like jvmParams) or objects (which can themselves contain arrays). Here's an example.
Moreover, according to Talend reference, the attribute should be called "context" and must be an object instead of an array, like so:
"context":{"varname1": "varvalue", "varname2": "varvalue2"}
I am trying to use https://github.com/ibm-watson-iot/openwhisk-package-watsoniotp in an OpenWhisk sequence (containing two actions) all code is node.js
Testing the sequence using Postman. Once the action completes, the action returns the variable, payload. The variable payload is passed over to the next action in the sequence which is the openwhisk-package-watsoniotp (added via a binding in the IBM Cloud console so I am unable to modify this code, it is locked).
I can post data from postman into Watson IoT platform via the sequence. However the format of the payload is interpreted as a String, not a JSON string.
This is the body I post from Postman, one of the variants I have tried.
{"payload": "{'speed': 10}"}
My node.JS actions return the input, unmodified.
return {payload: params.payload};
The value should be a JSON string. However WIOTP is unable to interpret the payload and basically tokenizes the values. This is evident when I try to create a board and a card. The property list lets me select each value in the array.
enter image description here
The openwhisk-package-watsontiotp code as far as I can tell just takes, params.payload as is and passes it along.
I found an example in the code that answer the question,
The payload, should be nested. I missed that originally.
{
"key": "sampleInput",
"value": {
"eventType": "status",
"payload": {
"temp": 4
},
"domain": "messaging.internetofthings.ibmcloud.com",
"typeId": "xxxx",
"deviceId": "xxxx01"
}
}
I am using Rexster to load data into a TitanDB. When posting / putting vertices, I can provide properties as JSON in the request's body. If a property's value is a number, it will correspondingly be stored as a number and can be retrieved as such. For example, the following body will in a post message will create a property "score" of type number:
{
"score": 5
}
When POSTing / PUTing edges, though, it seems properties can only be provided as query parameters, e.g.:
POST .../graphs/graph/edges?_outV=256&_label=review&_inV=512&score=5
In this case, unfortunately, the 5 is always considered as a string: "5". Consequently, queries including numeric operations / comparisons do not work. For example, the following query will still return the posted edge (despite the posted score being 5):
v(256).outE('review').filter{it.getProperty('score')>9}
Is there a way to POST / PUT edges and their properties so that the number type is considered?
I was reasonably sure you could POST JSON to the edge route, but even if you can't, you can use Rexster's explicit type system to post your integer properly:
$ curl -X POST "http://localhost:8182/graphs/tinkergraph/edges?_outV=1&_inV=2&_label=knows&score=(i,5)"
{
"version":"2.7.0-SNAPSHOT",
"results": {
"score":5,"_id":"0","_type":"edge","_outV":"1","_inV":"2","_label":"knows"
},
"queryTime":31.79554
}