I have a row and in that we have a column called data. The below json is present in the data column. So I want to retrieve single object from json like name present inside the survey node.
{"build":{"version":"8.3.4.23"},"survey":{"database":"abc","id":952,"name":"ravi","server":"ABCD0045","group":"JKLMN"}}
I will provide survey and name as the input parameters and my output has to be ravi. is there any syntax to get this in azure data factory dataflow activity.
You can use Parse transformation in Data Flow to parse your data column.
Screenshot:
Then you can use this expression json.survey.name to get the value ravi.
Related
So, I am creating a Copy activity that reads from SQL Server table and have to send the data to an API end point with the PATCH request.
API provider specified that the body must be in the form of
"updates":[{"key1":"value1","key2":"value2","key3":"value3" },
{"key1":"value1","key2":"value2","key3":"value3" }, ...
.... {"key1":"value1","key2":"value2","key3":"value3" }]
However, my sql table maps to json this way (without the wrapper 'updates:')
[{"key1":"value1","key2":"value2","key3":"value3" },
{"key1":"value1","key2":"value2","key3":"value3" }, ...
.... {"key1":"value1","key2":"value2","key3":"value3" }]
I use the copy activity with the sink data set being of type Rest ..
How can we modify the mapping, so that schema gets wrapped by "updates" object ?
Using copy data activity, there might not be any possibility to wrap the data (array of objects) to an updates key.
To do this, I have used a lookup activity to get the data, set variable activity to wrap the data with an updates object key and finally, use Web activity with PATCH method and above variable value as body to complete the activity.
The following is the sample data I have taken for my SQL server table.
Use look up activity to select the data from this table using table or query option (I used query option). The debug output would be as follows:
NOTE: If your data is not same as in sample table I have taken, try using the query option so the output would be something as shown below
In the set variable activity, I have used an array variable and used the following dynamic content to wrap the above array of objects with updates key.
#array(json(concat('{"updates":',string(activity('Lookup1').output.value),'}')))
Now in the Web activity, choose all the necessary settings (PATCH method, authorizations, headers, URL, etc.,) and give the body as follows (I used a fake REST api as a demo):
#variables('tp')[0]
Since I am using the Fake REST API, the activity succeeds, but checking the Web activity debug input shows what is the body that is being passed to the Rest API. The following is an image for reference:
I am filtering 2 json files contains ids and picking up non existing ids.
results some output to filter with list of ids, these ids should be copied to other json or next activity to copy data from rest API using these ids.
I have reproed with sample JSON files. Below is the filter output.
Use the below expression to use the output value of filter activity in later activities or you can create a variable and store the value in that variable using set variable activity and use it later.
Here I created a string variable to store the output.
Expression:
#string(activity('Filter1').output.value)
Set variable output:
how to parse JSON data based on a id in flutter? I want to parse a specific data from a JSON file but the JSON file returns a list of data so I want to compare this JSON file Id to another variable to parse that specific data please help
you can use this link to generate Dart Classes for you JSON data and parse them as dart object.
After that you can get data with specific ID or any other business logic.
https://app.quicktype.io/
I am trying to do schema compliance of an input file in ADF. I have tried the below.
Get Metadata Activity
The schema validation that is available in source activity
But the above seems to only check if a particular field is present or not in the specified position. Also Azure by default takes the datatype of all these fields as string since the input is flat file.
I want to check the position and datatype as well. for eg:-
empid,name,salary
1,abc,10
2,def,20
3,ghi,50
xyz,jkl,10
The row with empid as xyz needs to be rejected as it is not of number data type. Any help is appreciated.
You can use data flow and create a filter to achieve this.
Below is my test:
1.create a source
2.create a filter and use this expression:regexMatch(empid,'(\\d+)')
3.Output:
Hope this can help you.
I have a Json file like so as a payload returned from an API call - which is an http dataset type in data factory.
{
"count": 2,
"name": "DatasetABC",
"columnNames": [
"Column_1",
"Column_2"
],
"rows": ["1234",
"5678"
]
}
I would like to be able to use the count records returned in an If condition. Im wondering what I need to use to get the value of "count" which is 2.
Any help appreciated.
Based on your description, i suppose you could use LookUp Activity in Azure Data Factory.
Lookup activity can retrieve a dataset from any of the Azure Data Factory-supported data sources. Use it in the following scenario:
Dynamically determine which objects to operate on in a subsequent
activity, instead of hard coding the object name. Some object examples
are files and tables. Lookup activity reads and returns the content of
a configuration file or table. It also returns the result of executing
a query or stored procedure. The output from Lookup activity can be
used in a subsequent copy or transformation activity if it's a
singleton value. The output can be used in a ForEach activity if it's
an array of attributes.
For example,maybe you could access the count value by using #{activity('MyLookupActivity').output.firstRow.count} in the IF activity.