Variable value in the mail in script in ADF - azure-data-factory

I have a Adf flow , in which, i have stored the data in a array Variable(namely 'VariOutput'), that thing is present in a If Condition Activity.
and in IF condition False Activity , i have given the Web activity link. the code used is
'''{
"RunId":"Dummy",
"applicationName": "#{pipeline().parameters.AppName}",
"ErrorCode":"error",
"ErrorMessage":#{concat('File/Files not found-', string(variables('VariOutput')))},
}'''
But the webactivity is not processing the Array Variable VariOutput and throwing an error.
error :: error": {
"code": "InvalidRequestContent",
"message": "The request content is not valid and could not be deserialized: 'Unexpected character encountered while parsing value: F. Path 'ErrorMessage
ignore the line number 7 in the error , as i have deleted part of the error.

The request content is not valid and could not be deserialized: 'Unexpected character encountered while parsing value: F. Path 'ErrorMessage
I tried to repro this and got the same error.
This is because of double quotes in the variable. Replace the double quotes with single quotes and pass it in the web activity.
Give the below expression for error message
"ErrorMessage":"#{concat('File/Files not found-', replace(string(variables('VariOutput')),'"',''''))}"
When the pipeline is debugged, it is executed successfully, and mail is sent.

Related

Azure Data Factory LOOKUP possibilities

I'm trying to add a lookup activity that will look up a series of values(companyIds) and insert the values into here
["/apiCore/api/countries","/apiCore/api/Metrics/MyLookup"]
At present my configuration looks like the following:
I was wondering if it was possible to add Lookup activity to insert the values as follows:
And then enter a parameter like #activity('MyLookup').output.value to:
["/apiCore/api/countries","/apiCore/api/Metrics/MyLookup"] so it change to:
["/apiCore/api/countries","/apiCore/api/Metrics/** #activity('MyLookup').output.value**"]
Can someone let me know if the above would work? If not, do you have any suggestions?
I got the answer to my suggestion with the following error:
{
"errorCode": "2200",
"message": "ErrorCode=HttpRequestFailedWithClientError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Http request failed with client error, status code 400 BadRequest, please check your activity settings. If you configured a baseUrl that includes path, please make sure it ends with '/'.\nRequest URL: https://pm2.preqinsolutions.com/apiCore/api/Metrics/#activity('MyLookup').output.value.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (400) Bad Request.,Source=System,'",
"failureType": "UserError",
"target": "dynamoCompanies",
"details": []
}
You will notice the error is with:
#activity('MyLookup').output.value
here:
https://pm2.preqinsolutions.com/apiCore/api/Metrics/#activity('MyLookup').output.value.,
Any suggestions will be very much welcomed
Updated question
Using string interpolation, you can build the URL by adding the output of look up activity. Instead of using the URL as https://pm2.preqinsolutions.com/apiCore/api/Metrics/#activity('MyLookup').output.value, you can try the following:
I have taken a sample lookup which gives the following output:
Now, I have used a variable to build the URL value. I have taken the value /apiCore/api/Metrics/ in a parameter called relativeURL.
https://pm2.preqinsolutions.com#{pipeline().parameters.relativeUrl}#{activity('Lookup1').output.value}
This generates the required URL. When I call it in web activity, you can see the URL that is being passed in debug input (I don't have access to the activity fails).

fetch gives response Unexpected token ] in JSON

The JSON file is local, in public where it is accessible. The error references the text found in the JSON file where the error occurs.
Check for commas after the last item in arrays within the JSON file. Fetch doesn't like these, even if online JSON checkers give it a pass.
{"documents": [ {"important":"something"}, ]
^NO

Parameterize Parameter in API Body in dataflow

below is the image of the dynamic body in Web activity wherein I am leveraging a pipeline parameter
With the above set of configuration and values, the web activity is successfully getting executed.
Now for the same REST API, I am trying to use it as source in dataflow :
with the same aspect of configuration and it is failing with below error :
Dataflow param:
Error:
ailure to read most recent page request: DF-REST_001 - Error response from server: Some({"error":{"code":"DatasetExecuteQueriesError","pbi.error":{"code":"DatasetExecuteQueriesError","parameters":{},"details":[{"code":"DetailsMessage","detail":{"type":1,"value":"The JSON DDL request failed with the following error: Invalid JavaScript property identifier character: }. Path '', line 1, position 7.."}},{"code":"AnalysisServicesErrorCode","detail":{"type":1,"value":"3239182519"}}]}}}), Status code: 400. Please check your request url and body.
Can someone please help what I am doing Wrong?
To use dataflow parameter within the expression builder, we have to concatenate the parameter with our data. Look at the following demonstration.
I am using derived column to show how to use parameters within expression builder. Using the dynamic content same as yours, I get the following output:
'{"queries":[{"query":"{$query}"}],"serializerSettings":{"includeNulls":true}}'
String interpolation does not work in the dataflow expression builder. To get it correctly, you can concatenate the parameter using + to get desired result.
'{"queries":[{"query":"'+$query+'"}],"serializerSettings":{"includeNulls":true}}'
Using the following content in expression builder also works (using concat() function).
concat('{"queries":[{"query":"',$query,'"}],"serializerSettings":{"includeNulls":true}}')

How to reiterate json response to extract a token and to make a restapi call with for-each activity

i am using Copy Data activity(Can't use Web activity due to security reasons) to connect to Rest API(source) (via REST dataset)and I get a response in json format as shown below which is what i need, only that this is the first page and there are many other pages that i need to retrieve. The retrieved information is then saved in SQL database table (destination). In process to retrieve the next pages i need to extract the token from "continuationToken" and send it back to my REST dataset as an dynamic value. Not sure if this is the correct way but gave it a try.
[
{
"pagedResponse":[
{
"docType":"current",
"orgNumber":"98789765",
"persons":[
],
"clientId":43533,
"name":null,
"clientResponsible":{
"id":null,
"name":null
}
}
],
"continuationToken":{
"token":"-RID:Om1+ANeDbNMWASEAAAAAAAAA==#RT:1#TRC:10#ISrV:2#IEO:6554436",
"range":{
"min":"",
"max":"05C1DFFFFFFFFFFC"
}
}
}
]
I tried to run some test by saving the json response as a file. Created Lookup activity(Lookup1) to read in the json file and in the then created a for-each activity and in the items i added a dynamic content
#activity('Lookup1').output.value[0].continuationToken
but got this error message.:
{
"errorCode": "InvalidTemplate",
"message": "The function 'length' expects its parameter to be an
array or a string. The provided value is of type 'Object'.",
"failureType": "UserError",
"target": "ForEach1"
}
expected result was:
"token":"-RID:Om1+ANeDbNMWASEAAAAAAAAA==#RT:1#TRC:10#ISrV:2#IEO:6554436",
"range":{"min":"","max":"05C1DFFFFFFFFFFC"}
I think your solution mentioned in the question should work.I suppose that you want to know whether the continuationToken is null so that the loop should be ended,so you use length function.However,according to the error message:The function 'length' expects its parameter to be an array or a string. The provided value is of type 'Object'. It is stated in the document clearly:
Then continuationToken is an object,should be used with empty function:

Validate the data from a Json File

I got this error message while trying to execute the get method.
Validation failed: Values not equal for element '$..userName', expected 'admin' but was '["admin",null,"HiCitrus","Unisys1","dscsc"]'
This is the script I have used:
http().client(todoClient)
.send()
.get("/rest/api/user/allusers")
.contentType("application/json");
http().client(todoClient)
.receive()
.response(HttpStatus.OK)
.messageType(MessageType.JSON)
// .validate(path, controlValue);
.validate("$..userName","admin");"
There are multiple parameters available for the username - is there a way to validate if I find the username "Admin"? Tried with the code below, but it's throwing an error.
.validate("$..userName",contains("admin"));