In azure data factory, I am getting a single record back from a database.
I need to take one column from this and pass it to a web call body.
The body takes data in this format:
["cdd-lm-54"]
I have tried multiple expressions but none of them work. appreciate any advice on how to perform this.
The data returned from the database looks like this:
"value": [
{
"RowNumber": 1,
"Tag": "cdd-lm-54",
"Val1": "val 1",
"Val2": "val b",
"LastSyncDateTime": "2022-07-26T13:14:28Z",
"LastTimeModified": "2021-07-28T10:33:47.7Z"
}
]
The below expressions are the closest i have gotten, they output the data as i expect it to be but the web call still rejects it:
#concat('[','"',pipeline().parameters.DeviceRecord[0]['Tag'], '"',']')
#concat('[','''',pipeline().parameters.DeviceRecord[0]['Tag'], '''',']')
Odd thing is if i paste the exact value from ADF into postman, it works.
is adf doing something odd to the body?
Odd thing is if I paste the exact value from ADF into postman, it works.
The reason behind this is that the value that you take from the value and give it as ["cdd-lm-54"], it indicates that the body accepts an array containing required string.
Using #concat() to build ["cdd-lm-54"] will return a just a string, which is not the required data that the body accepts.
Instead use the following dynamic content
#array(pipeline().parameters.DeviceRecord[0]['Tag'])
The above returns an array containing the required value.
I have done everything needed to setup webservices on my moodle 3.11 instance, including roles/capabilities/user. However sending a test request always gives {
"exception": "dml_missing_record_exception",
"errorcode": "invalidrecord",
"message": "Can't find data record in database table external_functions."
}
The URL to access it is of the format https:///moodle/webservice/rest/server.php?wsfunction=core_user_create_user&service=mymoodleusermanage&moodlewsrestformat=json&users[0][username]=ABC&users[0][firstname]=VPTest&users[0][lastname]=None&users[0][email]=mail#xxx.com&users[0][password]=xxxxx&users[0][auth]=manual&wstoken=xxxxxxxxxxxxxx
The service parameter is correctly set to the shortname of the service. Does the service have to be defined anywhere additionally apart from Site Administration->Server->Web Services->External Services->Custom Services
Thanks for any help that can be given
The answer is very simple - you are trying to call a non-existent webservice function (hence the error message about being unable to find the database record for the function in the external_functions database table).
If you look in the Moodle code: https://github.com/moodle/moodle/blob/master/lib/db/services.php#L1717 you will see that the function is called core_user_create_users - with an "s" at the end of it.
If you add that extra "s" into the URL parameters you are using, then it should work.
https:///moodle/webservice/rest/server.php?wsfunction=core_user_create_user&service=mymoodleusermanage&moodlewsrestformat=json&users[0][username]=ABC&users[0][firstname]=VPTest&users[0][lastname]=None&users[0][email]=mail#xxx.com&users[0][password]=xxxxx&users[0][auth]=manual&wstoken=xxxxxxxxxxxxxx
you must change username all character small letter [username]=ABC like this [username]=abc and add s wsfunction=core_user_create_users
i am using Copy Data activity(Can't use Web activity due to security reasons) to connect to Rest API(source) (via REST dataset)and I get a response in json format as shown below which is what i need, only that this is the first page and there are many other pages that i need to retrieve. The retrieved information is then saved in SQL database table (destination). In process to retrieve the next pages i need to extract the token from "continuationToken" and send it back to my REST dataset as an dynamic value. Not sure if this is the correct way but gave it a try.
[
{
"pagedResponse":[
{
"docType":"current",
"orgNumber":"98789765",
"persons":[
],
"clientId":43533,
"name":null,
"clientResponsible":{
"id":null,
"name":null
}
}
],
"continuationToken":{
"token":"-RID:Om1+ANeDbNMWASEAAAAAAAAA==#RT:1#TRC:10#ISrV:2#IEO:6554436",
"range":{
"min":"",
"max":"05C1DFFFFFFFFFFC"
}
}
}
]
I tried to run some test by saving the json response as a file. Created Lookup activity(Lookup1) to read in the json file and in the then created a for-each activity and in the items i added a dynamic content
#activity('Lookup1').output.value[0].continuationToken
but got this error message.:
{
"errorCode": "InvalidTemplate",
"message": "The function 'length' expects its parameter to be an
array or a string. The provided value is of type 'Object'.",
"failureType": "UserError",
"target": "ForEach1"
}
expected result was:
"token":"-RID:Om1+ANeDbNMWASEAAAAAAAAA==#RT:1#TRC:10#ISrV:2#IEO:6554436",
"range":{"min":"","max":"05C1DFFFFFFFFFFC"}
I think your solution mentioned in the question should work.I suppose that you want to know whether the continuationToken is null so that the loop should be ended,so you use length function.However,according to the error message:The function 'length' expects its parameter to be an array or a string. The provided value is of type 'Object'. It is stated in the document clearly:
Then continuationToken is an object,should be used with empty function:
I'm testing out a few things in the OAuth 2.0 Playground and trying to get data in and out of Google Fit using their REST API
I have done this previously with success, I just didn't write down what I did.. now I've come back to make it a proper thing and can't get it working again.
I have access to Google Fit datasources via the dashboard. I can get a list of the dataSources that exist from:
https://www.googleapis.com/fitness/v1/users/me/dataSources
And that is successful. I have also created my own stream which has a single floating point weight value on it called
raw:com.google.weight:b6ac18c0:dten.sync
It already has data in it, I put it there last time I used it. I can select all that data by requesting a GET on the following
https://www.googleapis.com/fitness/v1/users/me/dataSources/raw:com.google.weight:b6ac18c0:dten.sync/datasets/0-1432193482000000000
It returns me all the data points I entered last time as JSON
I then try to PATCH the data adding my own data to the folliwng URL
https://www.googleapis.com/fitness/v1/users/me/dataSources/raw:com.google.weight:b6ac18c0:dten.sync/datasets/1432193482000000000-1432193482000000000
With this as a the request body
{
"minStartTimeNs": "1421912895000000000",
"maxEndTimeNs": "1432193482000000000",
"dataSourceId": "raw:com.google.weight:b6ac18c0:dten.sync",
"point": [
{
"startTimeNanos": "1421912895000000000",
"modifiedTimeMillis": "1421912895000",
"endTimeNanos": "1421912895000000000",
"value": [
{
"fPVal": 89.1
}
],
"dataTypeName": "com.google.weight"
}
]
}
But I get back
{
"error": {
"code": 400,
"message": "Unable to fetch DataSource for Dataset: raw:com.google.weight:b6ac18c0:dten.sync",
"errors": [
{
"domain": "global",
"message": "Unable to fetch DataSource for Dataset: raw:com.google.weight:b6ac18c0:dten.sync",
"reason": "invalidArgument"
}
]
}
}
I can't find any one referencing a similar anywhere soo I'm here
Also note if I miss spell my source it tells me off because they don't match the URL, if i include an empty list of data points I get the same error. I'm quite lost so I'm throwing it out there to see if anyone knows what that means
Thanks in advance
edit: i tried changing the hex code for my project's integer code and got an error about untrusted source. so i tried making a new test data source which works as expected. Slightly annoyed but guess I'll just start over..
OK I was stupid and didn't set up my own credentials in the OAuth settings in top right of the dashboard as it said to here. I forgot that bit -_- now I can access my own stream again and it shows my integer project id in the stream id not the hex one
https://developers.google.com/fit/rest/v1/get-started
Now I get invalid argument, but.. whatever >_<
edit 2:
invalid argument was because I have fPVal instead of fpVal and modifiedTimeMillis mills is not supposed to be submitted, obviously
Trying to get going on breeze but encountering the worst kind of error, which is none at all. It appears the metadata that I am producing is not being accepted by breeze. I know currently there are some issues with the metadata, such as 'foreignKeyNamesOnServer' has incorrect values in it and a bunch of others. The metadata I am producing can be viewed here (too large):
http://pastebin.com/ycP4jXxn
var serviceName = 'http://www.dockyard.com:8080/rest';
var entityManager = new breeze.EntityManager({serviceName: serviceName});
var entityQuery = new breeze.EntityQuery();
var query = breeze.EntityQuery.from("application")
entityManager.executeQuery(query)
.then(function (data) {
console.log(data);
}, function (error) {
console.log(error);
});
So the behaviour I am seeing is no javascript errors related to metadata parsing, the metadata is returning ok with 200 OK. The hit to /rest/application is returning 200 OK with the following data.
[{"#id":1,"id":1,"name":"dsad","deploymentStrategies":null,"versions":null,"groups":null},{"#id":2,"id":2,"name":"sss","deploymentStrategies":null,"versions":null,"groups":null},{"#id":3,"id":3,"name":"fdsfs","deploymentStrategies":null,"versions":null,"groups":null},{"#id":4,"id":4,"name":"fdsa","deploymentStrategies":null,"versions":null,"groups":null},{"#id":5,"id":5,"name":"dasda","deploymentStrategies":null,"versions":null,"groups":null}]
Promise is calling the error callback with: cannot execute _executeQueryCore until metadataStore is populated
The contents of the metadata store:
{"namingConvention":{"name":"camelCase"},"localQueryComparisonOptions":{"name":"caseInsensitiveSQL","isCaseSensitive":false,"usesSql92CompliantStringComparison":true},"dataServices":[{"serviceName":"http://www.dockyard.com:8080/rest/","hasServerMetadata":true,"jsonResultsAdapter":"webApi_default","useJsonp":false}],"_resourceEntityTypeMap":{"platform":"Platform:#com.psidox.dockyard.controller.model.dockyard","application":"Application:#com.psidox.dockyard.controller.model.application","host":"Host:#com.psidox.dockyard.controller.model.host","groupdeploymentstrategy":"GroupDeploymentStrategy:#com.psidox.dockyard.controller.model.application","dockyard":"Dockyard:#com.psidox.dockyard.controller.model.dockyard","configurationentry":"ConfigurationEntry:#com.psidox.dockyard.controller.model","hoststrategy":"HostStrategy:#com.psidox.dockyard.controller.model.application","dockerimage":"DockerImage:#com.psidox.dockyard.controller.model.docker","version":"Version:#com.psidox.dockyard.controller.model.application","docker":"Docker:#com.psidox.dockyard.controller.model.docker","hostproviderconfig":"HostProviderConfig:#com.psidox.dockyard.controller.model.host","hostprovider":"HostProvider:#com.psidox.dockyard.controller.model.host","metadataimpl":"MetadataImpl:#com.psidox.dockyard.controller.model","deployment":"Deployment:#com.psidox.dockyard.controller.model.dockyard","hosttype":"HostType:#com.psidox.dockyard.controller.model.host","group":"Group:#com.psidox.dockyard.controller.model.application","groupimplementation":"GroupImplementation:#com.psidox.dockyard.controller.model.application","deploymentstrategy":"DeploymentStrategy:#com.psidox.dockyard.controller.model.dockyard","groupdeployment":"GroupDeployment:#com.psidox.dockyard.controller.model.application","metadata":"Metadata:#com.psidox.dockyard.controller.model"},"_structuralTypeMap":{},"_shortNameMap":{},"_ctorRegistry":{},"_incompleteTypeMap":{},"_incompleteComplexTypeMap":{},"_id":0,"_deferredTypes":{}}"
I am pretty sure this error is related to Metadata store not being populated correctly from my metadata. Just wondering why Breeze is not throwing any type of error when it is encountering invalid metadata?
Edit:
After debugging the parse metadata call it appears that Breeze Metadata Schema Documentation is out of date. At a quick glance this is what it looks like has changed:
Key name "structuralTypeMap" has changed to "structuralTypes".
"structuralTypeMap" use to be a object with the key as the EntityTypeName and value was the Entity definition. Now it appears that "structuralTypes" is an array with the Entity definitions.
Suggestions also there should possibly be an exception thrown if the metadata doesn't contain any structuralTypes? Currently it is failing silently which isn't very helpful for debugging.
I fear they you've jumped into the deep end of the pool before learning to swim. I admire your bravery but I'm not surprised that you're struggling to stay afloat. You're not following any of the easy paths we've set out for you. I assume that is because none of these paths are suitable to your situation.
On the bright side, you've reinforced by sense that we soon must make it easier for developers who get their data from a custom REST service.
Problem #1
The Query results do not identify the EntityType and you didn't mention that you wrote a custom JsonResultsAdapter to cope with that. Your question and your MetadataStore contents below suggest that you are using the out-of-the-box Web API adapter which wouldn't know what to do with the JSON query results.
Here is one item in the JSON payload from your query, reformatted for readability
{
"#id": 1,
"id": 1,
"name": "dsad",
"deploymentStrategies": null,
"versions": null,
"groups": null
}
There's nothing in there to indicate to which EntityType this data belongs. Just looking I have no idea what type this is. Breeze won't know either.
You'll need to learn about the "JsonResultsAdapter" which is how Breeze interprets JSON data arriving from the server and maps it into instances of EntityTypes.
The Ruby Sample has a custom JsonResultsAdapter. It depends upon the fact that the server is explicit about the type of each object it returns; see how every Rails view adds a $type node (for example, the sessions:index view). This is the approach to take if you can control what the server sends the client.
The Edmunds Sample has a custom JsonResultsAdapter that has to infer the type by examining the characteristics of the JSON data. It's kind of a forensic exercise that you only want to indulge if you have to do so.
Problem #2
The MetadataStore you serialized is empty of all type information. Here it is reformatted for legibility
{
"namingConvention": {
"name": "camelCase"
},
"localQueryComparisonOptions": {
"name": "caseInsensitiveSQL",
"isCaseSensitive": false,
"usesSql92CompliantStringComparison": true
},
"dataServices": [
{
"serviceName": "http:\/\/www.dockyard.com:8080\/rest\/",
"hasServerMetadata": true,
"jsonResultsAdapter": "webApi_default",
"useJsonp": false
}
],
"_resourceEntityTypeMap": {
"platform": "Platform:#com.psidox.dockyard.controller.model.dockyard",
"application": "Application:#com.psidox.dockyard.controller.model.application",
... a bunch more ...
},
"_structuralTypeMap": {
},
"_shortNameMap": {
},
... more emptiness ...
}
}
I'm not really surprised, having discovered problem #3.
Problem #3
Your raw metadata doesn't match a format that Breeze understands. It looks like you cobbled it together by hand. It sure doesn't look like anything I recognize. It doesn't match the CSDL format from Entity Framework. It doesn't match the "Breeze Metadata Format" that you'd see when you exported a MetadataStore.
It's in trouble almost immediately. Here is how you start the definition of your first type:
"structuralTypeMap": {
"Group:#com.psidox.dockyard.controller.model.application": {
"shortName": "Group",
"namespace": "com.psidox.dockyard.controller.model.application",
Here is how it should begin:
"structuralTypes": [
{
"shortName": "Group",
"namespace": "com.psidox.dockyard.controller.model.application",
I accept your point that the Breeze Metadata Schema Documentation is incorrect. We should fix that.
I'm sympathetic with your argument that Breeze should have thrown an exception. I can see why it didn't throw. It simply ignored all the nodes that it didn't understand. A lot of parsers do that, not that that is a sufficient excuse.
In this case, it ignored the "structuralTypeMap" node and everything it had to say about types. When the parser was done, it had learned nothing at all about the types. Breeze can't know how many types you'll specify but it could act suspicious if there are none.
I confess I personally never thought to use this metadata schema description as my guide. That would be just about the hardest possible way to write metadata.
I suggest that you look at the documentation topic "Metadata by hand".
In sum
Please examine a simple example first. Maybe Edmunds. Maybe Ruby.
Learn to write metadata by hand; it's not hard.
Learn about the JsonResultsAdapter
We do hope soon to offer specific guidance for the developer who has a "vanilla" REST data service.