How to use Restangular when the service for a collection returns a dictionary instead of an array? - rest

I'm in the early phases of the development of a client app to an existing REST service and I'm trying to decide what to use for server communication. So far I'm loving Restangular documentation, it seems really solid, but I'm worried it's not going to work with the service because the responses look something like this:
{
"0": {
"name": "John",
"total": 230,
"score": 13
},
"1": {
"name": "Sally",
"total": 190,
"score": 12
},
"2": {
"name": "Harry",
"total": 3,
"score": 0
},
"...": "..."
}
I can't find in the docs if something like this is supported or how am I supposed to handle this type of response. Has anyone tried? Any ideas?

I'm the creator of Restangular :).
You can use that response with Restangular. You need to use the responseInterceptor. I guess that your response looks like that when you're getting an array. So you need to do:
RestangularProvider.setListTypeIsArray(false)
RestangularProvider.setResponseExtractor(function(response, operation) {
// Only for lists
if (operation === 'getList') {
// Gets all the values of the property and created an array from them
return _.values(response)
}
return response;
});
With this, it's going to work :)
Please let me know if this worked out fr you

You could do this if if you attached a "then" statement to the promise that you return.
That is, instead of this:
return <RectangularPromise>;
you have this:
return <RectangularPromise>.then(function(response){return _.values(response);};
Essentially, it's the same as #mgonto's answer, but doing it this way means that you don't have to modify your whole Rectangular Service if you only need to do this for one endpoint. (Although, another way to avoid this is to create multiple instances of the service as outlined in the readme).

unfortunately listTypeIsArray is not described in readme, so I haven't tried this option
but I resolved similar problem as described here
https://github.com/mgonto/restangular/issues/100#issuecomment-24649851

RestangularProvider.setListTypeIsArray() has been deprecated.
If you are expecting an object then handle it within addResponseInterceptor
app.config(function(RestangularProvider) {
// add a response interceptor
RestangularProvider.addResponseInterceptor(function(data, operation, what, url, response, deferred) {
var extractedData;
// .. to look for getList operations
if (operation === "getList") {
// .. and handle the data and meta data
extractedData = data.data.data;
extractedData.meta = data.data.meta;
} else {
extractedData = data.data;
}
return extractedData;
}); });
If you are expecting string response then convert it to array.
app.config(function(RestangularProvider) {
RestangularProvider.addResponseInterceptor(function(data, operation, what, url, response, deferred) {
var newData = data;
if (angular.isString(data)) {
newData = [data]; //covert string to array
}
return newData;
}); });

Related

Document AI Contract Processor - batchProcessDocuments ignores fieldMask

My aim is to reduce the json file size, which contains the base64 image sections of the documents by default.
I am using the Document AI - Contract Processor in US region, nodejs SDK.
It is my understanding that setting fieldMask attribute in batchProcessDocuments request filters out the properties that will be in the resulting json.
I want to keep only the entities property.
Here are my call parameters:
const documentai = require('#google-cloud/documentai').v1;
const client = new documentai.DocumentProcessorServiceClient(options);
let params = {
"name": "projects/XXX/locations/us/processors/3e85a4841d13ce5",
"region": "us",
"inputDocuments": {
"gcsDocuments": {
"documents": [{
"mimeType": "application/pdf",
"gcsUri": "gs://bubble-bucket-XXX/files/CymbalContract.pdf"
}]
}
},
"documentOutputConfig": {
"gcsOutputConfig": {
"gcsUri": "gs://bubble-bucket-XXXX/ocr/"
},
"fieldMask": {
"paths": [
"entities"
]
}
}
};
client.batchProcessDocuments(params, function(error, operation) {
if (error) {
return reject(error);
}
return resolve({
"operationName": operation.name
});
});
However, the resulting json is still containing the full set of data.
Am I missing something here?
The auto-generated documentation for the Node.JS Client Library is a little hard to follow, but it looks like the fieldMask should be a member of the gcsOutputConfig instead of the documentOutputConfig. (I'm surprised the API didn't throw an error)
https://cloud.google.com/nodejs/docs/reference/documentai/latest/documentai/protos.google.cloud.documentai.v1.documentoutputconfig.gcsoutputconfig
The REST Docs are a little more clear
https://cloud.google.com/document-ai/docs/reference/rest/v1/DocumentOutputConfig#gcsoutputconfig
Note: For a REST API call and for other client libraries, the fieldMask is structured as a string (e.g. text,entities,pages.pageNumber)
I haven't tried this with the Node Client libraries before, but I'd recommend trying this as well if moving the parameter doesn't work on its own.
https://cloud.google.com/document-ai/docs/send-request#async-processor

How to edit a value (list of entries) from an api response to use in a request body in Gatling/Scala

I have an issue that I'm hoping someone can help me with. I'm pretty new to coding and Gatling, so I'm not sure how to proceed.
I'm using Gatling (with Scala) to create a performance test scenario that contains two API-calls.
GetInformation
SendInformation
I'm storing some of the values from the GetInformation response so I can use it in the body for the SendInformation request. The problem is that some information from the GetInformation response needs to be edited/removed before it is included in the body for SendInformation.
Extract of the GetInformation response:
{
"parameter": [
{
"name": "ResponseFromGetInfo",
"type": "document",
"total": 3,
"entry": [
{
"fullUrl": "urn:uuid:4ea859d0-daa4-4d2a-8fbc-1571cd7dfdb0",
"resource": {
"resourceType": "Composition"
}
},
{
"fullUrl": "urn:uuid:1b10ed79-333b-4838-93a5-a40d22508f0a",
"resource": {
"resourceType": "Practitioner"
}
},
{
"fullUrl": "urn:uuid:650b8e7a-2cfc-4b0b-a23b-a85d1bf782de",
"resource": {
"resourceType": "Dispense"
}
}
]
}
]
}
What I want is to store the list in "entry" and remove the entries with resourceType = "Dispense" so I can use it in the body for SendInformation.
It would have been ok if the entry list always had the same number of entries and order, but that is not the case. The number of entries can be several hundred and the order of entries varies. The number of entries are equal to the "total" value that is included in the GetInformation response.
I've thought about a few ways to solve it, but now I'm stuck. Some alternatives:
Extract the entire "entry" list using .check(jsonPath("$.parameter[0].entry").saveAs("entryList")) and then iterate through the list to remove the entries with resourceTypes = "Dispense".
But I don't know how to iterate over a value of type io.gatling.core.session.SessionAttribute, or if this is possible. It would have been nice if I could iterate over the entry list and check if parameter[0].entry[0].resourceType = "Dispense", and remove the entry if the statement is true.
I'm also considering If I can use StringBuilder in some way. Maybe if I check one entry at the time using .check(parameter[0].entry[X].resourceType != dispense, and if true then append it to a stringBuilder.
Does someone know how I can do this? Either by one of the alternatives that I listed, or in a different way? All help is appreciated :)
So maybe in the end it will look something like this:
val scn = scenario("getAndSendInformation")
.exec(http("getInformation")
.post("/Information/$getInformation")
.body(ElFileBody("bodies/getInformtion.json"))
// I can save total, så I know the total number of entries in the entry list
.check(jsonPath("$.parameter[0].total").saveAs("total"))
//Store entire entry list
.check(jsonPath("$.parameter[0].entry").saveAs("entryList"))
//Or store all entries separatly and check afterwards who have resourceType = "dispense"? Not sure how to do this..
.check(jsonPath("$.parameter[0].entry[0]").saveAs("entry_0"))
.check(jsonPath("$.parameter[0].entry[1]").saveAs("entry_1"))
//...
.check(jsonPath("$.parameter[0].entry[X]").saveAs("entry_X"))
)
//Alternativ 1
.repeat("${total}", "counter") {
exec(session => {
//Do some magic here
//Check if session("parameter[0]_entry[counter].resourceType") = "Dispense" {
// if yes, remove entry from entry list}
session})}
//Alternativ 2
val entryString = new StringBuilder("")
.repeat("${total}", "counter") {
exec(session => {
//Do some magic here
//Check if session("parameter[0]_entry[counter].resourceType") != "Dispense" {
// if yes, add to StringBuilder}
// entryString.append(session("parameter[0]_entry[counter]").as[String] + ", ")
session})}
.exec(http("sendInformation")
.post("/Information/$sendInformation")
.body(ElFileBody("bodies/sendInformationRequest.json")))
I'm pretty new to coding
I'm using Gatling (with Scala)
Gatling with Java would probably be an easier solution for you.
check(jsonPath("$.parameter[0].entry").saveAs("entryList"))
This is going to capture a String, not a list. In order to be able to iterate, you have to use ofXXX/ofType[], see https://gatling.io/docs/gatling/reference/current/core/check/#jsonpath
Then, in order to generate the next request's body, you could consider a templating engine such as PebbleBody (https://gatling.io/docs/gatling/reference/current/http/request/#pebblestringbody) or indeed use StringBody with a function with a StringBuilder.

REST API Multiple PUT or DELETE in one time

Greeting everyone, I have a datatable in my html page that I populated using REST API. I can create new row and also update or delete by selecting a row and clicking the edit or delete button.
But currently I am unable to delete update or delete multiple row at once due to url error,
e.g : PUT http://127.0.0.1:8000/dashboard/content_detail/5,7,9/ 404 (Not Found)
how can I split this this into several separate url with respective id when I update or delete.
e.g :
/dashboard/content_detail/5
/dashboard/content_detail/7
/dashboard/content_detail/9
Below is my code, any help is much appreciated thank you.
idSrc: 'id',
ajax: {
create: {
type: 'POST',
url: content_path,
data: function (content_data) {
var create_data = {};
$.each(content_data.data, function (id, value) {
create_data['name'] = value['name'];
create_data['description'] = value['description'];
create_data['category'] = value['category'];
});
return create_data;
},
success: function () {
content_table.api().ajax.reload();
}
},
edit: {
type: 'PUT',
url: '/dashboard/content_detail/_id_/',
data: function (content_data) {
var updated_data = {};
$.each(content_data.data, function (id, value) {
updated_data['description'] = value['description'];
updated_data['category'] = value['category'];
updated_data['name'] = value['name'];
});
return updated_data;
},
success: function () {
content_table.api().ajax.reload();
}
},
remove: {
type: 'DELETE',
url: '/dashboard/content_detail/_id_/',
data: function (content_data) {
var deleted_data = {};
$.each(content_data.data, function (id, value) {
deleted_data['id'] = id;
});
return deleted_data;
},
success: function () {
content_table.api().ajax.reload();
}
}
},
If you're going to allow the update of a large number of items at once, then PATCH might be your friend:
Looking at the RFC 6902 (which defines the Patch standard), from the client's perspective the API could be called like
PATCH /authors/{authorId}/book
[
{ "op": "replace", "path": "/dashboard/content_detail/5", "value": "test"},
{ "op": "remove", "path": "/dashboard/content_detail", "value": [ "7", "9" ]}
]
From a design perspective you don't want several ids in your url.
I would prefer single calls for each change, thinking in resources you only manipulate one at a time.
In case this is a perfomance issue, I recommend a special url marked with action or something simliar, to make clear this ist not REST.
In HTTP it is not required for information to only exist on a single resource. It is possible to have multiple resources that represent the same underlying data.
It's therefore not out of the question to create a resource that 'represents' a set of other resources that you wish to DELETE or PUT to.
I do agree that it might not be the most desirable. I think we tend to prefer having information only exist in a single part of tree, and I think we like to avoid situations where updating a resource effects a secondary resource's state. However, if you are looking for a strictly RESTful solution to solve this problem, I think it's the right way.
Therefore a url design such as:
/dashboard/content_detail/5,7,9/
Is not necessarily non-RESTful or goes against the HTTP protocol. The fact that you're getting a 404 on that URL currently has to do with your application framework, not the protocol (HTTP) or architecture (REST) of your API.
However, for cases such as these I feel I would personally be inclined to sometimes create a separate POST endpoint that, acting outside of REST like an RPC endpoint. Specifically for these types of batch requests.

Is it possible to extract a set of database rows with RestTemplate?

I am having difficulties getting multiple datasets out of my database with RestTemplate. I have many routines that extract a single row, with a format like:
IndicatorModel indicatorModel = restTemplate.getForObject(URL + id,
IndicatorModel.class);
and they work fine. However, if I try to extract a set of data, such as:
Map<String, List<S_ServiceCoreTypeModel>> coreTypesMap =
restTemplate.getForObject(URL + id, Map.class);
this returns values in a
Map<String, LinkedHashMap<>>
format. Is there an easy way to return a List<> or Set<> in the desired format?
Fundamentally the issue is that your Java object model does not match the structure of your json document. You are attempting to deserialize a single json element into a java List. Your JSON document looks like:
{
"serviceCoreTypes":[
{
"serviceCoreType":{
"name":"ALL",
"description":"All",
"dateCreated":"2016-06-23 14:46:32.09",
"dateModified":"2016-06-23 14:46:32.09",
"deleted":false,
"id":1
}
},
{
"serviceCoreType":{
"name":"HSI",
"description":"High-speed Internet",
"dateCreated":"2016-06-23 14:47:31.317",
"dateModified":"2016-06-23 14:47:31.317",
"deleted":false,
"id":2
}
}
]
}
But you cannot turn a serviceCoreTypes into a List, you can only turn a Json Array into a List. For instance if you removed the unnecessary wrapper elements from your json and your input document looked like:
[
{
"name": "ALL",
"description": "All",
"dateCreated": "2016-06-23 14:46:32.09",
"dateModified": "2016-06-23 14:46:32.09",
"deleted": false,
"id": 1
},
{
"name": "HSI",
"description": "High-speed Internet",
"dateCreated": "2016-06-23 14:47:31.317",
"dateModified": "2016-06-23 14:47:31.317",
"deleted": false,
"id": 2
}
]
You should be able to then deserialize THAT into a List< S_ServiceCoreTypeModel>. Alternately if you cannot change the json structure, you could create a Java object model that models the json document by creating some wrapper classes. Something like:
class ServiceCoreTypes {
List<ServiceCoreType> serviceCoreTypes;
...
}
class ServiceCoreTypeWrapper {
ServiceCoreType serviceCoreType;
...
}
class ServiceCoreType {
String name;
String description;
...
}
I'm assuming you don't actually mean database, but instead a restful service as you're using RestTemplate
The problem you're facing is that you want to get a Collection back, but the getForObject method can only take in a single type parameter and cannot figure out what the type of the returned collection is.
I'd encourage you to consider using RestTemplate.exchange(...)
which should allow you request for and receive back a collection type.
I have a solution that works, for now at least. I would prefer a solution such as the one proposed by Ben, where I can get the HTTP response body as a list of items in the format I chose, but at least here I can extract each individual item from the JSON node. The code:
S_ServiceCoreTypeModel endModel;
RestTemplate restTemplate = new RestTemplate();
JsonNode node = restTemplate.getForObject(URL, JsonNode.class);
JsonNode allNodes = node.get("serviceCoreTypes");
JsonNode oneNode = allNodes.get(1);
ObjectMapper objectMapper = new ObjectMapper();
endModel = objectMapper.readValue(oneNode.toString(), S_ServiceCoreTypeModel.class);
If anyone has thoughts on how to make Ben's solution work, I would love to hear it.

How to document response fields for an object as Map(HashMap)

I am doing documentation for a REST service returning an object like this:
Map<String, HashMap<Long, String>>
and i find no way to describe response fields for such object.
Let's have a look at my code.
The service:
#RequestMapping(value = "/data", method = RequestMethod.GET)
public Map<String, HashMap<Long, String>> getData()
{
Map<String, HashMap<Long, String>> list = dao.getData();
return list;
}
My unit-test-based documentation:
#Test
public void testData() throws Exception
{
TestUtils.beginTestLog(log, "testData");
RestDocumentationResultHandler document = document(SNIPPET_NAME_PATTERN ,preprocessResponse(prettyPrint()));
document.snippets(
// ,responseFields(
// fieldWithPath("key").description("key description").type("String"),
// fieldWithPath("value").description("value as a Hashmap").type("String"),
// fieldWithPath("value.key").description("value.key description").type("String"),
// fieldWithPath("value.value").description("value.value description").type("String"),
// )
String token = TestUtils.performLogin(mockMvc, "user", "password");
mockMvc
.perform(get(APP_BUILD_NAME + "/svc/data").contextPath(APP_BUILD_NAME)
.header("TOKEN", token)
)
.andExpect(status().is(200))
.andExpect(content().contentType("application/json;charset=UTF-8"))
.andExpect(jsonPath("$").isMap())
.andDo(document);
TestUtils.endTestLog(log, "testData");
}
As you can see the code for response fields is commented out since I haven't had any solution for it yet. I am working on that but i really appreciate your help. Thank you in advance.
Your JSON contains a huge number of different fields. There looks to be over 1000 different entries in the map. Each of those entries is itself a map with a single key-value pair. Those keys all appear to vary as well. Potentially, that gives you over 2000 fields to document:
cancel
cancel.56284
year
year.41685
segment_de_clientele
segment_de_clientele.120705
…
This structure is making it hard to document and is also a strong indicator that it will be hard to consume by clients. Ideally, you would restructure the JSON so that each entry has the same keys and it's only the values the vary from entry to entry. Something like this, for example:
{
"translations": [ {
"name": "cancel",
"id": 56284,
"text": "Exit"
}, {
"name": "year",
"id": 41685,
"text": "Year"
}, {
"name": "segment_de_clientele",
"id": 120705,
"text": "Client segment"
}]
}
This would mean that you only have a handful of fields to document:
translations[]
translations[].name
translations[].id
translations[].text
If that's not possible, then I'd stop trying to use the response fields snippet to document the structure of the response. Instead, you should describe its structure manually in your main Asciidoctor source file.
There are two options:
1> Changing MAP to LIST of objects so that the response fields can be described easily.
2> Putting description manually to index.adoc file.
In my case, I go for option 2 because I have to stick to MAP.